this space intentionally left blank

February 5, 2014

Filed under: tech»web


After a busy couple of weeks, Seattle went and won the Super Bowl, leading to the world's most polite celebration in our neighborhood:

There was another prize for the weekend: a friend of ours gifted us a Chromecast, which will be much appreciated since there's currently no way to watch HBO on the PS4. On Monday, Google released the public SDK for the platform, so I decided to poke around a bit.

Chromecast has a decidedly-odd way of loading content. The device itself is just a thin shell around a Chrome window, and it loads web pages like any other browser. But there's no keyboard of any kind, so how does it know which page to load? The answer is that each "app" has an ID listed with Google, corresponding to a set of URLs that the developer provides. When a mobile app or a computer running Chrome triggers the Chromecast, it sends the app ID, which the device then sends to Google and gets a URL in return (or, if the app hasn't been listed, it does nothing). From that point on, you can send messages to the page over via Google's cloud, and your page can do whatever you want it to do. Getting your pages linked to an application ID on the Chromecast lookup servers costs $5.

Five dollars is a low price, but it's more than I really want to pay for a glorified DNS. I'm a little dismayed by the restrictions on the open web — I'd like the option to just send a URL directly. I'm also holding out for a pure JavaScript API, instead of piggybacking on the Chrome extension. So I probably won't be writing any Chromecast apps any time soon. But it's certainly not for a lack of ideas. The interaction model that Chromecast uses — where the screen is just a dumb display, but it can receive commands from other web-accessible devices — is strikingly similar to Microsoft's SmartGlass model. And where Microsoft seems to see it as a way to create companion apps for XBox, I think it's interesting to think about how this "distributed I/O" model could be used for standalone applications.

  • The Chromecast isn't going to rival any consoles, but it wouldn't have to be for a lot of group gaming experiences. Just having a screen that could be used as a scoreboard, or a trivia question where phones are used as buzzers, would be a cool usage that doesn't require precise controls or rich graphics. Turn-based games could easily use the screen as a board overview, while letting people zoom in and move their pieces from their local touchscreen. It also provides an interesting split between public and private information for players that many video games (excepting the Wii U and Dreamcast) couldn't duplicate.
  • I love maps. I think they're the real face of augmented reality, as any regular traveler can attest these days. But they don't have to be mobile. A Chromecast could easily serve as a map up on your wall, updated with whatever information you find interesting. Maybe that's as simple as the weather, but imagine being able to tag it with RFID information or last-known positions for people in your household. Systems like Google Now, which learn from your schedule, could even post notifications for the buses that are coming or traffic problems that you're likely to face.
  • Along those same lines, a simple dashboard could be helpful for businesses and individuals. Being able to throw metrics up on the wall with a web browser is not a new thing, but tying it to a smart, feed-aware service would open up all kinds of new tricks, like being able to leave yourself notes via a hashtag on social networks. There's not really any input needed: it's just a passive display of whatever you want to keep yourself caught up on, in an easy at-a-glance format.
  • Finally, it's probably just all the public speaking I've been doing lately, but it's tempting to think that a presentation app for Chromecast would be super-helpful for speakers. A lot of times, when I go to a meetup or a new classroom, it's hard to predict what kind of video hookups the projector will have, assuming that they even have a projector. But many times, there will be a big-screen LCD TV, with a handy HDMI input. Being able to carry a Chromecast with me to make my presentations, especially if the speaker notes can be viewed separately, would be awesome.
When we talk about the web being device-agnostic, the Chromecast is a perfect example of what we're talking about. It's radically different from other web clients: low DPI on a big screen, no local input, and unpredictable performance. But that's the power of the platform — as a toolkit, its reach is unparalleled. And the restrictions prove to be exciting inspiration for new uses, just as touchscreens came with their own unique challenges and advantages. I don't know if Chromecast is going to be successful, but the hacks for it are going to be really interesting.

January 23, 2014

Filed under: random»personal

This Week ridiculously busy. Class at SCCC has ramped up, I've been prepping for the University of Washington workshop, and of course I've got my everyday work at ArenaNet as well. In lieu of a more substantial post, here are some quick notes about what's on my plate.

  • The homepage for the news apps workshop (now formally COM499B at UW) is located here. It's not terribly comprehensive yet, but my goal is to update it with my presentations, things that I mention during class, and in-class work as I go. In other words, it's not a textbook, it's a record that students can refer back to later. I'd welcome suggestions for other additions to it.
  • While uploading the COM499 page, I accidentally overwrote the root index.html on my portfolio, which led to A) a panicky few minutes finding the Google cache of my page and recovering it, and B) the realization that I had no way to recreate my portfolio page, now that it's not generated via Blosxom anymore. As Mike Bostock wrote in his ode to Make, automating any production process is important because it formalizes things, and makes them reproducible. After an hour's work or so, is still a static page, but it's built from a template and a Node-based recipe, so I'll never lose it entirely again. Once I've cleaned things up a little, I may open-source the tool for other ex-Blosxom users.
  • Caret will break 30,000 users today or tomorrow. This is, to be honest, mind-boggling to me. The last time I wrote about it here, it had 1,500 users. Since that time, I've added a ton of new functionality, accepted contributions from other developers who want to help add features, found out that members of the Chrome team are actively using it (although, I assume, not to write Chrome itself), and filed several bugs against the chrome.* APIs. Caret's my daily editor for class work and personal projects, and I'm incredibly pleased by how well it compares against Sublime and other native editors.
  • Dates have been locked in for Urban Artistry's Soul Society festival in DC this year: April 14-20, with the main event toward the end of the week as usual. We'll be adding specific event schedules in the next couple days, so keep an eye out.

January 16, 2014

Filed under: gaming»software

Field of Streams

At the Consumer Electronics Show, Sony showed off the fruits of their acquisition of Gaikai, a company that streams video games over the Internet. This is something with which I have a little experience: in 2012, I worked for Big Fish on the now-discontinued Big Fish Unlimited, which did the same thing but for casual games. I was pretty sure that it was doomed then, and I'm pretty sure that similar platforms — including Sony's Playstation Now and OnLive — are doomed now, and for the forseeable future. The reason is simple: streaming just doesn't scale.

Let's say you're playing a game by streaming it to yourself from another computer you own, as in Nvidia's Shield or Valve's SteamOS. To do this you need two boxes: one to actually run the game, and one to act as a thin client, providing the display and input support. There are lots of things that can cause problems here — network connectivity, slow hardware, latency — but at the very least you're always going to have enough hardware to run the game, because you own both ends of it. Streaming scales in a linear fashion.

Now pretend you're doing the same thing, but instead of running the host machine yourself, it lives in a remote datacenter. For each person who's playing, the streaming service needs another computer to run the game — the scaling is still linear. You can't cache a game on a cheap edge node like you can a regular file, because it's an interactive program. And there's no benefit to running all those games simultaneously, the way that Google can leverage millions of GMail customers to lower e-mail transmission costs and spam processing algorithms cheaper. No, you're stuck with a simple equation: n players = n servers. And those servers are not cheap: they need hefty graphics cards, local storage, cooling systems, sound cards, etc.

And it gets worse, because those players are not going to obligingly spread their playtime around the clock to keep the load constant. No, they're going to pile in every evening in much higher numbers — much higher — than any other time of the day. League of Legends, a single (albeit very popular) game has had more than 5 million concurrent players. During peak hours, streaming providers will struggle to run enough host boxes. During off hours, all that expensive hardware just sits idle, completely unused.

At first, these problems seem solvable. When you don't have a lot of customers, it's not that bad to add new hosts to compensate for growth. Those earlier players may be more forgiving of wait times, attributing them to growing pains. But consider the endgame: if something like Playstation Now achieves real, widespread success (despite all the other network latency and quality of service issues these services always face), Sony's ultimate scenario is having literally millions of rack-mounted PS4s in datacenters around the country, many of them running at peak capacity for hours on end. That's more servers than Google, Microsoft, and Facebook put together.

At Big Fish, the business argument was that casual games could be run with multiple applications to a machine, so the scaling pressure was lower. But it's still linear: you've lowered the total number of machines you might need in the long run, but there's still a direct relationship between that number and your number of players. The only way to scale online gaming gracefully is to either find a way that games can share state (i.e., MMOs), or offload more of it to the client. As a front-end JavaScript specialist, I always thought Big Fish would have better luck porting its games to the browser instead of streaming them to a Java applet.

But there's only so much you can move to the client when your selling point is "next-gen games without next-gen hardware." In the case of Sony and OnLive, no amount of browser wizardry or Playstation branding is going to solve that fundamental scaling problem. It may be workable for instant demos, or beta previews. But for mainstream gaming, without a miraculous breakthrough in the way games are built and executed, the math just isn't there.

January 10, 2014

Filed under: journalism»new_media

App-y New Year

At the end of January, I'll be teaching a workshop at the University of Washington on "news apps," thanks to an offer from the outgoing news app editor at the Seattle Times. It's a great opportunity, and a chance to revisit my more editorial skills. From the description:

This bootcamp will introduce students to the basic components of creating news applications, which are data-powered digital stories tied together through design, programming and journalism. We’ll walk through all the components of creating a news application, look at industry examples of what works and what doesn’t, and learn the basic coding skills required to build a news app.
Sounds cool, but it's still a wide-open field — "data-powered digital stories" covers a huge range of approaches. What do you even teach, and how do you do it in two 4-hour workshops?

It turns out that for almost any definition of "news app," there's an exception. NPR's presidential election board is a data-powered news app, but it's not interactive beyond an auto-update. Snow Fall is certainly a news app, but it's hard to call it "data-powered." How can we craft a category that includes these, but also includes traditional, data-oriented interactives like The Atlantic's Netflix Genre Generator and the Seattle Times mayoral race comparison? More importantly, how do we get young journalists to be able to think both expansively and productively about telling stories online?

That said, I think there is, actually, a unifying principle for news apps. In fact, I think it cuts to the heart of what draws me to web journalism, and the web in general. News apps are journalistic stories told via hypermedia — or, to put it simply, they have links.

A link seems like a small thing after years on the web, so it's good to revisit just how fundamentally groundbreaking they are. Links can support or subvert their anchor, creating new rhetorical devices of their own. At the most basic level, they contextualize a story. More abstractly, they create non-linearity: users explore a news app at their own pace and with their own priorities, rather than the direct stream of narrative from a text story.

A link is a simple starting place. But it starts us down a path of thinking about more complicated applications and usage. I'm fond of saying that an interactive visualization is constructed in many layers, with users peeling open the onion as far as they may want. If we're thinking in terms of other hypertext documents (a.k.a., the TV Tropes Rabbit Hole) from the start, we're already prepared when readers use similar interaction patterns to browse data-based interactives — either by shallowly skipping around, or diving in depth for a specific feature.

By reconceptualizing news apps as being hypermedia instead of a specific technology or group of technologies, such as mapping or graphing, introducing students to web storytelling gets a lot easier — particularly since I won't have time to teach them much beyond some basic HTML and CSS (in the first workshop) and a little scripting (in the second).

It also leaves them plenty of room to think creatively when presenting stories. I'd love for budding news app developers to be as interested in wikis and Twine as they are in D3 and PostGIS. Most importantly, I'd love for an appreciation of hypertext to leak into their writing in general, if only to reduce the number of print die-hards in newsrooms around the country. You don't have to end up a programmer to create new, interesting journalism that's really native to the web.

December 19, 2013

Filed under: movies»television»elementary

The Watson Problem

The difficulty in making a Sherlock Holmes adaption for American television is that we've already got three or four of them. Hyper-observant detectives are a dime a dozen, from The Mentalist to Monk. Arguably, Psych is just Holmes and his deductive skills with an added dose of arrested development (and I say that as someone who enjoys Psych at its fluffiest). Dule Hill's Gus even serves as a Watson, but reduced to a pharmeceutical rep instead of a doctor to match his detective friend's lack of ambition. The BBC's Sherlock owes Psych a debt for the visual style illustrating the deductive process, although I doubt they'd ever admit it.

I don't envy the people who decided, after the British version aired to wide acclaim, to make another Sherlock Holmes show. That's some tough competition. But I've been watching the first season of Elementary, and I have to say I'm enjoying it. The cast is growing on me, I like the lack of romantic angst, and the infrequent references to the original stories (inasmuch as I can catch them, not being a die-hard fan) are often worth a chuckle.

The biggest problem that Elementary faces is Watson — specifically, figuring out what she's supposed to bring to the team. As played by Lucy Liu, Joan Watson is an ex-surgeon who initially serves as Sherlock's live-in addiction counselor. With the terms of that job running out, partway through the first season, Sherlock offers her a position being groomed as a detective-in-training: someone who can take on his methods and become an equal part of the sleuthing consultancy.

Unfortunately, this is where the show's writers seem to have run out of steam. They know where they want this Watson to end up, and they've told us about it repeatedly, but they don't know how to get her there. She's not shown doing much studying, as such, and Holmes mentions that she doesn't read his research. As a result, Liu's Watson ends up either solving minor b-plot mysteries, dropping medical clues, or providing a convenient anchor toward which Sherlock can toss exposition. It's possible she's learning by osmosis, but this hardly provides a reason why we should care about her character arc.

It's interesting to see how the BBC Sherlock has taken a different tack with its version of the character. The British Watson, played by Martin Freeman, leans heavily on the actor's likeability and finely-tuned air of irritation to create a companion who partners with Sherlock for the adrenaline rush of it. Freeman's Watson is muscle and heart: he humanizes Sherlock and provides support. Ultimately, the relationship between the pair on the BBC show is one of friends. They enjoy going on adventures together. They have a similar restlessness. But Sherlock doesn't need Watson to solve crimes. When the show begins, he's doing relatively fine without him, although Watson's blogging certainly helps build Sherlock's reputation as a detective.

Joan Watson, on the other hand, is interested in being Sherlock — or, at least, being a consulting detective armed with his deductive methods. And in contrast to the Cumberbatch version, Jonny Lee Miller's Holmes is not nearly as self-sufficient. He's abrasive without being charming, dependent on his father for income, and recovering from a drug problem that destroyed his ability to work. Lance Mannion has commented that this weakens Holmes, but I'm not sure that I agree. Given that the original Holmes was a bit of a Mary Sue (a great observer, master of disguise, amateur boxer and stick-fighter, chemist, polyglot, and former spy) I don't miss seeing a version of the character that's less omni-capable.

Elementary wisely forgoes flashy zoom cuts to "show" how Sherlock examines a scene. They don't seem to have developed much of a substitute, unfortunately, so too often the show falls back on simply having characters explain the mystery to us. But I think this is in part because the mysteries are honestly second priority to where Elementary actually wants to focus: on the relationship between Holmes and Watson, with two possibilities for its ultimate outcome. On the one hand, it's hinted that this version of the great detective is really the result of two people working together — that Holmes and Watson together are the equivalent of the BBC Sherlock. Alternately, we're watching the origin story for a second Sherlock embodied in Joan Watson: one that can avoid the mistakes of drug abuse and arrogance, and benefit from her richer life experience as a surgeon.

The danger in speculating about a TV show this way, I've found, is the tendency to write about the show you wish you were watching, not the one that's actually onscreen. It's an easy mistake to make. I remember being mystified by John Rogers' glowing commentary on Jericho, which does not at all resemble the mediocre show that aired under that name, until I realized that really we weren't watching the same program — that the version Rogers was watching was being filtered through all the cool stuff he could have done with its premise.

And so it may be with Elementary. I'm only three-quarters through the first season, and even I will admit that it's uneven at best. It's possible I'm just a sucker for training montages. But the idea that Watson is not just a point of view character or a sounding post, but just the latest heir to a legacy of nigh-uncanny sleuthing... I have to admit, that's like catnip to me. I've got high hopes for it in the second season, and I'd put up with a lot of flexibility around the source material to watch it happen.

December 10, 2013

Filed under: gaming»design

Policing Procedurals

Even if I'm sticking with Steam for most of my gaming, our new PS4 did get me interested in Warframe, the free-to-play shooter that's available on Playstation and PC. I don't normally care for free-to-play, if only because I feel guilty for never buying anything, but I liked the central conceit of Warframe: procedurally-generated levels and highly-mobile Mass Effect-style combat. In retrospect, I probably should have been more skeptical. To understand why, we have to look back at how shooters have been built over the last twenty years.

There was a time, way before Halo and before franchises like Battlefield ran the earth, when one of the main selling points of a first-person shooter was the quantity of unique weapons that it brought to the table--an actual arms race, peaking with Duke3D which (for all its flaws) had some clever joke guns to go with the ubiquitous pistol/shotgun/chaingun trio. I'm not saying this was a better time, or that they were better games, but there was definitely a sense that the genre was about creative destruction, in the same way that fighting games are about combo systems and special attacks.

Then id Software built a monster for competitive deathmatch: Quake and its successors had an incredibly bland set of weapons, because in "serious" multiplayer the goal is to streamline everything except moving and shooting. This was the second refinement of shooter design, and it focused on the levels themselves, but as topology instead of as setting. Players concentrated on learning the levels so that they could plot a path that would keep them supplied, while denying pickups to the other players. A good Quake or Unreal Tournament player knew the game's weapons and how to aim, but more importantly they knew where to go, and when. Navigation became the mark of quality for a deathmatch bot.

Since then, these tendencies have mellowed as the possibility of more complex interactions and narratives has become available. Environments are built more for realism and story, weapons are more traditional and not usually why you buy the game. Which brings us to Warframe, which has basically none of these things. There's hardly any story, the "levels" are randomly generated from a series of tilesets, and the weapons are part of the free-to-play grind: either buy a new gun with real money, or spend a lot of time crafting one within the game's economy. Unlike the Mass Effect and Gears of War titles it resembles, there's no explicit cover system, but players do have a much wider range of movement options than a typical shooter: there are slides, flips, and wall runs available through various key combos.

If Warframe's computer-generated levels were any good, this would be a different post. Good levels would give players a way to put their acrobatic movement skills to good use, rewarding people who have learned the parkour system and can improvise in response to new environments. But the random generator mostly builds long, boring hallways connecting wide-open, pre-designed rooms, none of which require any particular skill outside of strafing and taking cover behind walls. Since players can't learn the levels and their flow, they can't optimize or get better at moving through them. And since new weapons require an investment of serious cash or time, almost everyone's using the same rifle and the same melee weapon, which means you never see anything that makes you want to spend any money or time in the first place.

The irony of this problem is that someone already got the formula right for doing procedural FPS games, and they did it by almost exactly reversing Warframe's formula. Borderlands has hand-built levels and enemy placement, combined with randomized weapon generation: each gun consists of components assembled onto a set of base bodies, which vary by "manufacturer" with certain preset tendencies and aesthetics. For example, Mariwan guns always inflict status effects (poison, fire, etc), while Jacobs weapons are Western-themed and can often fire as fast as the player can mash the button. Within those simple parameters, however, the results when you pull the trigger can vary wildly.

The result is a game that combines the two old-school driving forces of FPS design — clever level design and weapon variety — with the collector's urge that powers massive multiplayer games (Borderlands even borrows the color-coded quality markings from World of Warcraft, making it easy to evaluate a weapon drop in an instant). The innovation is not proceduralism — games like Diablo have long offered that — but figuring out how to balance it with the formula for a replayable and rewarding shooter. As someone who almost totally lacks a collection instinct, but loves the classic FPS genre, Borderlands 2 hits the sweet spot with remarkably few missteps (it's also surprisingly smart and funny, which is a welcome change).

I don't think procedural generation is impossible for first-person games — indeed, I think it's likely to have a bright future, particularly as web games mature and optimize for delivery size — but it illustrates just how difficult the challenge is likely to be for anyone who attempts it. For all that people talk about the genre as if it's just a collection of bro-heavy manshooters, there is undeniably a huge amount of craft that goes into the fundamental mechanics. As Kevin Cloud notes in Dan Pinchbeck's analysis of Doom (now 20 years old!),

Every genre has its real strengths, but in a shooter... if running and shooting is not fun, doesn’t feel natural, doesn’t feel visceral and powerful, then I think you are going to lose out.
Movement in FPS games is not just about how the player transitions from point A to point B, but about all the obstacles and decisions that make up that route. As such, building procedural content is not impossible, but it needs to provide good options for cover, paths for moving between pickups, and unexpected chances to either ambush enemies or be ambushed. Warframe may do this one day, but right now it's failing miserably. In the process, it's showing how little the developers have really thought about how the game they're building fits into the traditions of its genre.

December 4, 2013

Filed under: gaming»hardware»ps4

Press Play

Belle and I were planning on getting a Roku to replace our five-year-old XBox this Christmas, since the games are drying up and it doesn't make any sense to pay for a Live subscription just to watch Netflix and HBO. I still kind of bear a grudge against Sony for the CD rootkit they passed around years ago, but then my employers at ArenaNet bought everyone a PS4 as a holiday bonus. I am, it turns out, not above being a hypocrite when it comes to free stuff.

You can explain a lot about the last three generations of consoles by remembering that, at heart, Microsoft is a software company and Sony is a hardware company. Why did the XBox 360 suffer regular heat failures? Why does the PS Vita interface look like an After Dark screensaver? Our 360 was clearly on the edge of another DVD failure, so I bear them no particular good will. But you have to admit: up to the point that a given XBox malfunctions in one way or another, Microsoft knows how to build a usable operating system. Sony... well, it's not so much a core skill of theirs.

For example, after you turn on the PS4, and after the hundreds of megabytes of updates are done downloading and installing themselves a few times, you're greeted with a row of boxes:

  • Some kind of activity stream box, because people really wanted another Facebook.
  • Every game you've ever played. In my case, one demo I tried and will never touch again.
  • A web browser, which for some inconceivable reason doesn't use the touchpad on the controller, but has to be controlled via the thumbsticks and triggers, like Capcom wrote a game called Devil May Surf.
  • An ad for Sony's video service, which nobody is going to use instead of Netflix, and which can't be removed.
  • An ad for Sony's music service, which nobody is going to use instead of anything, and which can't be removed.
  • Something called the Playroom, which is useless if you don't have the PS4 camera, but can't be removed (there is a theme here).
  • A single item that contains all of your video streaming applications, even though these are probably where most people spend at least 50% of their time these days.
  • Oh, and of course, an item marked "Library" that shows a subset of the same list, I guess in case you want to see them arranged in vertical rows instead of horizontally.
At least the list is in chronological order by use, but only for certain items: no matter how often I open Amazon's video app, it'll always be cordoned off in a submenu, alongside a bunch of other video providers that I haven't actually installed but which Sony really wants me to know about, like Vudu and Crackle.

Apparently I'm a little grumpy about the menus.

Anyway for us, this is a media player, which means we'd like to have a remote control, but those don't exist for PS4 yet and it can't use regular IR remotes. The controller layout may make sense to someone who owned a PS3, but it's just baffling to me: why is the button normally used to go backwards assigned here to play/pause duties? To be fair, the XBox never really had a great controller story for DVDs either (both of them put fast-forward on the triggers, where you're guaranteed to accidentally hit it while setting the controller down), but at least it tried to be consistent with the rest of the OS.

You can pair a smartphone with the PS4, which one would think could be a chance to show custom controls for media, what with the touchscreen and all. You'd be wrong: the PS4 app dedicates 90% of its surface to a swipeable touchpad, apparently on the assumption that the three directional inputs on the actual controller are insufficient.

The whole time you're watching a movie, of course, the controller will glow like some sort of demented blue firefly, which helps the camera (which I don't have) to see where I am (hint: the couch). Since you can't just turn off the LED, I've got the whole controller set to shut itself off after ten minutes. This solves the glow, and keeps the batteries from draining themselves at an alarming rate, but now when I want to actually use the controller for something — say, to pause the movie because our dog has started making that special "I'm going to throw up" face — it interrupts with a bright blue screen, every single time, to ask me who I am. Meanwhile, my movie keeps playing in the background.

This is worth some emphasis: on the XBox, a console where we actually had multiple accounts, each new controller that was activated would either log in as the current user or just kind of wait in "guest" mode until the player actually signed in. On the PS4, a console where we have one account, to which I was already signed in with our only controller 20 minutes ago, Sony needs to know my identity before I can perform the critical, account-bound task of pausing a movie. Meanwhile, the dog is now standing sheepishly in front of a vomit-stained rug.

I'm a little grumpy about the media functions, too.

I'm well aware it's a little ridiculous to gripe this much about a free game system. It's not that the PS4 is a bad machine — it's on par with your average DVD player in terms of usability — but I tend to feel like maybe they should aim a little higher. I'm really hoping that these kinds of fixes will be easy to update, since most of the UI is apparently built using web technology instead of painstakingly coded native widgets.

What's really interesting about comparing consoles from both companies is that the kinds of things I really miss from the XBox (pinned items, Kinect voice commands, good media apps) weren't there from the start. Microsoft has gone through at least three major revisions since they released the 360 in 2005. Even though there have been regressions (and the ads have certainly gotten bigger over time), the overall trend has been for the better — in part because they've been effectively allowed to throw the whole thing away and start over. As far as I can tell, the PS3 was also improved, even if it wasn't reinvented in the same way. It takes a lot of nerve to make sweeping changes like that, and as well as a conviction that the physical box is not what you're selling — a philosophy that's well-suited to Microsoft's software background, but that even hardware companies can no longer ignore.

I've been so embedded in a constantly-shifting web environment for so long that I sometimes forget that not everything updates on a monthly basis. Sony will be more conservative than Microsoft, but even they will be rolling out patches to the PS4, many of which will probably address my complaints. We live in a world where you can turn around and find that your DVD player, or your phone, or your browser suddenly looks and acts completely differently. That's great for people like me who thrive on novelty, but it now occurs to me just how disorienting this might be for ordinary people. It may be worth considering whether a little stability might be good for us — even if it means preserving the bad with the good — and whether the technical community might benefit from a little sympathy to users overwhelmed by our love of change.

November 22, 2013

Filed under: tech»coding

Plug In, Turn On, Drop Out

This is me, thinking about plugins for Caret, as I find myself doing these days. In theory, extensibility is my focus for the next major release, because I think it's a big deal and a natural next step for a code editor. In practice, it's not that simple.

Approach #1: Postal Services

Chrome has a pretty tight security model applied to packaged apps, not the least of which is a strict content security policy. You can't run code from outside your application. You can't construct code using eval (that's good!) or new Function (that's bad). You can't add new files to your application (mostly).

Chrome does expose an inter-app messaging system similar to postMessage, and I initially thought about using this to create a series of hooks that external applications could use. Caret would broadcast notifications to registered listeners when it did something, and those listeners could respond. They could also trigger Caret commands via message (I do still plan to add this, it's too handy not to have).

Writing plugins this way is neatly encapsulated and secure, but it's also going to be intensely frustrating. It would require auditing much of Caret's code to make sure that it's all okay with asynchronous operation, which is not usually the case right now. I'd have to make sure that Caret is sufficiently chatty, because we'd need hooks everywhere, which would clutter the code with broadcast/response blocks. And it would probably mean writing a helper app to serve as a patchboard between applications, and as a debugging tool.

I'm not wild about this one.

Approach #2: Repo, Man

I've been trying to think of a way around the whole inter-app messaging paradigm for about a month now. At the same time, I've been responding to requests for Git and remote filesystem support, which will not be a core Caret feature. For some reason, thinking about the two in close proximity started me thinking along a new track: what if there were a way to work around the security policy using the HTML5 file system? I decided to run some tests.

It turns out this is absolutely possible: Chrome apps can download a script from any server that's whitelisted in their manifest, write that out to the filesystem, and then get a special URL to load that file into a <script> tag. I assume this has survived security audits because it involves jumping through too many hoops to be anything other than deliberate.

The advantages of this approach are numerous. Plugin code would operate directly alongside of Caret's source, able to access the same functions and modules and call the same APIs that I use. It would be powerful, and would not require users to publish plugins to the Chrome store as if they were full applications. And it would scale well--all I would need to do is maintain the index and provide some helper functions for developers to use when downloading and caching their code.

Unfortunately, it is also apparently forbidden by the Chrome Web Store policies, which state:

Packaged apps should not ... Download or execute scripts dynamically outside a sandboxed environment such as a webview or a sandboxed iframe.
At that point, we're back to postMessage unless I want to be banned from the store. So much for the workaround.

Approach #3: Local hosting

So how can I make plugins work for end users? Well, honestly, maybe I don't. One of the nice things about writing developer tools, particularly oddball developer tools, is that the people using them and wanting to expand on them are expected to have some degree of technical knowledge. They can be trusted to figure out processes that wouldn't necessarily be acceptable for average computer users. In this case, that might mean running Caret as an unpacked app.

Loading Caret from source is not difficult--I do it all the time while I'm testing. Right now, if someone wants to fork Caret and add their own features, that's easy enough to do (and actually, a couple of people have done so already). What it lacks is a simple entry point for people who want to contribute functionality without digging into all the modules I've already written.

By setting up a plugins directory and a little bit of infrastructure, it's possible to reach a middle ground. Developers who really want extra packages can load Caret from source, dump their code into a designated location, and have their code bootstrapped automatically. It's not as friendly as having web store distribution, and it's not as elegant as allowing for a central repo, but it does deliver power without requiring major rewrites.

Working through all these different approaches has given me a new appreciation for insecurity, which sounds funny but it's true. Obviously I'm in favor of secure computing, but working with mobile operating systems and Chrome OS that strongly sandbox their code tends to make a person aware of how helpful a few security holes can be, and vice versa: the same points for easy extension and flexibility are also weak points that can be exploited by an attacker. At times like this, even though I should maybe know better, that tradeoff seems absolutely worth it.

November 14, 2013

Filed under: tech»coding

For Free Ninety Nine I'll Beat 99 Acts Down

Assuming that the hamster powering the Chrome web store stats is just resting, Caret clicked over to 10,000 installations sometime on Monday. That's a lot of downloads. At a buck a piece, even if only a fraction of those people had bought a for-pay version, that might be a lot of money. So why is Caret free? More importantly, why is it free and open source? Ultimately, there are three reasons:

  1. I feel like I owe the open source community for the value I've gotten from it (basically, everything on the Internet), and this is a way to repay that debt.
  2. Caret isn't really just mine. It's heavily influenced by Sublime, and builds on another open source project for its text processing. As such, it feels awkward to charge money for other peoples' work, even if Caret's unique code is significant in its own right.
  3. I think I get more value (i.e. job marketability, reputation, skill practice) out of being the person with a chart-topping Chrome app, in the long term, than I would get from sales.

Originally, I had planned on writing about how I reconcile being a passionate supporter of paid writing while giving away my hobby code, but I don't actually see any conflict. I expect a paycheck for freelance coding the same way I expect it for journalism — writing here (and coding Caret) doesn't directly benefit anyone but me, and it doesn't really cost me anything.

In fact, it turns out that both industries also share some uncomfortable habits when it comes to labor. Ashe Dryden writes:

Statistically, we expect that the demographic breakdown of people contributing to OSS would be about the same as the people who are participating in the OSS community, but we aren't seeing that. Ethnicity of computing and the US population breaks down what we would hope to see as far as ethnicity goes. As far as gender, women make up 24% of the industry, according to the same paper that gave us the 1.5% OSS contributor statistic.

Dryden was responding to a sentiment that I've seen myself (and even been guilty of, from time to time): using a person's open source record on sites like GitHub as a proxy for hireability. As she points out, however, building an open source portfolio is something that's a lot easier for white men. We're more likely to have free time, more often have jobs that will pay for open source contributions, and far less likely to be harassed or dismissed. I was aware of those factors, but I was still shocked to see that diversity numbers in open source are so low. We need to do better.

As eye-opening as that is, I think Dryden's middle section centers around a really interesting question: who profits?

I'd argue that the people who benefit the most from the unpaid labor of OSS as well as the underpaid labor of marginalized people in technology are business owners and stakeholders in these companies. Having to pay additional hundreds of thousands or millions of dollars for this labor would mean smaller profit margins. Technology is one of the most profitable industries in the US and certainly could support at least pay equality, especially considering how low our current participation is from marginalized people.

...Open source originally broke us free from the shackles of proprietary software which forced us to "pay to play" and gave us little in the way of choices for customization. Without realizing it, we've ended up in a similar scenario where we are now paying for the development of software that large companies financially benefit from with little cost to them.

Her conclusion — that the community benefits, but it's mostly businesses who boost their profits from free software — should be unsettling for anyone who contributes to open source, and particularly those of us who see it as a way to spread a little socialist good will. For this reason, if nothing else, I'll always prefer the GPL and other "copyleft" licenses, forcing businesses to play ball if they want to use my code.

November 5, 2013

Filed under: tech»innovation

Patently Nonsense

Now that Apple and Microsoft (with a few others) have formed a shell company in order to sue Google over a bunch of old Nortel patents, and IBM has accused Twitter of infringing on a similar set of bogus "inventions," it's probably time again to talk about software patents in general.

Software Patents: Frequently Asked Questions

Q. When it comes to software patents...

A. No.

Q. Hang on. Are they a valid thing? Are any of these suits valid?

A. See above: no, and no.

Q. Why not?

A. Because software patents, as a description of software, are roughly like saying that I could get a patent for the automobile by writing "four tires and an engine" on a piece of paper. They're vague to the point of uselessness, and generally obvious to anyone who thinks about a problem for more than thirty seconds.

Q. Yeah, but so what? Isn't the point of patents to create innovation? Haven't we had lots of innovation in software thanks to their protection?

A. True, the point of patents is to make sure that people let other people use their inventions in exchange for licensing fees, which is supposed to incentivize innovation. In patents for physical inventions, that makes sense: I need to know how to build something if I want to use it as a part of my product, and the patent application will tell me how to do that. But software patents are not descriptive in this way: nobody could write software based on their claims, because they're written in dense legal jargon, not in code.

Let's take one of the patents in question, IBM's "Programmatic discovery of common contacts." This patent covers the case where you have two contact lists for separate people, and you'd like to find out which contacts they have in common. It describes:

  1. Having one local list of contacts,
  2. getting a second from the server,
  3. comparing them, and
  4. presenting a list of hyperlinks to the user.
That is literally the extent of detail present in this patent. Now, do you think you can build a contact matching system by licensing that patent? Or will you do what every other software company on earth has done, and invent the stupid thing yourself (a task that would take a decent coder a couple of days)?

As for the innovation argument, it's impossible to prove a negative: I can't show you that it wasn't increased by patents. But consider this: most of the companies that we think of as Internet innovators are strikingly low on patent holdings. Reportedly, Twitter owns nine, and has pledged not to sue anyone over them. Google's only applied for a few, although they have purchased many as a defensive tactic. Facebook is not known for licensing them or taking them to court (indeed, just the opposite: enter the Winklevii). For the most part, patent litigation is limited to companies who are either no longer trailblazers (Microsoft), are trying to suppress market competition (Apple), or don't invent anything at all (Intellectual Ventures). Where's the innovation here?

This American Life actually did a pair of shows on patents, strongly arguing that they've been harmful: companies have been driven out of business by patent trolls. Podcasters have been sued for the undeniably disruptive act of "putting sound files on the Internet." The costs to the industry are in the billions of dollars, and it disproportionately affects new players — exactly the kind of people that patents are meant to protect.

Q. So there's no such thing as a valid software patent?

A. That's actually a really interesting question. For example, last week I was reading The Code Book, by Simon Singh. Much of the book is taken up by the story of public key encryption, which underlies huge swathes of information technology. The standard algorithm was invented by a trio of researchers: Ron Rivest, Adi Shamir, and Len Adleman. As RSA, the three patented their invention and successfully licensed it to software firms all over the world.

The thing about the RSA patent is that, unlike most software patents, it is non-obvious. It's extremely non-obvious, in fact, to the degree that Rivest, Shamir, and Adleman literally spent years thinking about the problem before they invented their solution, based on even more years of thinking on key exchange solutions by Whitfield Diffie, Martin Hellman, and Ralph Merkle. RSA is genuinely innovative work.

It is also work that was independently invented by the espionage community several years before (although obviously they weren't allowed to apply for patents). Moreover, a lot of the interesting parts of RSA are in the math, and math is not generally considered patentable. Nevertheless, if there's anything that would be a worthy software patent, RSA should qualify.

It goes without saying that matching contacts, or showing search ads, or scrolling a list based on touch, are not quite in the same league. And it's clear that patents are not creators of value in software. People aren't buying Windows computers or iPhones because they're patented. They're buying them because they run the software people want, or run on good-looking hardware, or any number of other reasons. In other words: software is valuable because of what it does, not because of how.

Q. So software shouldn't have any protection?

A. Sure, software should be protected. In fact, it already is. Code can be copyrighted, and you can sue people who take it and use it without your permission. But that's a different matter. Copyright says you can sue people who publish your romance novel. Software patents would be like suing anyone who writes about boy-meets-girl.

Q. Okay, fine. What's the answer?

A. Ultimately, we need patent reform. The steps for this are the same as any other reform, unfortunately: writing letters, asking your political candidates, and putting pressure on the government. It's not a sexy recommendation, but it's effective. If we could frame these as another type of frivolous lawsuit, the issue may even get some traction.

Personally, though, I'm trying to vote with my wallet. I'd like to not give money to companies that use patents offensively. Incidentally, this is why I'm cautiously optimistic for Valve's Steam Machines: it's much harder for me to not give money to Microsoft, since I play a lot of games on Windows (and XBox). A Linux box with a good game library and a not-terrible window manager would make my day.

Finally, there's a community site that can help. Ask Patents is a forum set up by the people who run the Stack Overflow help group for programmers. It takes advantage of a law that says regular people can submit "prior art" — previously-existing examples of the patented invention — that invalidate patents. Ask Patents has successfully blocked bad software patents from being granted in the first place, which means that they can't be used for infringement claims. Over time, success from finding prior art makes patents more expensive to file (because they're not automatically granted), which means fewer stupid ones will be filed and companies will need to compete in the market, not the courtroom.

Future - Present - Past