Earlier today I took the wraps off of the private repo for our Chambers Bay interactive flyover. You can find the source code here, and a post on our dev blog about it here. It was a really fun challenge, and a rare example of using WebGL in a news capacity (the NYT did the Dawn Wall, but that's the only one I can remember recently).
From a technical standpoint, this was my first three.js project, and the experience was largely positive. I think there's a strong case to be made that three.js is basically jQuery for WebGL: sure, you don't need it, but it only takes a couple of features to make it worthwhile. In this case, I didn't particularly feel like writing a model loader or a scene graph. There are still plenty of hooks to write the parts that I do enjoy, like the fragment shader for the landscape (check out that sweet dithering), or the UI for directing the camera. Sure, three.js is a relatively large library, but I'm loading 4MB of textures and another 4MB of gzipped landscape model, so what's a few more hundred KB of code?
WebGL itself runs surprisingly well these days, although failure modes do not seem to be its strong suit. For example, the browser may have WebGL support enabled, but then crash when it tries to render (or it may be lying about support, as with the remote VM sessions used in Times meeting rooms). That said, I was astonished to find that pretty much everything (mobile included) could run the landscape at a solid frame rate, despite the fact that it's a badly-optimized mesh with 150,000 triangles. Even iOS, which usually falls over and dies when WebGL pushes past its skimpy RAM limits, was able to run smoothly once I added a low-res texture for it to use.
This was an ambitious project using some pretty cutting-edge web technology, which makes it interesting in light of arguments that the web suffers from "featuritis". After all, when you're talking about feature overkill, WebGL is a barnstormer. But this story would have been tough to tell another way, and it would have never had the same reach siloed in an app store.
Or take Paul Ford's mind-boggling What is Code? in Businessweek this month: behind Bloomberg's Trapper Keeper design aesthetic, it's a powerful article that integrates animations, videos, and interactive demonstrations with the textual message. Ironically, I saw many of the same people that criticize web apps going wild for Ford's piece, a stance that I can only attribute to sophistry.
My team at the Seattle Times has gradually abandoned the term "news apps," since everyone who hears it assumes we're actually writing iPhone clients for the paper. As a term of art, it has always been clumsy. But it does strike at a crucial quality of what we do, which occupy a gray area between "text" and "program." And if it seems like I'm touchy about pundits who think we should abandon the web, this is in large part the reason why.
Arguments that browsers should just go back to being a document viewers ignore the fact that HTML is not just a text format: it's a hypermedia format, and those have always blurred the already-fuzzy lines between data and code (see also: Excel, Hypercard, and IPython notebooks). It's true that the features of the web platform are often abused. Nobody likes slow navigation, ad popups, or user tracking scripts. But it's those same features that make new kinds of storytelling possible — my journalism is built on the same heavily-structured, "over-tooled" web platform that critics find so objectionable. I wouldn't give that up for all the native apps in the world.
It's an accepted truth on the web that fast pages are better for users — people stay on them longer, follow more links from them, and generally report being happier with them. I think a lot about performance on my projects, because I want readers to be thinking about the story, not distracted by slow load times.
It's possible that I've been more aware of it, just because I've been working on a project that involves smoothly animating a chart using regular HTML instead of canvas, but it seems like it's been a bad month for that kind of thing. First Peter-Paul Koch wrote a diatribe about client-side templating, insisting that it's a needless performance hit. Then Flipboard wrote about discarding traditional elements entirely, instead rendering everything to a canvas tag in pursuit of 60 frames/second animations. Ironically, you'll notice that these are radically different approaches that both claim they create a better experience.
Instead of just sighing while the usual native app advocates use these posts to bash the web, and given that I am working on a page where high-performance mobile animations are a key part, I thought it'd be nice to talk about some experiments I've run with the approaches found in both. There are a lot of places where the web platform needs help competing on mobile, no doubt. But I'd prefer we talk about actual performance problems, and not get sidetracked into chasing down scattered criticisms without evidence.
At the other extreme is Flipboard's experiment with canvas rendering. Instead of putting everything in the document, like normal websites, they put a full-screen canvas image up and render everything — text, images, animations, etc. — manually to that buffer. You can try a demo out on your device here. On my Nexus 5, which is a reasonably new device running the latest version of Chrome, it's noticeably choppy. My experience with canvas is that Chrome's implementation is actually much faster than Safari, so I don't expect it to be smooth on iOS either (they've blacklisted tablets, so I can't be sure).
In order to get this "fluid" experience, here's what the Flipboard team threw away:
Again, I'm not claiming that my use case is a perfect analogue. I'm animating a graphic in response to a single button press, and they're attempting to create an "infinite scroll" (sort of — it's not really a scroll so much as an animated pager). But this idea that "the DOM is lava" and touching it will cause your reader's phones to instantly burst into flames of scorn seems patently ridiculous, especially when we look back at that list of everything that was sacrificed in the single-minded pursuit of speed.
Performance is important, and I care deeply and obsessively about it. As a gamer and a graphics nerd, I love tweaking out those last few frames per second, or adding flashy effects to a page. But it's not the most important thing. It's not more important than making your content available to the blind or visually impaired. It's not more important than providing standard UI actions like copy-and-paste or "open in new tab." And it's not more important than providing a fallback for older and less-powerful devices, the kind that are used by poor readers. Let's keep speed in perspective on the web, and not get so caught up in dogma that we abandon useful techniques like client-side templating and the DOM.
Similar to our news app template, I've put together a Grunt scaffolding for creating bundled custom elements, including HTML templating and CSS, all in a single standalone file. It's our component template — or, as I like to call it, the Poor Journalist's Polymer.
My second testbed project is a Leaflet map element that uses custom HTML to set the map configuration without ever writing a line of JSON (unless you really want to). It's intended to make mapping simple and fast for web producers, while still offering plenty of power for people like me who just want the boilerplate out of the way. Leaflet's a great candidate for this kind of declarative approach, and I think this is a really promising demo for the power of custom elements.
For standalone components like these, the template seems to be working well. I haven't yet solved the problem of easily embedding them in highly opinionated news apps, due to the way that dependencies are handled. It's useful for custom elements to be able to bundle their CSS and other assets into their package, similar to the way that HTML imports and shadow root offer embedded styles, but that means they may not integrate well into projects that already have their own build system. As far as I can tell, the best solution for now will probably be to load the packages from Bower and require() the standalone files from its build directory, which should work with whatever module system you like.
We came very close to using WebGL for a Seattle Times special report that will come out next week. Now that iOS 8 has shipped with support for WebGL, albeit in an unstable and slightly buggy form, it's common enough that I felt comfortable using it (with a scaled-down 2D fallback) for our audience. In the end, we went with a different design language and shelved the WebGL experiments, but the experience has left me very excited about the potential for mainstream usage.
There's an obvious parallel here, which is the first two major versions of Android. Because it was designed to run on low-end hardware, Android drew all its UI via software until 3.0 (and hardware acceleration didn't become widespread until 4.0). The resulting lag was never as bad as critics claimed, but it did mean that a lot of Android looked and felt a bit utilitarian. You wouldn't see something like Material Design emerge until the system supported using the GPU for rendering ordinary UI.
It's not a coincidence that Google's moving to Material Design on both Android and the web. Its design language — a smoothly-animated world of flat, geometric shapes — is attractive, but more importantly it's well-matched to the kinds of flat, geometric shapes that can be animated fluidly in a browser, using the 3D acceleration that's already built into the composition layer. Web Components will give developers a way to package those elements up, and make them reusable. Flexbox makes their layouts scalable and responsive.
But for the web platform to move forward, we need more than just a decent look and feel. We need the ability to write the kinds of applications that people insist that it can't run. WebGL is a step in that direction: graphics with near-native speed and capability, instantly deployed and paired with a surprisingly powerful UI toolkit. The kinds of apps and experiences we can write othe web, for a mainstream and mobile audience, just got a lot bigger. And I for one am looking forward to pushing those boundaries as much as I can.
Paul Kinlan's post, Add-to-homescreen Is Not What the Web Needs, is only the most recent in a long-running debate surrounding "apps" on mobile, but it is thought-provoking. Kinlan, who cheerleads for the Web Intents integration system in Chrome, naturally thinks that having an "add-to-homescreen" option misses the point:
I want to see something much more fundamental. The web offers something far richer: it encourages lightweight usage with no required installation and interaction with on-demand permissions. I never want to see an install button or the requirement to understand all the potential permissions requried before trying the app. The system should understand that I am using an app and how frequently that I use it and it should then automatically integrate with the launch points in the OS.
Kinlan has a great point, in that reducing the web to "just another app" is kind of a shame. The kinds of deeper integration he wants would probably be prone to abuse, but they're not at all impossible. Mozilla wants to do something similar with Firefox OS, although it probably gets lost in the vague muddle of its current state. Worse, Firefox OS illustrates the fundamental problem with web "apps" on mobile, and it's probably going to take a lot more than a clever bookmark to solve the problem. That's because the real problem with the web on mobile is URLs, and nobody wants to admit that.
As a web developer, I love URLs. They're the command line of the web: a powerful tool for organizing information and streaming it from place to place. Unfortunately, they're also like the command line in other ways: they're arbitrary, much-abused, and ultimately difficult to type on mobile. More importantly, nobody who isn't a developer really understands them.
There is a now-infamous example of the fact that people don't understand URLs, which you may remember as the infamous Facebook login of 2010. That was the point at which the web community realized that for a lot of users, logging into Facebook went a lot like this:
As a process, this was fine until ReadWriteWeb actually published a story about Facebook's unified login that rose to the top spot in the Google search listings, at which point hundreds of people began commenting on the article thinking that it was a new Facebook design. As long as they got to Facebook in the end, to these people, one skinny textbox was basically as good as another. I've actually seen people do this in my classes, and just about ground my teeth to nubs watching it happen.
In other words, the problem is discovery. An app store gives you a way to flip through the listings, see what's popular, and try it out. You don't need to search, and you certainly don't need to remember a cryptic address (all these clever .io and .ly addresses are, I'm pretty sure, much harder to remember than plain old .com). For most of the apps people use, they probably don't even scroll very far: the important stuff, like Facebook and Candy Crush, is almost certainly at the top of the store anyway. Creating add-to-homescreen mechanisms is addressing the wrong problem. It's not useless, but the real problem is not that people don't know how to make bookmarks, it's that they can't find your web app in the first place.
The current Firefox OS launcher isn't perfect, but it at least shows someone thinking about the problem. When you start the device, it initially shows a search box titled "I'm thinking of...". Tap into the box and even before you start typing it'll instantly start showing a set of curated sites sorted into categories like "social" and "games." If you want isn't there, you can continue to search the web as a whole. Sites launched from this view start in "app mode" with no URL bar, even though they're still just web sites and nothing's technically been installed. Press the bookmark button, and it's added to your homescreen. It's exactly as seamless as we've always claimed the web could be.
On top of this, sadly, Mozilla adds the Marketplace app, which can install "packaged" apps similar to Chrome OS. It's an attempt to solve the discoverability problem, but it lacks the elegant fluidity of the curated results from the launcher search (not to mention that it's kind of confusing). I'm not wild about curation at the best of times — app stores are a personal pet peeve — but it serves a purpose. We need both: an open web, because that's the spirit of things, and a market destination, because it solves the URL discovery process.
What we're left with is a tragedy of the commons. Mozilla's marketplace can't serve the purpose of the open web, because it's a curated and little-loved space that's only for Firefox OS users. Google is preoccupied with its own Chrome web store, even though it's certainly in a position to organically track the usage of web apps via user searches. Apple couldn't care less. In the meantime, web app discovery gets left with the scraps: URLs and search. There's basically no way, other than word of mouth, that your app will be discovered by normal people unless it comes from an app store. And that, not add-to-homescreen flaws, is why we can't have nice things on the web.
After a busy couple of weeks, Seattle went and won the Super Bowl, leading to the world's most polite celebration in our neighborhood:
There was another prize for the weekend: a friend of ours gifted us a Chromecast, which will be much appreciated since there's currently no way to watch HBO on the PS4. On Monday, Google released the public SDK for the platform, so I decided to poke around a bit.
Chromecast has a decidedly-odd way of loading content. The device itself is just a thin shell around a Chrome window, and it loads web pages like any other browser. But there's no keyboard of any kind, so how does it know which page to load? The answer is that each "app" has an ID listed with Google, corresponding to a set of URLs that the developer provides. When a mobile app or a computer running Chrome triggers the Chromecast, it sends the app ID, which the device then sends to Google and gets a URL in return (or, if the app hasn't been listed, it does nothing). From that point on, you can send messages to the page over via Google's cloud, and your page can do whatever you want it to do. Getting your pages linked to an application ID on the Chromecast lookup servers costs $5.
In the time since I last wrote about Caret, it's jumped up to 1.0 (and then 1.1). I've added tab memory, lots of palette search options, file modification watches, and all kinds of other features — making it legitimately comparable with Sublime. I've been developing the application using Caret itself since version 0.0.16, and I haven't really missed anything from native editors. Other people seem to agree: it's one of the top dev tools in the Chrome web store, with more than 1,500 users (and growing rapidly) and a 4.5/5 star rating. I'm beating out some of Google's own apps, at this point.
Belle's reaction: "Now you charge twenty bucks for it and make millions!" It's good to know one of us has some solid business sense.
Caret is already designed around message-passing for its internal APIs (as is Ace, the editing component I use), so it won't be too difficult to add external hooks, but it'll never have the same power as something like Sublime, which embeds its own Python interpreter. I can understand why Google made the security decisions they did, but I wish there was a way to relax them in this case.
I figure I have roughly six months to a year before Caret has any serious competition on Chrome OS. Most of the other editors aren't interested in offline editing or are poorly-positioned to do so for architectural reasons. The closest thing to Caret from the established players would be Brackets, which still relies on NodeJS for its back-end and can't yet run the front-end in a regular browser. They're working on the latter, and the former will be shimmable, but the delay gives me a pretty good head start. Google also has an app in the works, but theirs looks markedly less general-purpose (i.e. it seems aimed specifically at people building Chrome extensions only). Mostly, though, it's just hard to believe that someone hadn't really jumped in with something before I got there.
Between Caret, Weir, and my textbook, this has been a pretty productive year for me. I'm actually thinking that for my next project I may write another short book — one on writing Chrome apps using what I've learned. The documentation from Google's end is terrible, and I hate to think of other people having to stumble through the APIs the way I did. It might also be a good way to get some more benefit out of the small developer community that's forming around Caret, and to find out if there's actually a healthy market there or not. I'm hoping there is: writing Caret has been fun, and I'd to have the chance to do more of this kind of development in the future.
At this point, Caret has been in the Chrome Web Store for about a week. I think that's long enough to say that the store is a pretty miserable experience for developers.
When I first uploaded it last week, Caret had these terrible promo tiles that I threw together, mostly involving a big pile of carrots (ba dum bum). At some point, I made some slightly less terrible promo tiles, stripping it down to just bold colors and typography. Something set off the store's automated review process, and my new images got stuck in review for four days — during which time it was stuck at the very bottom of the store page and nobody saw it.
On Tuesday, I uploaded the first version of Caret that includes the go-to/command palette. That release is kind of a big deal--the palette is one of the things that people really love about Sublime, and I definitely wanted it in my editor. For some reason, this has triggered another automatic review, this one applied to the entire application. I can unpublish Caret, but I can't edit anything — or upload new versions — until someone checks off a box and approves my changes. No information has been provided on why it was flagged, or what I can do to prevent these delays in the future.
Even at the best of times, the store takes roughly 30 minutes to publish a new version. I'm used to pushing out changes continuously on the web, so slow updates drive me crazy. Between this and the approval hijinks, it feels like I'm developing for iOS, but without the sense of baseless moral superiority. What makes it really frustrating is the fact that the Play store for Android has none of these problems, so I know that they can be solved. There's just no indication that the Chrome team cares.
I was planning on publishing a separate, Google-free version of the app anyway, so I worked out how to deploy a standalone .crx file. The installation experience for these isn't great — the file has to be dragged onto the Chrome extensions list, and can't just be installed from the link — and it introduces another fun twist: even though they promised it would be possible for years, there's no way to download the private key that Google uses in the Chrome store, meaning that the two installations are treated as completely different applications when installed.
Fair enough: I'll just make the standalone version the "edge" release with a different icon, and let the web store lag behind a little bit. As a last twist of the knife, generating a .crx package as part of a process that A) won't include my entire Git history, and B) will work reliably across platforms, is a nightmare. Granted, this is partly due to issues with Grunt, but Chrome's not helping matters with its wacky packaging system.
All drama aside, everything's now set up in a way that, if not efficient, is at least not actively harmful. The new Caret home page is here, including a link to the preview channel file (currently 3 releases ahead of the store). As soon as Google decides I'm not a menace to society, I'll make it the default website for the store entry as well.
The problems with Google's web store bug me, not so much because they're annoying in and of themselves, but because they feel like they miss the point of what packaged web apps should be. Installing Caret is fast, secure, and easy to update, just as regular web apps are. Developing Caret, likewise, is exactly as easy and simple as writing a web app (easier, actually: I abuse flexbox like crazy for layout, because I know my users have a modern browser). Introducing this opaque, delay-ridden publication step in between development and installation just seems perverse. It won't stop people from using the store (if nothing else, external installation is too much of a pain not to go through the official channel), but it's certainly going to keep me from enjoying it.
As I mentioned in my Chromebook notes, one of the weak points for using Chrome OS as a developer is the total lack of good graphical editor. You can install Crouton, which lets you run Vim from the command line or even run a full graphical stack. But there aren't very many good pure text editors that run within Chrome OS proper — most of the ones that do exist are tied to hosted services like Cloud9 or Nitrous. If you just want to write local files without a lot of hassle, you're out of luck.
I don't particularly want to waste what little RAM the Chromebook has running a whole desktop environment just for a notepad, and I'm increasingly convinced that Vim is a practical joke perpetuated by sadists. So I built the Chrome OS editor I wanted to have as a packaged app (just in time!), and posted it up in the store this weekend. It's 100% open source, of course, and contributions are welcome.
Caret is a shell around the Ace code editor, which also powers the editor for Cloud9. I'm extremely impressed with Ace: it's a slick package that provides a lot of must-have features, like syntax highlighting, multiple cursors, and search/replace, while still maintaining typing responsiveness. On top of that base, Caret adds support for tabbed editing, local file support, cloud settings storage, and Sublime-compatible keystrokes.
In fact, Sublime has served as a major inspiration during the development of Caret. In part, this is just because it's the standard for web developers that must be met, but also because it got a lot of things right in very under-appreciated ways. For example, instead of having a settings dialog that adds development complexity, all of Sublime's settings are stored in JSON files and edited through the same window as any other text files — the average Sublime user probably finds this as natural as a graphical interface (if not more so). Caret uses the same concept for its settings, although it saves the files to Chrome's sync service, so all your computers can share your preferences automatically.
The current release of Caret, 0.0.10, is usable enough that I think you could do serious editing with it — I've certainly done professional work with less effective tools, including the initial development on Caret itself — but I'm on a roll adding features and expect to have a lot of improvements made by the end of next week. My first priorities are getting the keybindings into full working condition and adding a command palette, but from that point on it's mostly just polish, bugfixes, and investigating how to get plugin support past Chrome's content security policy. Once I'm at 1.0, I'll also be posting a standalone CRX package that you can use to install Caret without needing a Google account (it'll even auto-update for you).
Working with Chrome's new packaged app support has been rough at times: there are still a lot of missing capabilities, and calling the documentation "patchy" is an insult to quilts everywhere. But I am impressed with what packaged apps can do, not the least of which is the ease of installation: if you have Chrome, you can now pretty much instantly have a professional-grade text editor available, no matter what your operating system of choice. This has always been a strong point for web apps anyway, but Chrome apps combine that with the kinds of features that have typically been reserved for native programs: local file access, real network sockets, or hardware device access. There's a lot of potential there.
If you'd like to help, even something as simple as giving Caret a chance and commenting with your impressions would be great. Filing bugs would be even better. Even if you're not a programmer, having a solid document editor may be something you'd find handy, and together we can make that happen.
Once Microsoft announced IE 11 will offer WebGL, that was pretty much the last straw: Apple may drag their feet at enabling it in Safari, but everyone else seems to have decided that it's secure enough and capable enough for production. I still think it's a little nutty, but I don't really have an excuse to avoid it anymore. So when an EaselJS-based visualization at work started having performance issues, I wrote a WebGL shim as a learning project.
Of course, these are not uncommon mistakes in programming tutorials. In fact, they're extremely common--I just haven't had to learn anything from scratch in a while, and had forgotten how confusing the process could be. Anyone writing for beginners would do well to keep these errors in mind.
There are a few walkthroughs that I found more helpful. As I've mentioned, Greg Tavares's series on WebGL was eye-opening, and Brandon Jones provided the only worthwhile explanation of attribute array setup that I found. Between those two, and countless Google searches, I managed to cobble together a basic understanding of how the GL state machine actually works.
As a way of distilling out that knowledge, I've assembled the WebGL demo script that I would have wanted when I started out. It uses no external code--everything's right there on the same page. It explains each parameter that's used, and what each function call does. And it's only concerned with drawing a basic 2D shape--no matrix math is involved. It's stored in a Github Gist, so feel free to file pull requests against anything you find confusing. Also, feel free to look through EaselGL: it's a bit more advanced and I need to add more comments, but as a 2D API I think it's quite a bit easier to understand than the typical game library, particularly for ex-ActionScript developers like myself.