Mile
Zero

Next week a new quarter of Intro to JavaScript will start, and while I'm almost sick of writing about it, it's been constantly on my mind. I've talked a little here about some of the changes I'll be making--primarily I'll be teaching jQuery first--but there's another big experiment I'm going to try: using test-driven development tools to give students feedback on assignments and quizzes throughout the class. I did this in a limited way during the Winter quarter, with an assignment and an exam built using unit tests, but now I want to make it much more widespread. Why use testing this way? Two reasons: to force students to code, and to encourage them to think more critically about their mistakes.

The first reason may seem odd--they're in a programming class, shouldn't that force them to code already?--but it is really an extension of drill learning. I am not good at teaching via drills: I appreciate the value of repetition when I'm learning myself, but I hate standing in class and going through the same exercise again and again. I'd much rather give lectures about theory and application than do drills. But it's clear from last quarter that many students do not get enough practice from lectures. They may be too busy to write code at home, and assignments don't usually force them to use all the constructs of the language.

Drill and repetition are important because, just as with a spoken language, it's not enough to learn the rules of grammar. You need to go out and practice, so that the mechanics become natural and you can concentrate on expressing yourself. That's the motivation behind Zed Shaw's Learn Python the Hard Way series, and I tend to agree. Take the humble for loop, for example. I've written loops for so long I could do them in my sleep, but if you're just starting out there are a lot of moving parts that need to be grasped: for (var i = 0; i < list.length; i++) { //loop goes here } We've written a keyword (for), a series of semicolon-delimited statements inside parentheses, a block (represented by the curly braces), and we haven't even gotten to the actual loop. Just within this boilerplate, we're initializing a counter with the arbitrary name of "i" and giving it an initial value, testing it, and incrementing it. All of this is expressed in a peculiar syntax (the test and increment are tossed in after the setup statement, not placed in the order where they actually get executed), with a lot of very particular punctuation, plus a lot of bookkeeping to track mentally.

Writing for loops (or conditionals, or functions, or almost anything else) is therefore as much about learning a very specific arrangement of characters as it is about the overarching concepts. Languages can make this slightly easier to grasp (as with Python's comprehension-style loops and no-braces indentation), but I don't think you ever get away from it entirely. It is my goal this quarter to make students write as many of any given construct as possible in class, so that the low-level syntax becomes automatic, while using tests to keep the high-level goals in sight.

Which brings us to the second reason for using constant testing: students need to learn how to find mistakes by thinking critically about their output. Part of that process involves teaching them to use the developer tools: how to read a Firebug log, understand an interpreter error, and examine values in the debugger. But it also means they need to learn to check their code constantly, and to have thought about what they expect to get out of any given block. While I'll be providing tests on quizzes and homework assignments, so that students can immediately see if their answer was correct, the ultimate goal is to have them write their own simple tests to check their own code.

It's my belief that most of my students know what they want to do at the very highest level, but they're not able to break those goals into smaller pieces, and they don't know how to figure out when the smaller pieces stop working. My hypothesis is that writing tests may not be enough to create critical thinking where none exists, but it will serve as a good starting place for discussions of problem-solving strategy: what caused this test to fail? What were our expectations, and how were the results different? Was this test actually a good test to begin with, even if it "passes?" These are questions that good programmers ask, perhaps naturally, but I refuse to believe that they can't be encouraged in any student.

At this point, with the winter quarter at SCCC drawing to a close, I'd like to add onto my earlier comments on teaching JavaScript. I would still argue that it's not a great first language--too much reliance on other technology stacks (the browser, HTML, CSS), among other things. But John Resig's original post also talks about working around that complexity with libraries, and that speaks to a deeper divide in the JavaScript community: should newcomers learn "JavaScript," or "jQuery?" In this, I'm increasingly on Resig's side.

It's easy to understand why many JavaScript programmers are so adamant that newcomers should learn the language first. Sites like Stack Overflow are full of people whose answer to every JavaScript question is "use jQuery," even in response to simple problems that should be solved more directly. There are a lot of people out there whose idea of being a JavaScript programmer means "I can include jQuery and a few plugins on my page." Not that there's anything wrong with that.

But is the solution to teach people the DOM? That seems like cruel and unusual punishment--a kind of Protestant work ethic: real programmers suffer. The browser document object model ranks up there in the history of terrible API design: it's overly-verbose, inconsistent, and tedious. If you use it for any amount of time, you will end up recreating at least the traversal parts of jQuery--searching up and down the tree for related elements--if not the animation and CSS methods as well. Traversal isn't a hard problem per se, but as with anything involving recursion, it's kind of a lot to throw at a beginner.

In fact, it probably gets in the way of doing any real, productive learning. This quarter, caught up in the desire to do things "the right way," I taught JavaScript from the DOM methods up. As a result, we didn't get to jQuery until the sixth week--halfway through our time!--and several students asked "why would we ever use this?" The jump from the DOM to actually being effective is just too far for beginners, and without that reinforcement, they're likely to throw up their hands and give up.

The other argument for JavaScript-first education is that learning jQuery will leave you with blind spots in the underlying language. That sounds reasonable, but is it true? I myself didn't learn the DOM methods first--I started with jQuery, and then I picked up the lower-level functions as I went along, usually as a way to write more efficient event listeners or work with XML. I would have a hard time pointing out any ways in which learning jQuery held me back from learning the language.

I suspect the opposite is actually true: jQuery serves as a better introduction to modern JavaScript than the DOM methods do. Over the last few years, the way people write JavaScript has evolved rapidly: greater use of closures and functional coding, creative abuse of objects as hashes, and a variety of inheritance methods. Using the DOM will not help you learn these common patterns, while jQuery relies on them heavily. Teaching the DOM methods first means less time to cover the many uses of first-class functions, meaning that students who go on to use underscore.js or d3.js (among others) are likely to be completely baffled by the code style they see there.

Ultimately the call to teach "real JavaScript" first is exactly the kind of macho posturing that's far too common throughout tech culture, like saying that only assembly programmers are "real programmers" and only people who build their own computers "really understand" them. It's ridiculous there, and it's still ridiculous in the browser. No-one would suggest starting a basic programming class by introducing the Win32 COM API, but we're supposed to force JavaScript programmers to learn at the lowest, least-useful level of abstraction? Most tellingly, if you asked people who are advocating for DOM methods, I have no doubt that they're all using jQuery (or another library) when it comes time to get work done.

As with any subject, foundation is important. But that doesn't always make it the right starting place. When teaching bass, for example, we wouldn't make someone master chord theory before we taught them a simple scale. Instead, we can start at one level of abstraction and work both up and down. It's the same when coding: by starting people with jQuery, I suspect it's easier to get them into the DOM later. We can still teach the fundamental patterns of the language without being caught up in the implementation of a bad API.

In December, jQuery creator John Resig (who is now working as a coder for self-education startup Khan Academy) wrote a post about teaching JavaScript as a first programming language. I'm a big fan of JavaScript, and by that point I knew I was going to be teaching it to all-new programming students, so I read it with interest. As Resig notes, there's a lot going for JavaScript as a starter language, mainly in the sense that it is ubiquitous, in high demand from employers, and easy to share.

But after spending some time teaching it, I'm not sure that JavaScript is actually a good language for first-time programmers at all--and it has little to do with the caveats that Resig raises, like its falsy values or block scoping. Those are tricky for people who have previous programming experience, because they don't work the way that other languages do. But students who have never programmed before have more fundamental problems that they're trying to wrap their heads around, and JavaScript (in the browser, at least) adds unnecessary complications to the process. My students so far have had the most trouble with:

• Loops - JavaScript has two kinds, one for objects and one for arrays, and they're both confusing for beginners.
• Input - apart from prompt(), there's no standard input. Learning to write "real" inputs means dealing with event listeners, which are complicated.
• Output - There's document.write(), or alert(), or console.log()... or the DOM. I don't know which is worse.
• The difference between defining a function and running a function - including figuring out what arguments mean, and why we use them.

And don't get me started on the DOM. As a complex, tree-shaped data structure, it's already tough enough to teach students how to traverse and alter it. When you start getting into the common ways that JavaScript interacts with data stored in HTML and CSS, even in relatively simple ways, it becomes extremely confusing for people who have never tried to write a program before--particularly if they're not very experienced with HTML/CSS in the first place. Now they're effectively learning three languages, each with its own quirks, just to do basic things.

This quarter I taught Intro to Programming using JavaScript. I built a simple scratchpad for students to use, so they wouldn't have to install any browser tools or understand HTML/CSS, but the result felt clunky--a lot of work to monkey-patch flaws in the language for the purposes of learning. Next quarter, while there's little I can do about all-new programmers who jump right into Intro to JavaScript, I do plan on using Processing for Intro to Programming itself. While the language syntax is similar to JavaScript, the all-in-one nature of the Processing environment--an editor with a built-in, always-visible console and a big "play" button for running the current sketch--means that I can spend my time teaching the fundamentals of coding, instead of explaining (or hiding) the idiosyncrasies of scripting in a browser.

As a web developer, it's easy to get the feeling that the browser makers are out to get you (the standards groups definitely are). The latest round of that sinking feeling comes from WebGL which is, as far as I can tell, completely insane. It's a product of the same kind of thinking that said "let's literally just make SQLite the web database standard," except that for some reason Mozilla is going along with it this time.

I started messing with WebGL because I'm a graphics programmer who never really learned OpenGL, and that always bothered me a little. And who knows? While I love Flash, the idea of hardware-accelerated 3D for data visualization was incredibly tempting. But WebGL is a disappointment on multiple levels: it's completely alien to JavaScript development, it's too unsafe to be implemented across browsers, and it's completely out of place as a browser API.

A Square Peg in a JavaScript-shaped Hole

OpenGL was designed as an API for C a couple of decades ago, and even despite constant development since then it still feels like it. Drawing even a simple shape in OpenGL ES 2.0 (the basis for WebGL) requires you to run some inscrutable setup functions on the GL context using bit flags, assemble a shader program from vertex and fragment shaders written in a completely different language (we'll get to this later), then pass in an undistinguished stream of vertex coordinates as a flat, 1D array of floating point numbers. If you want other information associated with those vertices, like color, you get to pass in another, entirely separate, flat array.

Does that sound like sane, object-oriented JavaScript? Not even close. Yet there's basically no abstraction from the C-based API when you write WebGL in JavaScript, which makes it an incredibly disorienting experience, because the two languages have fundamentally different design goals. Writing WebGL requires you to use the new ArrayBuffer types to pack your data into buffers for the GL context, and acquire "pointers" to your shader variables, then use getter and setter functions on those pointers to actually update values. It's confusing, and not much fun. Why can't we just pass in objects that represent the vertexes, with x, y, z, vertex color, and other properties? Why can't the GL context hand us an object with properties matching the varying, uniform, and attribute variables for the shaders? Would it kill the browser, in other words, to pick up even the slightest amount of slack?

Now, I know that API design is hard and probably nobody at Mozilla or Google had the time to code an abstraction layer, not to mention that they're all probably old SGI hackers who can write GL code in their sleep. But it cracks me up that normally browser vendors go out of their way to reinvent the wheel (WebSockets, anyone?), and yet in this case they just threw up their hands and plunked OpenGL into the browser without any attempt at impedance matching. It's especially galling when there are examples of 3D APIs that are intended for use by object-oriented languages: the elephant in the room is Direct3D, which has a slightly more sane vertex data format that would have been a much better match for an object-oriented scripting language. Oh, but that would mean admitting that Microsoft had a good idea. Which brings us to our second problem with WebGL.

Unsafe at Any Speed

Microsoft has come right out and said that they won't add WebGL to IE for security reasons. And although they've caught a lot of flack for this, the fact is that they're probably right. WebGL is based on the programmable pipeline version of OpenGL, meaning that web developers write and deliver code that is compiled and run directly on the graphics card to handle scene transformation and rendering. That's pretty low-level access to memory, being granted to arbitrary people on the Internet, with security only as strong as your video driver (a huge attack surface that has never been hardened against attack). And you thought Flash was a security risk?

The irony as I see it is that the problem of hostile WebGL shaders only exists in the first place because of my first point: namely, that the browsers using WebGL basically just added direct client bindings to canvas, including forcing developers to write, compile, and link shader programs. A 3D API that was designed to abstract OpenGL away from JavaScript developers could have emulated the old fixed-function pipeline through a set of built-in shaders, which would have been both more secure and dramatically easier for JavaScript coders to tackle.

Distraction via Abstraction

Most programming is a question of abstraction. I try to train my team members at a certain level to think about their coding as if they were writing an API for themselves: write small functions to encapsulate some piece of functionality, test them, then wrap them in slightly larger functions, test those, and repeat until you have a complete application. Programming languages themselves operate at a certain level of abstraction: JavaScript hides a number of operating details from the developer, such as the actual arrangement of memory or threads. That's part of what makes it great, because managing those things is often a huge pain that's irrelevant to making your web application work.

Ultimately, the problem of WebGL is one of proper abstraction level. I tend to agree with JavaScript developer Nicolas Zakas that a good browser API is mid-level: neither so low that the developer has to understand the actual implementation, nor so high that they make strong assumptions about usage patterns. I would argue that WebGL is too low--that it requires developers to effectively learn a C API in a language that's unsuited for writing C-style code, and that the result is a security and support quagmire.

In fact, I suspect that the reason WebGL seems so alien to me, even though I've written C and Java code that matched its style in the past, is that it's actually a lower-level API than the language hosting it. At the very minimum, a browser API should be aligned with the abstraction level of the browser scripting language. In my opinion, that means (at the very least) providing a fixed-function pipeline, using arrays of native JavaScript objects to represent vertex lists and their associated data, and providing methods for loading textures and data from standard image resources.

In Practice (or, this is why I'm still a Flash developer)

Let me give an example of why I find this entire situation frustrating--and why, in many ways, it's a microcosm of my feelings around developing for so-called "HTML5." Urban Artistry is working on updating our website, and one of the artistic directors suggested adding a spinning globe, with countries where we've done international classes or battles marked somehow. Without thinking too much I said sure, I could do that.

In Flash, this is a pretty straightforward assignment. Take a vector object, which the framework supports natively, color parts of it, then either project that onto the screen using a variation of raycasting, or actually load it as a texture for one of the Flash 3D engines, like PaperVision. All the pieces are right there for you, and the final size of the SWF file is probably about 100K, tops. But having a mobile-friendly site is a new and exciting idea for UA, so I thought it might be nice to see if it could be done without Flash.

In HTML, I discovered, fewer batteries come included. Loading a vector object is easy in browsers that support SVG--but for Android and IE, you need to emulate it with a JavaScript library. Then you need another library to emulate canvas in IE. Now if you want to use WebGL or 3D projection without tearing your hair out, you'd better load Three.js as well. And then you have to figure out how to get your recolored vector image over into texture data without tripping over the brower's incredibly paranoid security measures (hint: you probably can't). To sum up: now you've loaded about half a meg of JavaScript (plus vector files), all of which you have to debug in a series of different browsers, and which may not actually work anyway.

When faced with a situation like this, where a solution in much-hated Flash is orders of magnitude smaller and easier to code, it's hard to overstate just how much of a setback HTML development can be--and I say that as someone who has grown to like HTML development much more than he ever thought possible. The impression I consistently get is that neither the standards groups nor the browser vendors have actually studied the problems that developers like me commonly rely on plugins to solve. As a result, their solutions tend to be either underwhelming (canvas, the File APIs, new semantic elements) or wildly overcomplicated (WebGL, WebSQL, Web Sockets, pretty much anything with a Web in front of it).

And that's fine, I'll still work there. HTML applications are like democracy: they're the worst platform possible, except for most of the others. But every time I hear someone tell me that technologies like WebGL make plugins obsolete, my eyes roll so hard I can see my optic nerve. The replacements I'm being sold aren't anywhere near up to the tasks I need to perform: they're harder to use, offer me less functionality and lower compatibility, and require hefty downloads to work properly. Paranoid? Not if they're really out to get me, and the evidence looks pretty convincing.

Traveling in the modern age, no matter the distance, can be partially described as "the hunt for the next electrical outlet." The new stereo in our car has a USB port, which was a lifesaver, but each night played out similarly during our cross country trip: get into the hotel, let out the cat, comfort the dog, feed both, and then ransack every electrical socket in the place to recharge our phones and laptops.

I don't drive across the entire southern USA often, but I do spend a lot of time on public transit. So I've come to value devices that don't require daily charging, and avoid those that do. Until recently, I would have put my Kindle 2 in the former category, but over the last year, it has started discharging far too quickly--even after a battery replacement. There's clearly something wrong with it.

It's possible that the problem is simply three years of rough handling and being carried around in a messenger bag daily. Maybe something got bent a little bit more than the hardware could handle. I suspect that it's gotten stuck on some kind of automated process, indexing maybe, and as such it's not going to sleep properly. That has led to all kinds of voodoo trouble-shooting: deleting random books, disabling collections, even running a factory reset and reloading the whole thing. Has any of it worked? Um... maybe?

The thing is, I don't know. I don't know for certain what's wrong, or how to fix it, or if any of my ad-hoc attempts are doing any good. There's a huge void where knowledge should be, and it's driving me crazy. This is one of the few cases where I actually want to spend time providing my own tech support (instead of needlessly buying a whole new device)--and I can't. Because the Kindle, even though it's a Linux device under the hood, is an almost totally closed system. There's nowhere that I, or anyone else, can look for clues.

I was reminded of how much I've come to rely on open systems, not just by my Kindle worries, but by checking out some of the source code being used for Occupy Wall Street, particularly the OpenWRT Forum. That's a complete discussion forum written to run on a Linksys WRT wireless router--the same router I own, in fact. Just plug it into the wall, and instantly any wifi device within range can share a private space for discussion (handy, when you're not allowed to have amplified sound for meetings).

The fact that some enterprising coders can make a just-add-electricity collaboration portal out of off-the-shelf electronics owes everything to open source, albeit reluctantly. Linksys was forced, along with a lot of other router manufacturers, to release the source code for their firmware, since it was based on GPL code. That meant lots of people started messing with these boxes, in some cases to provide entirely new functionality, but in some cases just to fix bugs. That's how I ended up with one, actually: after we had a router crash repeatedly, I went looking for one that would have community support, as opposed to the original Linksys "release and forget" approach to customer service.

When something as faceless as a router can get new life, and clever new applications, through open source, it makes it all the more galling that I'm still having to play guessing games with my Kindle's battery issues. Indeed, it's frustrating that these "post-PC" closed systems are the way so many companies seem hellbent on pursuing: closed software running on disposable, all-in-one hardware. The result is virtually impossible to repair, at either the software or the physical level.

I'm not asking for open source everything--your microwave is safe. I'm not even necessarily arguing for strictly open source: I still run Windows at home, after all. I don't want to hack things for no good reason. What I think we deserve is open access, a backdoor into our own property. Enough of our lives are black boxes already. Ordinary people don't need to see it or use it, but if something goes wrong, the ability to at least look for an answer shouldn't be too much to ask.

I spend a lot of time at work straddling four programming languages: PHP, SQL, JavaScript, and ActionScript. Many of our projects use at least three of these, if not all four. Yet while there's certainly some degree of domain-specific knowledge in each, there's more technique shared between them, floating off in the indefinite space of "software engineering."

Granted, I didn't study computer science in college. I had done some programming before and didn't really want anything to do with it professionally--I wanted to work for the Travel Channel! So when I fell into doing data journalism for CQ, a job that's halfway between storytelling and interactive coding, I knew there were skills where I was probably behind. And now that I feel like I'm relatively up to speed on the languages themselves, I want to catch back up on some of what I missed, starting with various low-level data structures.

The result is Typedefs, a simple blog where, in each entry, I pick an item from Wikipedia's list of data structures, implement it in JavaScript, and then explain how I did it and provide a quick demonstration. So far, I've done linked lists (the old classic), AA trees, and heaps. Next I want to try a chunk-based file, like PNG, and also a trie or bloom filter for text lookup.

I can already tell that working through these examples has been good for me--not because I expect to implement a lot of AA trees, because in my experience that's pretty rare, but because building these structures gives me a better understanding of how languages actually work, and a wider range of algorithms for solving other problems. The mechanics of a heap, for example, define a set of interesting ways to use arrays for storage and processing. AA trees really force you to examine the implications of pass-by-reference and pass-by-value. Linked lists are always a good experiment in miniature API design. As bite-sized, highly-technical exercises, they give me a chance to stretch my skills without having to build a full-sized JavaScript application.

These posts are also intended to leverage a truism: that the best way to learn is to teach someone else. By writing the blog as a teaching tool for other people, it forces me to organize my thoughts into a logical, coherent narrative: what are the foundations of this structure? What's it actually doing during this algorithm? Why is this useful? When the goal is to educate, I can't just get away with refactoring someone else's example. I need to know how and why it's built that way--and that knowledge is probably more useful than the example itself.

After a month of much weeping and gnashing of teeth, my venerable laptop is back from out-of-warranty repair. That month was the longest I've been without a personal computer in years, if not decades, but I've survived--giving me hope for when our machine overlords rise up and replace Google Reader with endless reruns of The Biggest Loser.

After the second day, I felt good. In fact, I started to think that computing without a PC was doable. I do a lot of work "in the cloud," after all: I write posts and browse the web using Nano and Lynx via terminal window, and my social networks are all mobile-friendly. Maybe I'd be perfectly happy just using my smartphone for access--or anything that could open an SSH session, for that matter! Quick, fetch my Palm V!

Unfortunately, that was just the "denial" stage. By the second week of laptoplessness, my optimism had faded and I was climbing the walls. It's a shame, but I don't think cloud computing is there yet. As I tried to make it through a light month of general tasks, I kept running into barriers to acceptance, sorted into three categories: the screen, the keyboard, and the sandbox.

The Screen

Trying to do desktop-sized tasks in a browser immediately runs into problems on small-screen devices. It's painful to interact with desktop sites in an area that's under 13" across. It's even more annoying to lose half of that space for a virtual keyboard. When writing, the inability to fit an entire paragraph onscreen in a non-squint font makes it tougher to write coherently. I probably spend more time zooming in and out than actually working.

But more importantly, the full-screen application model is broken for doing real work. That sounds like nerd snobbery ("I demand a tiling window manager for maximum efficiency snort"), but it's really not. Consider a simple task requiring both reading and writing, like assembling a song list from a set of e-mails. On a regular operating system, I can put those two tasks side by side, referring to one in order to compile the other. But on today's smartphones, I'm forced to juggle between two fullscreen views, a process that's slow and clumsy.

There's probably no good way to make a multi-window smartphone. But existing tablets and thin clients (like Chromebooks) are also designed around a modal UI, which makes them equally impractical for tasks that involve working between two documents at the same time. I think the only people who are even thinking about this problem are Microsoft with their new Windows 8 shell, but it's still deep in development--it won't begin to influence the market for at least another year, if then.

The Keyboard

I bought a Thinkpad in the first place because I'm a sucker for a good keyboard, but I'm not entirely opposed to virtual input schemes. I still remember Palm's Graffiti--I even once wrote a complete (albeit terrible) screenplay in Graffiti. On the other hand, that was in college, when I was young and stupid and spending a lot of time in fifteen-passenger vans on the way to speech tournaments (this last quality may render the previous two redundant). My patience is a lot thinner now.

Input matters. A good input method stays out of your way--as a touch-typist, using a physical keyboard is completely effortless--while a weaker input system introduces cognitive friction. And what I've noticed is that I'm less likely to produce anything substantial using an input method with high friction. I'm unlikely to even start anything. That's true of prose, and more so for the technical work I do (typical programming syntax, like ><{}[];$#!=, is truly painful on a virtual keyboard).

Proponents of tablets are often defensive about the conventional wisdom that they're oriented more towards consumption, less toward creativity. They trumpet the range of production software that's available for making music and writing text on a "post-PC" device, and they tirelessly champion whenever an artist uses one to make something (no matter how many supporting devices were also used). But let's face it: these devices are--as with anything--a collection of tradeoffs, and those tradeoffs almost invariably make it more difficult to create than to consume. Virtual keyboards make it harder to type, interaction options are limited by the size of the screen, input and output can't be easily expanded, and touch controls are imprecise at best.

Sure, I could thumb-type a novel on my phone. I could play bass on a shoebox and a set of rubber bands, too, but apart from the novelty value it's hard to see the point. I'm a pragmatist, not a masochist.

The Sandbox

I almost called this section "the ecosystem," but to be honest it's not about a scarcity of applications. It's more about process and restriction, and the way the new generation of mobile operating systems are designed.

All these operating systems, to a greater or lesser extent, are designed to sandbox application data from each other, and to remove the heirarchical file system from the user. Yes, Android allows you to access the SD card as an external drive. But the core applications, and most add-on applications, are written to operate with each other at a highly-abstracted level. So you don't pick a file to open, you select an image from the gallery, or a song from the music player.

As an old PalmOS user and developer, from back in the days when they had monochrome screens and ran on AAAs, this has an unpleasantly familiar taste: Palm also tried to abstract the file system away from the user, by putting everything into a flat soup of tagged databases. Once you accumulated enough stuff, or tried to use a file across multiple applications, you were forced to either A) scroll through an unusably lengthy list that might not even include what you want, or B) run normal files through an external conversion process before they could be used. The new metaphors ('gallery' or 'camera roll' instead of files) feel like a slightly hipper version of the PalmOS behavior that forced me over to Windows CE. We're just using Dropbox now instead of Hotsync, and I fail to see how that's a substantial improvement.

Look, heirarchical file systems are like democracy. Everyone agrees that they're terrible, but everything else we've tried is worse. Combined with a decent search, I think they can actually be pretty good. It's possible that mobile devices can't support them in their full richness yet, but that's not an argument that they've gotten it right--it shows that they still have a lot of work to do. (The web, of course, has barely even started to address the problem of shared resources: web intents might get us partway there, one day, maybe.)

The Future

When I was in high school, Java had just come out. I remember talking with some friends about how network-enabled, platform-independent software would be revolutionary: instead of owning a computer, a person would simply log onto the network from any device with a Java VM and instantly load their documents and software--the best of thin and thick clients in one package.

Today's web apps are so close to that idea that it kind of amazes me. I am, despite my list of gripes, still optimistic that cloud computing can take over much of what I do on a daily basis, if only for environmental reasons. My caveats are primarily those of form-factor: it's technically possible for me to work in the cloud, but the tools aren't productive, given the recent craze for full-screen, touch-only UIs.

Maybe that makes me a holdover, but I think it's more likely that these things are cyclical. It's as though on each new platform, be it mobile or the browser, we're forced to re-enact the invention of features like windows, multitasking, application management, and the hard drive. Each time, we start with the thinnest of clients and then gradually move more and more complexity into the local device. By the time "the cloud" reaches a level where it's functional, will it really be that different from what I'm using now?

Patents are important. They create an incentive for inventors to release their inventions into the public domain in return for a temporary monopoly on those inventions. Everybody loves patents. Software patents, on the other hand, must die.

Lately there has been a lot of news about patents, as almost every mobile company in existence is suing every other company for infringement. That many happy lawyers should have been a warning sign that something fishy was going on. And this being the Internet, everyone online wants to leap to the defense of their favorite. But ultimately, there are no winners in this game, because software patents have done nothing but harm innovation. They're counter-productive, anti-competitive, and insulting.

The most well-written defense of software patents that I've seen comes from Nilay Patel, who is a former lawyer, and focuses mainly on why patents in general are beneficial to society. I don't have any issues with that. But let's examine his claims on software patents in particular, as a way of getting at why they're broken. He writes:

Patents publicly disclose some of the most advanced work ever done by some of the most creative and resourceful people in history, and it'll all be free for the taking in several years. Stop offering patent protection and there's no more required disclosure -- all this stuff stays locked up as trade secrets as long as it offers a competitive advantage, after which point it may well be forgotten.

...

What might surprise you is that the USPTO has historically resisted efforts to patent software, only to have the courts chart a different course. The conflict itself is a simple one: software is ultimately just the automated expression of various algorithms and math, and you can't patent math. To many, it then seems like a forgone conclusion that software patents shouldn't exist -- preventing other people from using math is solidly outside the boundaries of the patent system. The problem isn't software patents -- the problem is that software patents don't actually exist.

But look a little closer and it's easy to see that the boundaries between "just math" and "patentable invention" are pretty fuzzy. Every invention is "just math" when it comes right down to it -- traditional mechanical inventions are really just the physical embodiments of specific algorithms.

That last bit, the part about how everything is "just math" (Patel uses the example of a special beer bottle shape), is where this argument runs into trouble (well, that and the point where he admits that he's neither a patent lawyer nor a developer, meaning that he brings almost no actual expertise to a highly-technical table). As Patel is not a developer, it's not immediately apparent to him the difference between pages of source code and a physical mechanism. But they are, in fact, very different: computer programs are, at heart, a series of instructions that the device follows. They're more like a recipe than a machine. Can you imagine patenting a recipe--claiming to have "invented" a type of bread, or a flavor of pie, and insisting that anyone else making an apple pie must pay your licensing fees? It's ridiculous.

To add insult to injury, what most programmers will tell you is that the majority of software is not "some of the most advanced work ever done," but a series of obvious steps connected together. Take, for example, Patel's own example, drawn from Apple's multitouch patent.

You may think that looks really complicated--Patel clearly does--but it's really not. That chunk of text, once you pull it apart and turn the math into actual working code, reduces to three steps:

1. Use the Pythagorean theorum to determine the distance between two arbitrary points.
2. Repeat step one.
3. Divide the difference between the distances found in steps 1 and 2 by the elapsed time between them, thus determining the speed at which your arbitrary points are moving closer or farther away.

Now, Patel is careful to say that this is not the complete patent. But I've looked over the complete patent text, and frankly there's little I see there that goes beyond this level of complexity. It describes the kind of common-sense steps that anyone would take if they were implementing a multitouch UI (as well as a number of steps that, as far as I can tell, no-one--including the patent-holder--has actually implemented, such as distinguishing between a palm or elbow and a stylus). There's nothing novel or non-obvious about it--it's just written in so much legalese that it looks extremely complicated.

Such obfuscation overturns the entire argument for patents in the first place. After all, if I were trying to code something, there's no way I'd turn to a software patent like this to figure out how to implement it--it leaves out too many steps, overcomplicates those that it does describe, adds a bunch of extraneous notes that I have to discard, and includes no source code at all. It'd be easier for me to figure it out myself, or learn from an open-source implementation.

Moreover, if I worked for a big company, like Apple or Microsoft, I'd be forbidden from reading patents in the first place, because the penalties for "knowing" infringement are much higher than "unknowing" infringement (a particularly ironic term, given that someone can "unknowingly" infringe by inventing the process independently). Meanwhile, as Lukas Mathis points out, we don't actually need the patent for an actually innovative piece of programming (Google's original PageRank) because it was published openly, as an academic paper.

I'm not saying, by the way, that multitouch hardware can't--or shouldn't--be patented. By all means, the inventor of the capacitative screen--a non-obvious, physical work of engineering--should be rewarded for their creativity. But that's not what Patel is defending, and it's not what the patent in question actually covers.

So software patents don't cover brilliant work, and they're not encouraging innovation. In fact, they discourage it: thanks to patent trolls who sue small companies over broad software patents, indie developers may not be able to afford the cost of business at all. If this seems difficult to reconcile with the arguments for the existence of software patents, that's because it undermines their entire case. Software patents are just broken: that's all there is to it.

What bothers me about this is not so much the ongoing lawsuit frenzy--I could really care less when large companies sue each other as a proxy for their market competition--but that there's this idea floating around that some software patent lawsuits are worthwhile, based on who's doing the suing. Everyone agrees that trolls like Lodsys (a shell company suing small developers over a purported patent on "in-app purchases") are a drain on the system. Whereas if Apple, or Microsoft, or Nokia files a patent suit, their fans charge to their defense against "copycats," as if their patents are the only really innovative ones in existence. Even this week, pro-Apple pundit John Gruber cheered Google's action against Lodsys while simultaneously insisting that Android and Linux should be sued out of existence for patent theft.

But there are no justified software patent lawsuits, because there are no justified software patents. You don't get to say that patent trolls are despicable, while defending your favorite company's legal action as a valid claim--it's the same faulty protection racket at work. Software patents are a plague on all our houses, and tribalism is no excuse for preserving them.

After the LulzSec hacking rampage, I finally found the motivation to do something I've been putting off for a long time: I switched to more secure passwords using password management software, so that I'm not using the same five passwords everywhere anymore. Surprisingly, it was a lot less painful than I thought it would be.

Similar to what Brinstar did, I'm using KeePass to store my passwords--I don't want to pay for a service, and I don't really like using closed-source tools for this kind of thing. But since I'm not feeling incredibly confident about Dropbox for secure materials right now (less because they've admitted being able to open your files to the government, more because they left the whole system wide open for four hours the other day), I'm not using them to store my database.

Instead, I'm taking advantage of the fact that Android phones act like USB hard drives when they're plugged into the computer. The 1.X branch of the KeePass desktop client is a portable executable, so it can run from the phone's memory card and use the same database as KeePassDroid. If I need a password from a real computer, I can just plug in my phone. I do keep a backup of the encrypted database uploaded to Google Docs storage, but that's behind two-factor authentication, so I think it's reasonably safe.

It's shallow of me, but for a long time I held off on this move because the screenshots on the KeePassDroid site are incredibly ugly. Fortunately, those are out of date. It's still not quite as attractive as alternative like Pocket or Tiny Password, but with the group font size turned down it can pass for "functional." And I like that it's not dependent on a third-party cloud provider like those are (Pocket has a client, and it's even in cross-platform Java, but it doesn't expose its database for USB access). I don't know if the author will take my patches, but I've submitted a few changes to the KeePassDroid layouts that make it look a little bit less "open source."

So what's the point? A password manager does almost nothing to keep my local data safe, or to protect me if someone steals my laptop with its cached passwords in Firefox. On the other hand, a common weakness in recent hacking incidents has been the use of shared (often weak) passwords across sites, so that if one falls the others go as well. Now my passwords are stronger, but more importantly, they're different from site to site. If someone acquires my Facebook login info, for example, that no longer gives them credentials to get into anything else.

It all comes down to the fact that I can secure my own data, but once it goes out on the web, I'm at the mercy of random (probably untrained) server administrators. That does not fill me with confidence, and it should probably make you a little uneasy as well. If so, my advice based on this experience would be to go ahead and make the switch to some kind of password-management system. Like keeping good backups, it's not nearly as hard as it sounds, and it'll be time well spent when the script kiddies strike again.

Summer is here, bringing with it 100° weather and a new series of CQ data projects--which, in turn, means working with Excel again. Here is a list of all the things I hate about Excel:

1. Dates.

Why have I rediscovered my enthusiasm for Excel? It's kind of funny, actually. In the past couple of years, fueled by a series of bizarre experiments in Visual Basic scripting, I've often solved spreadsheet dilemmas using brute-force automation. But now that I'm working more often with a graphics reporter who uses the program on OS X, where it no longer supports scripting, I'm learning how to approach tables using the built-in cell functions (the way I probably should have done all along). The resulting journey is a series of elegant surprises as we dig deeper into the Excel's capabilities.

I mean, take the consolidate operation and the LOOKUP function. If I had a dime for every time I'd written a search-and-sum macro for someone that could have been avoided by using these two features, I'd have... I don't know, three or four bucks, at least. Consolidate and LOOKUP are a one-two combo for reducing messy, unmatched datasets into organized rows, the kind of information triage that we need all too often. I've been using Excel for years, and it's only now that I've discovered these incredibly useful features (they're much more prominent in the UI of the post-Ribbon versions, but the office is still using copies of Excel 2000). It's tremendously exciting to realize that we can perform these kinds of analysis on the fly, without having to load up a full-fledged database, and that we're only scratching the surface of what's possible.

I find that I don't miss the challenge of coding in Excel, because formula construction scratches the same problem-solving itch. Besides, spreadsheets are also programs, of a sort. They may not be Turing-complete, but they mix data and code in much the same way as a binary program, they have variables and "pointers" (cell references), and they offer basic input and output options. Every cell formula is like its own little command line. Assembling them into new configurations offers the same creative thrill of a programming task--at smaller doses, maybe, but in a steady drip of productivity.

But honestly, efficiency and flexibility are only part of my affection for Excel. I think on some level I just really like the idea of a spreadsheet. As a spacially-oriented thinker, the idea of laying out data in a geometric arrangement is instantly intuitive to me--which, for all that I've grown to like SQL, is not something I can say for relational database queries. "We're going to take some values from over there," says Excel, "and then turn them into new values here." A fully-functioning spreadsheet, then, is not just a series of rows and columns. It's a kind of mathematical geography, a landscape through which information flows and collects.

By extension, whenever I start up Excel and open up a new sheet, the empty grid is a field of undiscovered potential. Every cell is a question waiting to be asked and answered. I get paid to dive in and start filling them up with information, see where they'll take me, and turn the results into stories about the world around us. How could anyone not find that thrilling? And how could you not love a tool that makes it possible?