this space intentionally left blank

August 15, 2012

Filed under: tech»education

VM Where?

My third quarter of teaching Intro to Programming and Intro to JavaScript at SCCC ends today. Over the last eight months, I've learned a bit about what gives new programmers trouble, and how to teach around those problems. I definitely wouldn't suggest JavaScript as a first language unless the students already know HTML and CSS very well--otherwise, they're learning three languages at once, and that's more than a little overwhelming.

Outside of class I've also started to daydream about ways to teach the basics of programming, not just for web development, but in general. I think a simpler (but still dynamic) language is probably best for beginners--even though I think its whitespace model is insane, Python (or its cheery little cousin Ruby) would probably be a good option. It's got a good library, not a lot of punctuation or syntax, and it would get people to indent their code (which, for some reason that I can't understand, is like pulling teeth no matter how many times you show people that it makes the code more readable). More importantly, it's straightforward imperative code--none of the crazy functional malarkey that emerges, marmot-like, whenever JavaScript starts doing any kind of user interaction or timing. And people actually use Python, so it's not like you're teaching them Lisp or something.

But let's pretend we weren't bound by the idea of "teach skills that are directly marketable"--i.e., let's pretend I'm not working for a community college (note: there's nothing wrong with teaching marketable skills, it's just a thought exercise). What's a good way to introduce people to the basic problems of programming, like looping and conditionals and syntax?

What about assembly?

Let's go ahead and get some reasons out of the way as to why you shouldn't teach programming with assembly. First, it offers no abstractions or standard libraries, so students won't be learning any immediately-useful skills for outside the classroom. It's architecture-specific, meaning they probably can't even take it from one computer to another. And of course, assembly is not friendly. It doesn't have nice, easy-to-type keywords like "print" and "echo" and "document.getElementById" (okay, so that's not all bad).

What you gain, given the right choice of architecture, is simplicity. Assembly does not give you syntax for loops, or for functions. It doesn't give you structures. You get some memory, a few registers to serve as variables, and some very basic control structures. It's like BASIC, but without all the user-friendliness. Students who have trouble keeping track of what their loops are doing, or how functions work, might respond better to the simple Turing tape-like flow of assembly.

But note that huge caveat: given the right choice of architecture. Ideally, you want a very short set of instructions, so students don't have to learn very much. That probably rules out x86. 6502 has a reasonably small set of opcodes, but they're organized in that hilarious table depending on what goes where and what addressing scheme you're in, which is kind of crazy. 68000 looks like a bigger version of 6502 to me. Chip-8 might work, but I don't really like the sprite system. I'd rather just have text output. What we need is an artificial VM designed for teaching--something that's minimal but fun to work with.

That's why I'm really interested in 0x10c, the space exploration game that's in development by Notch, the creator of Minecraft. The game will include an emulated CPU to run ship functions and other software, and it's programmed in a very simple, friendly form of assembly. There are a ton of people already writing tools for it, including virtual screen access--and the game's not even out yet. It's going to be a really great tool for teaching some people how computers work at a low level (in a simplified model, of course).

I'm not the first person to wonder if assembler could make a decent teaching tool. And I'm not convinced it actually would--it's certainly not going to do anything but confuse and frustrate my web development students. But I do think programming instruction requires a language that, against some conventional wisdom, isn't too high level. JavaScript's functions are great, but they're too abstract to translate well for most students--this and the DOM form a serious conceptual barrier to entry. On the other hand, students need to feel effective to keep their attention, meaning that it shouldn't take 50 lines of opcodes to print "hello, world." That flaw may make assembly itself ineffective, but thinking about what it does teach effectively may give us a better frame of reference for teaching with other toolkits.

August 8, 2012

Filed under: tech»green


The Thinkpad Hardware Maintenance Manual is kind of amazing. It is specifically laid out to walk someone through the process of disassembling a laptop, all the way down to the pads on the motherboard, and including (so helpful!) the types of screws used at each step of the way. You can fix pretty much anything on a Thinkpad with the HMM and the right spare parts. I know this now because I have been through the entire thing forward and backward.

About two months ago, I fried my motherboard (literally) by replacing the cooling fan with one meant for a laptop with integrated graphics, instead of the Nvidia chip that was actually installed. At 105°C, it wasn't long before the chip melted itself down into a puddle of silicon and coltan. I was able to eke out a few more days of use by turning off anything that could possibly require acceleration--Flash, videos of any kind, Aero Glass--but soon enough it just gave up on me completely.

What's frustrating about this is that you can't simply replace the shiny patch of metal that used to be a GPU on a laptop. Even though Lenovo refers to the Nvidia option as "discrete" graphics, it's actually soldered to the motherboard (perhaps they mean it won't gossip about you, which is probably true, because in my experience Nvidia's mobile chips don't live long enough to learn anything embarrassing anyway). So your only repair option is to replace the entire system board: about $600 with parts and labor, assuming you take it to some place where they know what they're doing, which is not a guarantee. Not that I'm apparently any better, but at least I only destroy my own property.

Stil, given the comprehensive hardware maintenance manual, a used motherboard from eBay, and a spare pill case to hold the screws, fixing the laptop turned out to be no big deal--I even converted it to the Intel graphics chip, which is less likely to overheat since it's got all the horsepower of a Nintendo Super FX chip. The experience has left me impressed with Lenovo's engineering efforts. The construction is solid without "cheating"--it's honestly pretty easy to open it up, and there's no glue or clips that only the manufacturer can remove/replace. As an inveterate tinkerer, I believe everything should be built this way.

But in that, at least, Lenovo and I am in the minority. Increasingly (led by Apple), laptops and other computing devices are sealed boxes with few or no user-serviceable parts. Want to upgrade your RAM or hard drive? Too bad, it's soldered in place. Break the screen? Better send it to the factory (or not, I guess). If a video card burns out on its own (as the Nvidia 140M chips did, repeatedly, even before I started tinkering with the cooling system) or another hardware problem occurs, they don't want you to fix it. They want you to buy another, most likely throwing the old one away. Even the Thinkpad, when it's outpaced by more demanding software, can only be upgraded so far. That means it also has a limited lifespan as a donation to a school or needful friend.

So in addition to fixing my old laptop, I'm moving away from a dependence on tightly-coupled, all-in-one computing, and back to modular desktop units. I bought a Maingear F131 PC (the most basic model), which I can upgrade or repair piece by piece (I already started by adding a wireless PCI card--Maingear doesn't sell those directly). It's better for the environment, better for my wallet, and it represents a vote against wasteful manufacturing. My goal is to make it the last whole computer (but not the last hardware) I'll buy for a long, long time.

Should you do the same? I think so. Manufacturers are moving to the integrated device model because it's cheaper for them and consumers don't seem to care. Change the latter, and we can change the direction of the industry. And I think you should care: in addition to the problem of e-waste, user-replacable components help to support an entire cottage industry of small repair shops, consultants, and boutique builders--the mom-and-pop businesses of the tech world. Moving to one-piece technology will kill those, just as it killed television, vaccuum, and radio repair shops.

You can argue, I'm sure, that many people do not want to learn how to install or repair their own hardware. That's true, and they shouldn't have to. But I also deeply believe that user-servicable and ease-of-use are not mutually-exclusive qualities, nor do they require users to actually learn everything there is to know about a computer. We don't expect everyone to know how to rebuild a car engine, but I think most people would also agree that a thriving industry for aftermarket repair parts and service is a good thing--who hasn't at least had a new radio installed? Who wants to be forced into a dealership every time a filter needs to be changed?

It is tempting to see the trend toward disposable devices as one with the trend toward walled gardens in software--a way to convert users from one-time payers to a constant revenue stream. I don't actually think most companies set out to be that sinister. But it's obvious that they also won't do the right thing unless we hold their feet to the fire. The time has come to put our money where our mouths are, and insist that the environment is not a suitable sacrifice for a few extra millimeters shaved off a product spec sheet.

May 9, 2012

Filed under: tech»os

Apropos of Nothing

Over the last six months, I've consistently given three pieces of advice to my students at SCCC: get comfortable with the Firebug debugger, contribute to open source, and learn to use Linux. The first is because all developers should learn how to debug properly. The second, because open source is a great way to start building a resume. And the last, in part, because Linux is what powers a large chunk of the web--not to mention a dizzying array of consumer devices. Someone who knows how to use an SSH shell and a few basic commands (including one of the stock editors, like vi or emacs) is never without tools--it may not be comfortable, but they can work anywhere, like Macgyver. It all comes back to Macgyver, eventually.

At Big Fish, in an effort to take this to its logical extreme, I've been working in Linux full-time (previously, I've used it either on the server or through a short-lived VM). It's been an interesting trip, especially after years of using mostly Windows desktops (with a smattering of Mac here and there). Using Linux exclusively for development plays to its strengths, which helps: no fighting with Wine to run games or audio software. Overall, I like it--but I'll also admit to being a little underwhelmed.

To get the bad parts out of the way: there are something like seven different install locations for programs, apparently chosen at random; making changes in the graphical configuration still involves arcane shell tricks, all of which will be undone in hilariously awful ways when you upgrade the OS; and Canonical seems intent on removing all the things that made Ubuntu familiar, like "menus" and "settings." I ended up switching to the XFCE window manager, which still makes me angry because A) I don't want to know anything about window managers, and B) it's still impossible to theme a Linux installation so that everything looks reasonably good. Want to consistently change the color of the window decorations for all of your programs? Good luck with that. XFCE is usable, and that's about all you can honestly say for it.

The best part of Linux by far is having a native web stack right out of the box, combined with a decent package manager for anything extra you might need. Installing scripting languages has always been a little bit of a hassle on Windows: even if the base package is easily installed, invariably you run into some essential library that's not made for the platform. Because these languages are first-class citizens on Linux, and because they're just an apt-get away, it opens up a whole new world of utility scripts and web tools.

I love combining a local server with a set of rich commmand-line utilities. Finally, I can easily use tools like the RequireJS optimizer, or throw together scripts to solve problems in my source, without having to switch between contexts. I can use all of my standard visual tools, like Eclipse or Sublime Text, without going through a download/upload cycle or figuring out how to fool them into working over SFTP. Native source control is another big deal: I've never bothered installing git on Windows, but on Linux it's almost too easy.

So there is one axis along which the development experience is markedly superior. It's not that Linux is better built (it has its fair share of potholes) so much as it's where people curently go to build neat tools, and then if we're lucky they bring them over to Windows. Microsoft is trying to fix this (see: their efforts to make NodeJS a first-class Windows platform), but it'll probably always be an uphill battle. The open-source developer culture just isn't there.

On the other hand, I was surprised by the cases where web development is actually worse on Linux compared to Windows. There's no visual FTP client that's anywhere near as good as WinSCP that I can find. The file managers are definitely clumsier than Explorer. Application launching, of all things, can be byzantine--there's no option to create a shortcut to an program, you have to manually assemble a .desktop file instead, and then XFCE will invariably position its window someplace utterly unhelpful. Don't even get me started on the configuration mess: say what you like about the registry, at least it's centralized.

None of these things are dealbreakers, the same way that it's not a dealbreaker for me to need GOW for a decent Windows command line. But if I was considering trying to dual-boot or switch to Linux as a work environment, instead of just keeping a headless VM around for when I need Ruby, I've given that up now. When all is said and done, I spend much of my time in either Eclipse or Firefox anyway, and they're the same no matter where you run them. I still believe strongly that developers should learn a little Linux--it's everywhere these days!--but you can be perfectly productive without living there full time. Ultimately, it's not how you build something, but what you build that matters.

If you do decide to give it a chance, here are a few tips that have made my life easier:

  • Install a distribution to a virtual machine using VirtualBox instead of trying to use a live CD or USB stick. It's much easier to find your way along if you can still run a web browser alongside your new OS.
  • The title of this post is a reference to the apropos command, which will search through all the commands installed on your system and give you a matching list. This is incredibly helpful when you're sitting at a prompt with no idea what to type. Combined with man for reading the manual pages, you can often muddle through pretty well.
  • By default, the config editor for a lot of systems is vi. Lots of people like vi, I think it's insane, and I guarantee that you will have no idea what to do the first time it opens up unexpectedly. Eventually you'll want to know the basics, but for right now you're much better off setting nano as the default editor. Type "export EDITOR=nano" to set that up.

March 29, 2012

Filed under: tech»coding

Test Driven Testing

Next week a new quarter of Intro to JavaScript will start, and while I'm almost sick of writing about it, it's been constantly on my mind. I've talked a little here about some of the changes I'll be making--primarily I'll be teaching jQuery first--but there's another big experiment I'm going to try: using test-driven development tools to give students feedback on assignments and quizzes throughout the class. I did this in a limited way during the Winter quarter, with an assignment and an exam built using unit tests, but now I want to make it much more widespread. Why use testing this way? Two reasons: to force students to code, and to encourage them to think more critically about their mistakes.

The first reason may seem odd--they're in a programming class, shouldn't that force them to code already?--but it is really an extension of drill learning. I am not good at teaching via drills: I appreciate the value of repetition when I'm learning myself, but I hate standing in class and going through the same exercise again and again. I'd much rather give lectures about theory and application than do drills. But it's clear from last quarter that many students do not get enough practice from lectures. They may be too busy to write code at home, and assignments don't usually force them to use all the constructs of the language.

Drill and repetition are important because, just as with a spoken language, it's not enough to learn the rules of grammar. You need to go out and practice, so that the mechanics become natural and you can concentrate on expressing yourself. That's the motivation behind Zed Shaw's Learn Python the Hard Way series, and I tend to agree. Take the humble for loop, for example. I've written loops for so long I could do them in my sleep, but if you're just starting out there are a lot of moving parts that need to be grasped: for (var i = 0; i < list.length; i++) { //loop goes here } We've written a keyword (for), a series of semicolon-delimited statements inside parentheses, a block (represented by the curly braces), and we haven't even gotten to the actual loop. Just within this boilerplate, we're initializing a counter with the arbitrary name of "i" and giving it an initial value, testing it, and incrementing it. All of this is expressed in a peculiar syntax (the test and increment are tossed in after the setup statement, not placed in the order where they actually get executed), with a lot of very particular punctuation, plus a lot of bookkeeping to track mentally.

Writing for loops (or conditionals, or functions, or almost anything else) is therefore as much about learning a very specific arrangement of characters as it is about the overarching concepts. Languages can make this slightly easier to grasp (as with Python's comprehension-style loops and no-braces indentation), but I don't think you ever get away from it entirely. It is my goal this quarter to make students write as many of any given construct as possible in class, so that the low-level syntax becomes automatic, while using tests to keep the high-level goals in sight.

Which brings us to the second reason for using constant testing: students need to learn how to find mistakes by thinking critically about their output. Part of that process involves teaching them to use the developer tools: how to read a Firebug log, understand an interpreter error, and examine values in the debugger. But it also means they need to learn to check their code constantly, and to have thought about what they expect to get out of any given block. While I'll be providing tests on quizzes and homework assignments, so that students can immediately see if their answer was correct, the ultimate goal is to have them write their own simple tests to check their own code.

It's my belief that most of my students know what they want to do at the very highest level, but they're not able to break those goals into smaller pieces, and they don't know how to figure out when the smaller pieces stop working. My hypothesis is that writing tests may not be enough to create critical thinking where none exists, but it will serve as a good starting place for discussions of problem-solving strategy: what caused this test to fail? What were our expectations, and how were the results different? Was this test actually a good test to begin with, even if it "passes?" These are questions that good programmers ask, perhaps naturally, but I refuse to believe that they can't be encouraged in any student.

March 7, 2012

Filed under: tech»coding

jQuery as a Second Language

At this point, with the winter quarter at SCCC drawing to a close, I'd like to add onto my earlier comments on teaching JavaScript. I would still argue that it's not a great first language--too much reliance on other technology stacks (the browser, HTML, CSS), among other things. But John Resig's original post also talks about working around that complexity with libraries, and that speaks to a deeper divide in the JavaScript community: should newcomers learn "JavaScript," or "jQuery?" In this, I'm increasingly on Resig's side.

It's easy to understand why many JavaScript programmers are so adamant that newcomers should learn the language first. Sites like Stack Overflow are full of people whose answer to every JavaScript question is "use jQuery," even in response to simple problems that should be solved more directly. There are a lot of people out there whose idea of being a JavaScript programmer means "I can include jQuery and a few plugins on my page." Not that there's anything wrong with that.

But is the solution to teach people the DOM? That seems like cruel and unusual punishment--a kind of Protestant work ethic: real programmers suffer. The browser document object model ranks up there in the history of terrible API design: it's overly-verbose, inconsistent, and tedious. If you use it for any amount of time, you will end up recreating at least the traversal parts of jQuery--searching up and down the tree for related elements--if not the animation and CSS methods as well. Traversal isn't a hard problem per se, but as with anything involving recursion, it's kind of a lot to throw at a beginner.

In fact, it probably gets in the way of doing any real, productive learning. This quarter, caught up in the desire to do things "the right way," I taught JavaScript from the DOM methods up. As a result, we didn't get to jQuery until the sixth week--halfway through our time!--and several students asked "why would we ever use this?" The jump from the DOM to actually being effective is just too far for beginners, and without that reinforcement, they're likely to throw up their hands and give up.

The other argument for JavaScript-first education is that learning jQuery will leave you with blind spots in the underlying language. That sounds reasonable, but is it true? I myself didn't learn the DOM methods first--I started with jQuery, and then I picked up the lower-level functions as I went along, usually as a way to write more efficient event listeners or work with XML. I would have a hard time pointing out any ways in which learning jQuery held me back from learning the language.

I suspect the opposite is actually true: jQuery serves as a better introduction to modern JavaScript than the DOM methods do. Over the last few years, the way people write JavaScript has evolved rapidly: greater use of closures and functional coding, creative abuse of objects as hashes, and a variety of inheritance methods. Using the DOM will not help you learn these common patterns, while jQuery relies on them heavily. Teaching the DOM methods first means less time to cover the many uses of first-class functions, meaning that students who go on to use underscore.js or d3.js (among others) are likely to be completely baffled by the code style they see there.

Ultimately the call to teach "real JavaScript" first is exactly the kind of macho posturing that's far too common throughout tech culture, like saying that only assembly programmers are "real programmers" and only people who build their own computers "really understand" them. It's ridiculous there, and it's still ridiculous in the browser. No-one would suggest starting a basic programming class by introducing the Win32 COM API, but we're supposed to force JavaScript programmers to learn at the lowest, least-useful level of abstraction? Most tellingly, if you asked people who are advocating for DOM methods, I have no doubt that they're all using jQuery (or another library) when it comes time to get work done.

As with any subject, foundation is important. But that doesn't always make it the right starting place. When teaching bass, for example, we wouldn't make someone master chord theory before we taught them a simple scale. Instead, we can start at one level of abstraction and work both up and down. It's the same when coding: by starting people with jQuery, I suspect it's easier to get them into the DOM later. We can still teach the fundamental patterns of the language without being caught up in the implementation of a bad API.

February 23, 2012

Filed under: tech»coding

JavaScript as a Second Language

In December, jQuery creator John Resig (who is now working as a coder for self-education startup Khan Academy) wrote a post about teaching JavaScript as a first programming language. I'm a big fan of JavaScript, and by that point I knew I was going to be teaching it to all-new programming students, so I read it with interest. As Resig notes, there's a lot going for JavaScript as a starter language, mainly in the sense that it is ubiquitous, in high demand from employers, and easy to share.

But after spending some time teaching it, I'm not sure that JavaScript is actually a good language for first-time programmers at all--and it has little to do with the caveats that Resig raises, like its falsy values or block scoping. Those are tricky for people who have previous programming experience, because they don't work the way that other languages do. But students who have never programmed before have more fundamental problems that they're trying to wrap their heads around, and JavaScript (in the browser, at least) adds unnecessary complications to the process. My students so far have had the most trouble with:

  • Loops - JavaScript has two kinds, one for objects and one for arrays, and they're both confusing for beginners.
  • Input - apart from prompt(), there's no standard input. Learning to write "real" inputs means dealing with event listeners, which are complicated.
  • Output - There's document.write(), or alert(), or console.log()... or the DOM. I don't know which is worse.
  • The difference between defining a function and running a function - including figuring out what arguments mean, and why we use them.
These are common beginner hurdles in any language, but JavaScript does not make them any easier. Unlike, say, PHP, JavaScript does not offer a loop construct that can be used on any collection (foreach), and it doesn't offer an output mechanism that students will be able to rely on: the console is developer-only, and the other methods (while simple and effective) are deprecated in place of the DOM for displaying information to users.

And don't get me started on the DOM. As a complex, tree-shaped data structure, it's already tough enough to teach students how to traverse and alter it. When you start getting into the common ways that JavaScript interacts with data stored in HTML and CSS, even in relatively simple ways, it becomes extremely confusing for people who have never tried to write a program before--particularly if they're not very experienced with HTML/CSS in the first place. Now they're effectively learning three languages, each with its own quirks, just to do basic things.

This quarter I taught Intro to Programming using JavaScript. I built a simple scratchpad for students to use, so they wouldn't have to install any browser tools or understand HTML/CSS, but the result felt clunky--a lot of work to monkey-patch flaws in the language for the purposes of learning. Next quarter, while there's little I can do about all-new programmers who jump right into Intro to JavaScript, I do plan on using Processing for Intro to Programming itself. While the language syntax is similar to JavaScript, the all-in-one nature of the Processing environment--an editor with a built-in, always-visible console and a big "play" button for running the current sketch--means that I can spend my time teaching the fundamentals of coding, instead of explaining (or hiding) the idiosyncrasies of scripting in a browser.

December 9, 2011

Filed under: tech»web

Trapped in WebGL

As a web developer, it's easy to get the feeling that the browser makers are out to get you (the standards groups definitely are). The latest round of that sinking feeling comes from WebGL which is, as far as I can tell, completely insane. It's a product of the same kind of thinking that said "let's literally just make SQLite the web database standard," except that for some reason Mozilla is going along with it this time.

I started messing with WebGL because I'm a graphics programmer who never really learned OpenGL, and that always bothered me a little. And who knows? While I love Flash, the idea of hardware-accelerated 3D for data visualization was incredibly tempting. But WebGL is a disappointment on multiple levels: it's completely alien to JavaScript development, it's too unsafe to be implemented across browsers, and it's completely out of place as a browser API.

A Square Peg in a JavaScript-shaped Hole

OpenGL was designed as an API for C a couple of decades ago, and even despite constant development since then it still feels like it. Drawing even a simple shape in OpenGL ES 2.0 (the basis for WebGL) requires you to run some inscrutable setup functions on the GL context using bit flags, assemble a shader program from vertex and fragment shaders written in a completely different language (we'll get to this later), then pass in an undistinguished stream of vertex coordinates as a flat, 1D array of floating point numbers. If you want other information associated with those vertices, like color, you get to pass in another, entirely separate, flat array.

Does that sound like sane, object-oriented JavaScript? Not even close. Yet there's basically no abstraction from the C-based API when you write WebGL in JavaScript, which makes it an incredibly disorienting experience, because the two languages have fundamentally different design goals. Writing WebGL requires you to use the new ArrayBuffer types to pack your data into buffers for the GL context, and acquire "pointers" to your shader variables, then use getter and setter functions on those pointers to actually update values. It's confusing, and not much fun. Why can't we just pass in objects that represent the vertexes, with x, y, z, vertex color, and other properties? Why can't the GL context hand us an object with properties matching the varying, uniform, and attribute variables for the shaders? Would it kill the browser, in other words, to pick up even the slightest amount of slack?

Now, I know that API design is hard and probably nobody at Mozilla or Google had the time to code an abstraction layer, not to mention that they're all probably old SGI hackers who can write GL code in their sleep. But it cracks me up that normally browser vendors go out of their way to reinvent the wheel (WebSockets, anyone?), and yet in this case they just threw up their hands and plunked OpenGL into the browser without any attempt at impedance matching. It's especially galling when there are examples of 3D APIs that are intended for use by object-oriented languages: the elephant in the room is Direct3D, which has a slightly more sane vertex data format that would have been a much better match for an object-oriented scripting language. Oh, but that would mean admitting that Microsoft had a good idea. Which brings us to our second problem with WebGL.

Unsafe at Any Speed

Microsoft has come right out and said that they won't add WebGL to IE for security reasons. And although they've caught a lot of flack for this, the fact is that they're probably right. WebGL is based on the programmable pipeline version of OpenGL, meaning that web developers write and deliver code that is compiled and run directly on the graphics card to handle scene transformation and rendering. That's pretty low-level access to memory, being granted to arbitrary people on the Internet, with security only as strong as your video driver (a huge attack surface that has never been hardened against attack). And you thought Flash was a security risk?

The irony as I see it is that the problem of hostile WebGL shaders only exists in the first place because of my first point: namely, that the browsers using WebGL basically just added direct client bindings to canvas, including forcing developers to write, compile, and link shader programs. A 3D API that was designed to abstract OpenGL away from JavaScript developers could have emulated the old fixed-function pipeline through a set of built-in shaders, which would have been both more secure and dramatically easier for JavaScript coders to tackle.

Distraction via Abstraction

Most programming is a question of abstraction. I try to train my team members at a certain level to think about their coding as if they were writing an API for themselves: write small functions to encapsulate some piece of functionality, test them, then wrap them in slightly larger functions, test those, and repeat until you have a complete application. Programming languages themselves operate at a certain level of abstraction: JavaScript hides a number of operating details from the developer, such as the actual arrangement of memory or threads. That's part of what makes it great, because managing those things is often a huge pain that's irrelevant to making your web application work.

Ultimately, the problem of WebGL is one of proper abstraction level. I tend to agree with JavaScript developer Nicolas Zakas that a good browser API is mid-level: neither so low that the developer has to understand the actual implementation, nor so high that they make strong assumptions about usage patterns. I would argue that WebGL is too low--that it requires developers to effectively learn a C API in a language that's unsuited for writing C-style code, and that the result is a security and support quagmire.

In fact, I suspect that the reason WebGL seems so alien to me, even though I've written C and Java code that matched its style in the past, is that it's actually a lower-level API than the language hosting it. At the very minimum, a browser API should be aligned with the abstraction level of the browser scripting language. In my opinion, that means (at the very least) providing a fixed-function pipeline, using arrays of native JavaScript objects to represent vertex lists and their associated data, and providing methods for loading textures and data from standard image resources.

In Practice (or, this is why I'm still a Flash developer)

Let me give an example of why I find this entire situation frustrating--and why, in many ways, it's a microcosm of my feelings around developing for so-called "HTML5." Urban Artistry is working on updating our website, and one of the artistic directors suggested adding a spinning globe, with countries where we've done international classes or battles marked somehow. Without thinking too much I said sure, I could do that.

In Flash, this is a pretty straightforward assignment. Take a vector object, which the framework supports natively, color parts of it, then either project that onto the screen using a variation of raycasting, or actually load it as a texture for one of the Flash 3D engines, like PaperVision. All the pieces are right there for you, and the final size of the SWF file is probably about 100K, tops. But having a mobile-friendly site is a new and exciting idea for UA, so I thought it might be nice to see if it could be done without Flash.

In HTML, I discovered, fewer batteries come included. Loading a vector object is easy in browsers that support SVG--but for Android and IE, you need to emulate it with a JavaScript library. Then you need another library to emulate canvas in IE. Now if you want to use WebGL or 3D projection without tearing your hair out, you'd better load Three.js as well. And then you have to figure out how to get your recolored vector image over into texture data without tripping over the brower's incredibly paranoid security measures (hint: you probably can't). To sum up: now you've loaded about half a meg of JavaScript (plus vector files), all of which you have to debug in a series of different browsers, and which may not actually work anyway.

When faced with a situation like this, where a solution in much-hated Flash is orders of magnitude smaller and easier to code, it's hard to overstate just how much of a setback HTML development can be--and I say that as someone who has grown to like HTML development much more than he ever thought possible. The impression I consistently get is that neither the standards groups nor the browser vendors have actually studied the problems that developers like me commonly rely on plugins to solve. As a result, their solutions tend to be either underwhelming (canvas, the File APIs, new semantic elements) or wildly overcomplicated (WebGL, WebSQL, Web Sockets, pretty much anything with a Web in front of it).

And that's fine, I'll still work there. HTML applications are like democracy: they're the worst platform possible, except for most of the others. But every time I hear someone tell me that technologies like WebGL make plugins obsolete, my eyes roll so hard I can see my optic nerve. The replacements I'm being sold aren't anywhere near up to the tasks I need to perform: they're harder to use, offer me less functionality and lower compatibility, and require hefty downloads to work properly. Paranoid? Not if they're really out to get me, and the evidence looks pretty convincing.

December 6, 2011

Filed under: tech»activism

Open House

Traveling in the modern age, no matter the distance, can be partially described as "the hunt for the next electrical outlet." The new stereo in our car has a USB port, which was a lifesaver, but each night played out similarly during our cross country trip: get into the hotel, let out the cat, comfort the dog, feed both, and then ransack every electrical socket in the place to recharge our phones and laptops.

I don't drive across the entire southern USA often, but I do spend a lot of time on public transit. So I've come to value devices that don't require daily charging, and avoid those that do. Until recently, I would have put my Kindle 2 in the former category, but over the last year, it has started discharging far too quickly--even after a battery replacement. There's clearly something wrong with it.

It's possible that the problem is simply three years of rough handling and being carried around in a messenger bag daily. Maybe something got bent a little bit more than the hardware could handle. I suspect that it's gotten stuck on some kind of automated process, indexing maybe, and as such it's not going to sleep properly. That has led to all kinds of voodoo trouble-shooting: deleting random books, disabling collections, even running a factory reset and reloading the whole thing. Has any of it worked? Um... maybe?

The thing is, I don't know. I don't know for certain what's wrong, or how to fix it, or if any of my ad-hoc attempts are doing any good. There's a huge void where knowledge should be, and it's driving me crazy. This is one of the few cases where I actually want to spend time providing my own tech support (instead of needlessly buying a whole new device)--and I can't. Because the Kindle, even though it's a Linux device under the hood, is an almost totally closed system. There's nowhere that I, or anyone else, can look for clues.

I was reminded of how much I've come to rely on open systems, not just by my Kindle worries, but by checking out some of the source code being used for Occupy Wall Street, particularly the OpenWRT Forum. That's a complete discussion forum written to run on a Linksys WRT wireless router--the same router I own, in fact. Just plug it into the wall, and instantly any wifi device within range can share a private space for discussion (handy, when you're not allowed to have amplified sound for meetings).

The fact that some enterprising coders can make a just-add-electricity collaboration portal out of off-the-shelf electronics owes everything to open source, albeit reluctantly. Linksys was forced, along with a lot of other router manufacturers, to release the source code for their firmware, since it was based on GPL code. That meant lots of people started messing with these boxes, in some cases to provide entirely new functionality, but in some cases just to fix bugs. That's how I ended up with one, actually: after we had a router crash repeatedly, I went looking for one that would have community support, as opposed to the original Linksys "release and forget" approach to customer service.

When something as faceless as a router can get new life, and clever new applications, through open source, it makes it all the more galling that I'm still having to play guessing games with my Kindle's battery issues. Indeed, it's frustrating that these "post-PC" closed systems are the way so many companies seem hellbent on pursuing: closed software running on disposable, all-in-one hardware. The result is virtually impossible to repair, at either the software or the physical level.

I'm not asking for open source everything--your microwave is safe. I'm not even necessarily arguing for strictly open source: I still run Windows at home, after all. I don't want to hack things for no good reason. What I think we deserve is open access, a backdoor into our own property. Enough of our lives are black boxes already. Ordinary people don't need to see it or use it, but if something goes wrong, the ability to at least look for an answer shouldn't be too much to ask.

September 21, 2011

Filed under: tech»coding


I spend a lot of time at work straddling four programming languages: PHP, SQL, JavaScript, and ActionScript. Many of our projects use at least three of these, if not all four. Yet while there's certainly some degree of domain-specific knowledge in each, there's more technique shared between them, floating off in the indefinite space of "software engineering."

Granted, I didn't study computer science in college. I had done some programming before and didn't really want anything to do with it professionally--I wanted to work for the Travel Channel! So when I fell into doing data journalism for CQ, a job that's halfway between storytelling and interactive coding, I knew there were skills where I was probably behind. And now that I feel like I'm relatively up to speed on the languages themselves, I want to catch back up on some of what I missed, starting with various low-level data structures.

The result is Typedefs, a simple blog where, in each entry, I pick an item from Wikipedia's list of data structures, implement it in JavaScript, and then explain how I did it and provide a quick demonstration. So far, I've done linked lists (the old classic), AA trees, and heaps. Next I want to try a chunk-based file, like PNG, and also a trie or bloom filter for text lookup.

I can already tell that working through these examples has been good for me--not because I expect to implement a lot of AA trees, because in my experience that's pretty rare, but because building these structures gives me a better understanding of how languages actually work, and a wider range of algorithms for solving other problems. The mechanics of a heap, for example, define a set of interesting ways to use arrays for storage and processing. AA trees really force you to examine the implications of pass-by-reference and pass-by-value. Linked lists are always a good experiment in miniature API design. As bite-sized, highly-technical exercises, they give me a chance to stretch my skills without having to build a full-sized JavaScript application.

These posts are also intended to leverage a truism: that the best way to learn is to teach someone else. By writing the blog as a teaching tool for other people, it forces me to organize my thoughts into a logical, coherent narrative: what are the foundations of this structure? What's it actually doing during this algorithm? Why is this useful? When the goal is to educate, I can't just get away with refactoring someone else's example. I need to know how and why it's built that way--and that knowledge is probably more useful than the example itself.

September 7, 2011

Filed under: tech»i_slash_o

In the Thick of It

After a month of much weeping and gnashing of teeth, my venerable laptop is back from out-of-warranty repair. That month was the longest I've been without a personal computer in years, if not decades, but I've survived--giving me hope for when our machine overlords rise up and replace Google Reader with endless reruns of The Biggest Loser.

After the second day, I felt good. In fact, I started to think that computing without a PC was doable. I do a lot of work "in the cloud," after all: I write posts and browse the web using Nano and Lynx via terminal window, and my social networks are all mobile-friendly. Maybe I'd be perfectly happy just using my smartphone for access--or anything that could open an SSH session, for that matter! Quick, fetch my Palm V!

Unfortunately, that was just the "denial" stage. By the second week of laptoplessness, my optimism had faded and I was climbing the walls. It's a shame, but I don't think cloud computing is there yet. As I tried to make it through a light month of general tasks, I kept running into barriers to acceptance, sorted into three categories: the screen, the keyboard, and the sandbox.

The Screen

Trying to do desktop-sized tasks in a browser immediately runs into problems on small-screen devices. It's painful to interact with desktop sites in an area that's under 13" across. It's even more annoying to lose half of that space for a virtual keyboard. When writing, the inability to fit an entire paragraph onscreen in a non-squint font makes it tougher to write coherently. I probably spend more time zooming in and out than actually working.

But more importantly, the full-screen application model is broken for doing real work. That sounds like nerd snobbery ("I demand a tiling window manager for maximum efficiency snort"), but it's really not. Consider a simple task requiring both reading and writing, like assembling a song list from a set of e-mails. On a regular operating system, I can put those two tasks side by side, referring to one in order to compile the other. But on today's smartphones, I'm forced to juggle between two fullscreen views, a process that's slow and clumsy.

There's probably no good way to make a multi-window smartphone. But existing tablets and thin clients (like Chromebooks) are also designed around a modal UI, which makes them equally impractical for tasks that involve working between two documents at the same time. I think the only people who are even thinking about this problem are Microsoft with their new Windows 8 shell, but it's still deep in development--it won't begin to influence the market for at least another year, if then.

The Keyboard

I bought a Thinkpad in the first place because I'm a sucker for a good keyboard, but I'm not entirely opposed to virtual input schemes. I still remember Palm's Graffiti--I even once wrote a complete (albeit terrible) screenplay in Graffiti. On the other hand, that was in college, when I was young and stupid and spending a lot of time in fifteen-passenger vans on the way to speech tournaments (this last quality may render the previous two redundant). My patience is a lot thinner now.

Input matters. A good input method stays out of your way--as a touch-typist, using a physical keyboard is completely effortless--while a weaker input system introduces cognitive friction. And what I've noticed is that I'm less likely to produce anything substantial using an input method with high friction. I'm unlikely to even start anything. That's true of prose, and more so for the technical work I do (typical programming syntax, like ><{}[];$#!=, is truly painful on a virtual keyboard).

Proponents of tablets are often defensive about the conventional wisdom that they're oriented more towards consumption, less toward creativity. They trumpet the range of production software that's available for making music and writing text on a "post-PC" device, and they tirelessly champion whenever an artist uses one to make something (no matter how many supporting devices were also used). But let's face it: these devices are--as with anything--a collection of tradeoffs, and those tradeoffs almost invariably make it more difficult to create than to consume. Virtual keyboards make it harder to type, interaction options are limited by the size of the screen, input and output can't be easily expanded, and touch controls are imprecise at best.

Sure, I could thumb-type a novel on my phone. I could play bass on a shoebox and a set of rubber bands, too, but apart from the novelty value it's hard to see the point. I'm a pragmatist, not a masochist.

The Sandbox

I almost called this section "the ecosystem," but to be honest it's not about a scarcity of applications. It's more about process and restriction, and the way the new generation of mobile operating systems are designed.

All these operating systems, to a greater or lesser extent, are designed to sandbox application data from each other, and to remove the heirarchical file system from the user. Yes, Android allows you to access the SD card as an external drive. But the core applications, and most add-on applications, are written to operate with each other at a highly-abstracted level. So you don't pick a file to open, you select an image from the gallery, or a song from the music player.

As an old PalmOS user and developer, from back in the days when they had monochrome screens and ran on AAAs, this has an unpleasantly familiar taste: Palm also tried to abstract the file system away from the user, by putting everything into a flat soup of tagged databases. Once you accumulated enough stuff, or tried to use a file across multiple applications, you were forced to either A) scroll through an unusably lengthy list that might not even include what you want, or B) run normal files through an external conversion process before they could be used. The new metaphors ('gallery' or 'camera roll' instead of files) feel like a slightly hipper version of the PalmOS behavior that forced me over to Windows CE. We're just using Dropbox now instead of Hotsync, and I fail to see how that's a substantial improvement.

Look, heirarchical file systems are like democracy. Everyone agrees that they're terrible, but everything else we've tried is worse. Combined with a decent search, I think they can actually be pretty good. It's possible that mobile devices can't support them in their full richness yet, but that's not an argument that they've gotten it right--it shows that they still have a lot of work to do. (The web, of course, has barely even started to address the problem of shared resources: web intents might get us partway there, one day, maybe.)

The Future

When I was in high school, Java had just come out. I remember talking with some friends about how network-enabled, platform-independent software would be revolutionary: instead of owning a computer, a person would simply log onto the network from any device with a Java VM and instantly load their documents and software--the best of thin and thick clients in one package.

Today's web apps are so close to that idea that it kind of amazes me. I am, despite my list of gripes, still optimistic that cloud computing can take over much of what I do on a daily basis, if only for environmental reasons. My caveats are primarily those of form-factor: it's technically possible for me to work in the cloud, but the tools aren't productive, given the recent craze for full-screen, touch-only UIs.

Maybe that makes me a holdover, but I think it's more likely that these things are cyclical. It's as though on each new platform, be it mobile or the browser, we're forced to re-enact the invention of features like windows, multitasking, application management, and the hard drive. Each time, we start with the thinnest of clients and then gradually move more and more complexity into the local device. By the time "the cloud" reaches a level where it's functional, will it really be that different from what I'm using now?

Future - Present - Past