this space intentionally left blank

April 14, 2016

Filed under: tech»coding

Calculated Amalgamation

In a fit of nostalgia, I've been trying to get my hands on a TI-82 calculator for a few weeks now. TI BASIC was probably the first programming language in which I actually wrote significant amounts of code: although a few years later I'd start working in C for PalmOS and Windows CE, I have a lot of memories of trying to squeeze programs for speed and size during slow class periods. While I keep checking Goodwill for spares, there are plenty of TI calculator emulation apps, so I grabbed one and loaded up a TI-82 ROM to see what I've retained.

Actually, TI BASIC is really weird. Things I had forgotten:

  • You can type in all-caps text if you want, but most of the time you don't, because all of the programming keywords (If, Else, While, etc.) are actually single "character" glyphs that you insert from a menu.
  • In fact, pretty much the only code that's typed manually are variable names, of which you get 26 (one for each letter). There are also six arrays (max length 99), five two-dimensional matrices (limited by memory), and a handful of state variables you can abuse if you really need more. Everything is global.
  • Variables aren't stored using =, which is reserved for testing, but with a left-to-right arrow operator: value → dest I imagine this clears up a lot of ambiguity in the parser.
  • Of course, if you're processing data linearly, you can do a lot without explicit variables, because the result of any statement gets stored in Ans. So you can chain a lot of operations together as long as you just keep operating on the output of the previous line.
  • There's no debugger, but you can hit the On key to break at any time, and either quit or jump to the current line.
  • You can call other programs and they do return after calling, but there are no function definitions or return values other than Ans (remember, everything is global). There is GOTO, but it apparently causes memory leaks when used (thanks, Dijkstra!).

I'd romanticized it over time — the self-contained hardware, the variable-juggling, the 1-bit graphics on a 96x64 screen. Even today, I'm kind of bizarrely fascinated by this environment, which feels like the world's most cumbersome register VM. But loading up the emulator, it's obvious why I never actually finished any of my projects: TI BASIC is legitimately a terrible way to work.

In retrospect, it's obviously a scripting language for a plotting library, and not the game development environment I wanted it to be when I was trying to build Wolf3D clones. You're supposed to write simple macros in TI BASIC, not full-sized applications. But as a bored kid, it was a great playground, and the limitations of the platform (including its molasses-slow interpreter) made simple problems into brainteasers (it's almost literally the challenge behind TIS-100).

These days, the kids have it way better than I did. A micro:bit is cheaper and syncs with a phone or computer. A Raspberry Pi is a real computer of its own, as is the average smartphone. And a laptop or Chromebook with a browser is miles more productive than a TI-82 could ever be. On the other hand, they probably can't sneak any of those into their trig classes and get away with it. And maybe that's for the best — look how I turned out!

November 20, 2015

Filed under: tech»coding

Weir, year two

I realized the other day that Google Reader shut down in June of 2013, which means I've been using Weir as my RSS reader for more than two years now. It's my longest-running software project, and still one of the most complex things I've built in Node. And apart from occasional revisions, it's been up and running constantly, in mobile and desktop browsers, that entire time.

I don't log out a lot of metrics from Weir, so there's a lot of stuff that I'm not tracking. But I can say that there are currently 113 subscriptions, with around 6,000 stories in the database. The server that hosts the app (as well as my various domains) downloads about 20GB of data each month, most of which is probably Weir (the rest is e-mail and server updates, and I'm frankly not that popular). It also hovers around 10% of available memory, which is pretty good for a garbage-collected language on a piddly little VM.

On the client side of things, the Angular code has definitely started to show its age. This was the project that I used to learn Angular, and since then I've learned a lot about the framework. Would I use it if I were writing Weir from scratch? I'm not sure. I still love the databinding aspect of Angular, but I suspect I could write a smaller, nimbler version of the UI in vanilla JavaScript pretty easily. At some point, I may give it a shot: the server API is clean enough that writing a new client should be relatively straightforward.

As an experiment in self-hosting a cloud service, Weir is a mixed success. But I have grown to love the way that something I wrote has become a fixture of my life. I clear out my stream on the bus in the morning. Throughout the day, Weir's purple tab icon lights up to let me know that new items are available. It feels like wearing clothes that I tailored for myself — using it feels a little nicer than it should, just from the pride in its construction.

February 27, 2014

Filed under: tech»coding

Just Use Ed

There's a regular, recurring movement to replace text-based programming with some kind of graphical version. These range from Scratch (offering "blocks" to make text syntax more friendly) to Pure Data (node-based dataflow programming). Rarely do any of them take off (Scratch and pd are successful within education and audio, respectively, but little-used elsewhere), but that doesn't stop anyone from trying.

It may be the fact that I started as a writer, or that I was a language nut in college, but I've always felt that text-based programming doesn't get a lot of respect. The written word is one of the great advances of civilization. You can pack a lot of meaning into a line of text, and code is no different. Good source code can range from whimsical to workmanlike, a gamut that's hard to imagine existing in the nest of wiring that is the graphical languages.

As a result, text editing is important to me. It's important to a lot of people, but most of them don't write an editor, and I ended up doing that. I figured I'd write up some notes on the different ways people have written their editors, and why I picked one model in particular for Caret. It may be news to many people that there are even multiple models to consider, but that's programming for you: there's at least four ways to put letters into a document, and bitter wars between factions for each of them.

The weirdest editor still in common usage, of course, is Vim. Born from the days when network connections were too slow to actually update text in realtime, Vim uses a shorthand language for text editing. You don't hold delete until some amount of text is gone in Vim — instead, you type "d2w", meaning "delete two words." You also can't type directly until you switch into the "insert" mode with the "i" or "a" commands. Like many abusive subcultures, people who learn this shorthand will swear up and down that it's the only way to work, even though it's clearly a relic of a savage, bygone age.

(Vim and Emacs are often mentioned in comparison to each other, because they tend to be used by very similar kinds of people who, nevertheless, insist that they're very different. I don't really know very much about Emacs, other than it's written in Lisp and it's not as eyeball-rolling weird as Vim, so I'm ignoring it for the purposes of this discussion.)

Acme tends to look a little more traditional, but it is actually (I think) more radical than Vim, because it redefines the relationship between interface and editor. Acme turns all documents into hypertext: middle clicking a filename opens that file, and clicking a word (like "copy" or "paste") actually runs that command (either in a shell, or in Acme). There's no fixed interface in Acme, just a set of menu bars that are also text fields. I love the elegance of this idea, where a person builds an text editor's UI just by... editing text.

Which brings us to Sublime. I've been very clear that Caret is modeled closely on Sublime, with a few changes to account for quirks of the platform and my own preferences. That's partly because it's generally considered the tool of choice for web developers, and partly because it's genuinely the editor that has my favorite workflow tools. Insofar as Sublime has a philosophy, it is to prioritize clarity and transparency over power. That's not to say it's not powerful — it certainly is. But it tries to be obvious in a way that other editors do not.

For example, say you need to change a variable name throughout a function. Instead of immediately writing a regex or a macro, Sublime lets you select all the instances of that variable with the mouse or keyboard, which creates multiple cursors. Then you just type the new name. It's not as powerful as a regular expression, but 90% of the time, it's probably what you wanted to do anyway. Sublime's command/go-to palette is another smart-but-obvious idea: instead of hunting through the menu or the filesystem, open the palette and type to fuzzy-filter the list. It's the speed of a command line without the hostility.

To paraphrase an old saw, the best feature is the one you have with you. That's why putting the command palette in Caret was a must, since it puts all the menu items just a few keystrokes away. Even now, I don't always remember where a given menu item is in the toolbar in my own editor, because I hardly ever use the mouse. There was a good week when menus looked completely wrong, and I never even noticed.

The reason I've started looking over other editors now is that I think Caret can reach for more than just parity with Sublime. I'm intrigued by the ways that Acme makes it easy to jump around files, and lately I've been thinking about what it means to be an editor built in "web technology." Adding the ability to open links from a URL is a given, but it's only the start: given that OAuth provides a simple, standard method of authenticating against a remote server, a File implementation for Caret could easily open files against service endpoints for something like Github or Ghost in a generic way. It would be a universal cloud editor, but easily capable of running locally.

Of course, Caret won't be the last editor to try something different (just this week, Github announced their own effort), but it's still pretty amazing how many ways we have to solve a simple problem like "typing letters into a file." As a writer and a coder, I love being spoiled for choice.

November 22, 2013

Filed under: tech»coding

Plug In, Turn On, Drop Out

This is me, thinking about plugins for Caret, as I find myself doing these days. In theory, extensibility is my focus for the next major release, because I think it's a big deal and a natural next step for a code editor. In practice, it's not that simple.

Approach #1: Postal Services

Chrome has a pretty tight security model applied to packaged apps, not the least of which is a strict content security policy. You can't run code from outside your application. You can't construct code using eval (that's good!) or new Function (that's bad). You can't add new files to your application (mostly).

Chrome does expose an inter-app messaging system similar to postMessage, and I initially thought about using this to create a series of hooks that external applications could use. Caret would broadcast notifications to registered listeners when it did something, and those listeners could respond. They could also trigger Caret commands via message (I do still plan to add this, it's too handy not to have).

Writing plugins this way is neatly encapsulated and secure, but it's also going to be intensely frustrating. It would require auditing much of Caret's code to make sure that it's all okay with asynchronous operation, which is not usually the case right now. I'd have to make sure that Caret is sufficiently chatty, because we'd need hooks everywhere, which would clutter the code with broadcast/response blocks. And it would probably mean writing a helper app to serve as a patchboard between applications, and as a debugging tool.

I'm not wild about this one.

Approach #2: Repo, Man

I've been trying to think of a way around the whole inter-app messaging paradigm for about a month now. At the same time, I've been responding to requests for Git and remote filesystem support, which will not be a core Caret feature. For some reason, thinking about the two in close proximity started me thinking along a new track: what if there were a way to work around the security policy using the HTML5 file system? I decided to run some tests.

It turns out this is absolutely possible: Chrome apps can download a script from any server that's whitelisted in their manifest, write that out to the filesystem, and then get a special URL to load that file into a <script> tag. I assume this has survived security audits because it involves jumping through too many hoops to be anything other than deliberate.

The advantages of this approach are numerous. Plugin code would operate directly alongside of Caret's source, able to access the same functions and modules and call the same APIs that I use. It would be powerful, and would not require users to publish plugins to the Chrome store as if they were full applications. And it would scale well--all I would need to do is maintain the index and provide some helper functions for developers to use when downloading and caching their code.

Unfortunately, it is also apparently forbidden by the Chrome Web Store policies, which state:

Packaged apps should not ... Download or execute scripts dynamically outside a sandboxed environment such as a webview or a sandboxed iframe.
At that point, we're back to postMessage unless I want to be banned from the store. So much for the workaround.

Approach #3: Local hosting

So how can I make plugins work for end users? Well, honestly, maybe I don't. One of the nice things about writing developer tools, particularly oddball developer tools, is that the people using them and wanting to expand on them are expected to have some degree of technical knowledge. They can be trusted to figure out processes that wouldn't necessarily be acceptable for average computer users. In this case, that might mean running Caret as an unpacked app.

Loading Caret from source is not difficult--I do it all the time while I'm testing. Right now, if someone wants to fork Caret and add their own features, that's easy enough to do (and actually, a couple of people have done so already). What it lacks is a simple entry point for people who want to contribute functionality without digging into all the modules I've already written.

By setting up a plugins directory and a little bit of infrastructure, it's possible to reach a middle ground. Developers who really want extra packages can load Caret from source, dump their code into a designated location, and have their code bootstrapped automatically. It's not as friendly as having web store distribution, and it's not as elegant as allowing for a central repo, but it does deliver power without requiring major rewrites.

Working through all these different approaches has given me a new appreciation for insecurity, which sounds funny but it's true. Obviously I'm in favor of secure computing, but working with mobile operating systems and Chrome OS that strongly sandbox their code tends to make a person aware of how helpful a few security holes can be, and vice versa: the same points for easy extension and flexibility are also weak points that can be exploited by an attacker. At times like this, even though I should maybe know better, that tradeoff seems absolutely worth it.

November 14, 2013

Filed under: tech»coding

For Free Ninety Nine I'll Beat 99 Acts Down

Assuming that the hamster powering the Chrome web store stats is just resting, Caret clicked over to 10,000 installations sometime on Monday. That's a lot of downloads. At a buck a piece, even if only a fraction of those people had bought a for-pay version, that might be a lot of money. So why is Caret free? More importantly, why is it free and open source? Ultimately, there are three reasons:

  1. I feel like I owe the open source community for the value I've gotten from it (basically, everything on the Internet), and this is a way to repay that debt.
  2. Caret isn't really just mine. It's heavily influenced by Sublime, and builds on another open source project for its text processing. As such, it feels awkward to charge money for other peoples' work, even if Caret's unique code is significant in its own right.
  3. I think I get more value (i.e. job marketability, reputation, skill practice) out of being the person with a chart-topping Chrome app, in the long term, than I would get from sales.

Originally, I had planned on writing about how I reconcile being a passionate supporter of paid writing while giving away my hobby code, but I don't actually see any conflict. I expect a paycheck for freelance coding the same way I expect it for journalism — writing here (and coding Caret) doesn't directly benefit anyone but me, and it doesn't really cost me anything.

In fact, it turns out that both industries also share some uncomfortable habits when it comes to labor. Ashe Dryden writes:

Statistically, we expect that the demographic breakdown of people contributing to OSS would be about the same as the people who are participating in the OSS community, but we aren't seeing that. Ethnicity of computing and the US population breaks down what we would hope to see as far as ethnicity goes. As far as gender, women make up 24% of the industry, according to the same paper that gave us the 1.5% OSS contributor statistic.

Dryden was responding to a sentiment that I've seen myself (and even been guilty of, from time to time): using a person's open source record on sites like GitHub as a proxy for hireability. As she points out, however, building an open source portfolio is something that's a lot easier for white men. We're more likely to have free time, more often have jobs that will pay for open source contributions, and far less likely to be harassed or dismissed. I was aware of those factors, but I was still shocked to see that diversity numbers in open source are so low. We need to do better.

As eye-opening as that is, I think Dryden's middle section centers around a really interesting question: who profits?

I'd argue that the people who benefit the most from the unpaid labor of OSS as well as the underpaid labor of marginalized people in technology are business owners and stakeholders in these companies. Having to pay additional hundreds of thousands or millions of dollars for this labor would mean smaller profit margins. Technology is one of the most profitable industries in the US and certainly could support at least pay equality, especially considering how low our current participation is from marginalized people.

...Open source originally broke us free from the shackles of proprietary software which forced us to "pay to play" and gave us little in the way of choices for customization. Without realizing it, we've ended up in a similar scenario where we are now paying for the development of software that large companies financially benefit from with little cost to them.

Her conclusion — that the community benefits, but it's mostly businesses who boost their profits from free software — should be unsettling for anyone who contributes to open source, and particularly those of us who see it as a way to spread a little socialist good will. For this reason, if nothing else, I'll always prefer the GPL and other "copyleft" licenses, forcing businesses to play ball if they want to use my code.

May 30, 2013

Filed under: tech»coding

Project Seymour

A month from now, Google will shut down Reader, leaving RSS addicts in the lurch. I suspect this will be both more and less disruptive than anticipated: expect replacement services to go through another set of growing pains, but RSS isn't exactly a high lock-in situation, and most people will find a new status quo fairly quickly.

I am not eager to move from one hosted service to another (once burned, twice shy), nor do I want to go back to native applications that can't share progress, so as soon as the shutdown was announced I started working on a self-hosted RSS reader. I applied the same techniques I'd used for Big Fish Unlimited: an easy-to-configure router, a series of views talking to the database only through model classes, and heavy use of closures for dependency management and callbacks. I built a wrapper around PHP's dismal cURL library. It was a nice piece of architecture.

It also bogged down very, very quickly. My goal was a single-page application with straightforward database queries, but I was building the foundation for a sprawling, multi-page site. Any time I started to dip in and add functionality, I found myself frustrated by how much plumbing I needed in order to do it "the right way." I was also annoyed by the difficulty of safely requesting a large number of feeds in parallel in PHP. The language just isn't built for that kind of task, even with the adaptations and improvements that have been pasted on.

This week I decided to start over, this time using Node.js and adopting a strict worse is better philosophy. When I use Reader, 99% of my time is spent in "All Items" pressing the spacebar (or, on mobile, clicking "Mark Items as Read") to advance the stream. So I made that functionality my primary concern, and wrote only as much as I needed to (both in terms of code size and elegance) to make that happen. In two days, I've gotten farther than I had with the PHP, and I'm much happier with the underlying platform as well--Node is unsurprisingly well suited to firing off tens and hundreds of concurrent requests.

I've just posted the work-in-progress code for the application, which I'm calling Weir (just barely winning out over "Audrey II"), to a public GitHub repo. It is currently ugly, badly-documented, and patchy in places. The Angular code I'm using for the front-end is obviously written by a someone with very little experience using the library. There's lots of room for improvement. On the other hand, my momentum is very good. By next week, I expect Weir will be good enough for me to dogfood it full time, and at that point improvements will come naturally whenever I need to smooth out the rough edges.

I like this way of working--"worse is better"--quite a bit. It's not always pretty, but it seems effective so far. It also fits in well with my general coding style, which is (perhaps unsurprisingly) on the left-ish side of Steve Yegge's developer politics. I like elegance and architecture as much as the next person, but when it all comes down to it, there's no point in elegant code that never gets used.

Writing my own Reader alternative is also proving educational. The conventional wisdom is that RSS readers benefit greatly from running at scale: operations like feed retrieval can be performed once for all subscribers, spreading the costs out. The flip side is that you're at the mercy of the server bot for when you get updates. High-frequency feeds, such as politics or news, get batched up instead of coming in as they're posted. I'm also able to get a lot more feedback on which feeds are dead, which came as a surprise: Reader just swallowed the errors whole. All in all, I doubt the experience will be any worse.

Currently, Weir isn't much good for public consumption. I've made a sanitized copy of my config file in the repo, but there's no setup script for the database, and no import step for getting your subscriptions loaded up. I hope to have that ready soon, and the code is licensed under the GPL, so pull requests and feature suggestions are welcomed as it becomes usable for other people.

October 14, 2012

Filed under: tech»coding

Repo Man

Although I've had a GitHub account for a while, I didn't really use it much until last week, when I taught my Seattle Central students how to use version control for project sharing. That lesson was the first chance I'd had to play with GitHub's Windows client (although I develop most of my web code on Linux, I do a lot of JavaScript work on my Thinkpad in Windows). It seemed like a good time to clean up my account and create a few new repositories for projects I'm working on, some of which other people might find interesting.


Code is my personal JavaScript utility belt--I use it for throwing together quick projects, when I don't want to hunt down a real library for any given task. So it provides a grab-bag of functionality: Futures/Promises, an extremely limited template system, shims for Function.bind and Base64 encoding, basic object/array utilities, and a couple of useful mixins. I started building this at CQ and took it with me because it was just too handy to lose. While I doubt anyone else will be using this instead of something like Underscore, I wanted to put it in a repo so that I can keep a history when I start removing unused portions or experimenting.


Grue is undoubtedly a cooler project: it's a small library for quickly building text games like Zork. I originally wanted to port Inform7 to JavaScript, but found myself stymied by A) Inform7's bizarre, English-like syntax, and B) its elaborate rule matching system. The fact that there's no source to study for the Inform7 compiler (not to mention that it's actually recompiling to an older version of Inform, then compiling from there to z-machine bytecode) puts this way beyond my "hobby project" threshold.

Instead, I tried to think about how to bring the best parts of Inform7, like its declarative syntax and simple object heirarchy, to JavaScript. Instead of connecting items directly to each other, Grue provides a "Bag" collection type that can be queried by object property--whether something is portable, or flammable, or contains a certain keyword--using a CSS3-like syntax. Objects also come with built-in getter/setter functions that can be "proxied" to temporarily override a value, or change it based on world conditions.

It takes about 30 lines of Grue code to write the opening scene of Zork I, which is not quite as concise as Inform7, but it's pretty close. I figure Grue is about halfway done--I still need to add more vocabulary, regional rulesets, and some additional types (Regions, Doors, Devices, etc)--but it's close enough to start dogfooding it. Feel free to pull the repo and open "index.html" to see what I've gotten so far.


My fork of KeepassDroid exists entirely to scratch a particular itch: I like the Android port of Keepass, but I find its UI to be functional, at best. The fonts are often too small, and forms end up underneath the virtual keyboard more often than not. So I've changed the view styles, and some of the layout XML, just for my own use (there's a .zip with the compiled application package, in case anyone else is interested). The main project doesn't seem interested in my changes, which is fine by me, but it does mean that every now and then I have to merge in changes from trunk if I want mine up to date. Increasingly, I don't bother.

Urban Artistry

And then there's one project I've been working on that's not located on GitHub, but went live this weekend. Ever since I started maintaining the web presence for Urban Artistry, it's been a mess of PHP files accreted since they first went online. There was an abortive attempt to move to WordPress in 2010, but it never got anywhere, and it would have used the same theme that someone once described as "a bit like a dark nightclub."

When UA went fully non-profit in the state of Maryland, and asked me to be on the board, one of my goals was to turn the site into something that would be a bit more appealing to the typical grant donor. The new site is intented to do exactly that: my design takes its cues from the UA logo with a lightweight, modern feel. The site is also responsive across three sizes--phone, tablet/netbook, and desktop--and since it's built on WordPress, it's easy for other members of the company to log in and make changes if they need to do so. I'm pretty happy with how things turned out, but the design was the easy part: content is much harder, and that's what we're tackling next.

October 3, 2012

Filed under: tech»coding

Teachable Moments

When you're on top of the world, it's the perfect time to start kicking the little people who lifted you up. At least, that's the only conclusion I can draw from Bret Victor's newest post on teaching code. After he did his presentation on "Inventing on Principle" a while back, the tech community went nuts for Victor's (admittedly impressive) visualization work and approach to live programming. This admiration culminated in Khan Academy's computer science curriculum, which integrates a live Processing environment very similar to Victor's demos. In response, he's written a long post bashing the crap out of it.

Instead, he has a plan to redesign programming itself in a new, education- oriented direction. I'm generally a fan of Victor's presentation work (his touch-based animation UX is phenomenal), but I find that his ideas for teaching tend to be impractical when they're examined closely, and I suspect that's the case here. I don't think it's a coincidence that Victor doesn't seem to spend a lot of time asking if anyone else has solved these problems. A little research should have turned up that someone already wrote the language he's proposing: Scratch.

Scratch isn't terribly pretty--it's designed for kids, and it shows--but it provides almost everything Victor claims he wants. Variables are provided in context, with an instant visual environment that lets users examine routines at any time. The syntax is all drag-and-drop, with clear indications of what is nested where, and there's a stepping debug mode that visually walks through the code and provides output for any variables in use. And as much as Victor wants to push the comparison to "pushing paint," Scratch's sprite-based palette is probably as good as that'll get for programming. That no mainstream programming languages have followed its lead doesn't necessarily indicate anything, but should at least give Victor pause.

In his essay, however, Scratch is nowhere to be found. Victor draws on four other programming paradigms to critique Processing: Logo, Smalltalk, Hypercard, and Rocky's Boots. To say that these references are dated is, perhaps, the least of their sins (although it does feel like Victor's research consisted of "stuff I remember from when I was a kid"). The problem is that they feel like four random things he likes, instead of coherent options for how to structure a learning program. They couldn't possibly be farther from each other, which suggests that these lessons are not easy to integrate. Moreover, using Logo as a contrast to Processing is ironic, since the latter's drawing instructions are strikingly similar (I typically use the Logo turtle to introduce people to canvas graphics). And in Smalltalk's case, the syntax he's applauding is deceptively complicated for beginners (even I find the message rules a little confusing).

Meanwhile, where are the examples that aren't twenty years old? The field hasn't stood still since the Apple IIGS, but you wouldn't know it from Victor's essay. Scratch is the most well-known educational programming environment, but there's no shortage of others, from the game-oriented (Kodu, Game Maker) to actual games (The Incredible Machine, SpaceChem). Where's the mention of the vibrant mod community (UnrealScript is many a coder's first language, and I've had several students whose introduction to coding was writing Lua scripts for World of Warcraft)? Like his Braid-inspired live coding demonstration, Victor's essay gives the impression that he's proposing some incredible innovation only by ignoring entire industries working on and around these problems. It's unclear whether he thinks they're not worth examining, or if he just can't be bothered to use Google.

There's also a question of whether these essays solve problems for anyone but Bret Victor. His obsession with visual programming and feedback is all well and good, but it ignores the large class of non-visual problems and learning styles that exist. As a result, it's nearly all untested, as far as I can tell, whereas its polar opposite (Zed Shaw's Learn Code the Hard Way) has a huge stream of actual users offering feedback and experience.

Let me clarify, in case it seems like I'm simply blaming Victor for failing to completely reinvent computing in his spare time. These essays repeatedly return to visualization as the method of feedback: visualization of time, visualization of data, and code that itself performs visualization. Unfortunately, there's an entire field of programming where a graphical representation is either impossible or misleading (how much of web programming is just pushing strings around, after all?).

Frankly, in actual programming, it's counterproductive to try to examine every value by stepping through the code: if I reach that point, I've already failed all other approaches. My goal when teaching is explicitly not for students to try to predict every value, but to think of programming as designing a process that will be fed values. In this, it's similar to this Quora answer on what it's like to be an advanced mathematician:

Your intuitive thinking about a problem is productive and usefully structured, wasting little time on being aimlessly puzzled. For example, when answering a question about a high-dimensional space (e.g., whether a certain kind of rotation of a five-dimensional object has a "fixed point" which does not move during the rotation), you do not spend much time straining to visualize those things that do not have obvious analogues in two and three dimensions. (Violating this principle is a huge source of frustration for beginning maths students who don't know that they shouldn't be straining to visualize things for which they don't seem to have the visualizing machinery.) Instead... When trying to understand a new thing, you automatically focus on very simple examples that are easy to think about, and then you leverage intuition about the examples into more impressive insights.

"Show the data" is a fine mantra when it comes to news graphics, but it's not really helpful when coding. Beginning coders should be learning to think in terms of data and code structure, not trying to out-calculate the computer. Working with exact, line-by-line values is a distraction at best--and yet it's the primary focus of Victor's proposed learning language, precisely because he's so graphically-focused. The idea that the goals of a visualization (to communicate data clearly) and the goals of a visualization programmer (to transform that data into graphics via abstraction) are diametrically opposed does not seem to have occurred to him. This is kind of shocking to me: as a data journalist, my goal is to use the computer to reduce the number of individual values I have to see at any time. A programming language that swamps me in detail is exactly what I don't want.

I'm glad that people are pushing the state of tech education forward. But changing the way people learn is not something you can do in isolation. It requires practical research, hands-on experience, and experimentation. It saddens me that Victor, who has some genuinely good feedback for Khan Academy in this essay, insists on framing them in grandiose proclamations instead of practical, full-fledged experiments. I don't know that I could honestly say which would be worse: if his ideas were imitated or if they were ignored. Victor, it seems, can't decide either.

March 29, 2012

Filed under: tech»coding

Test Driven Testing

Next week a new quarter of Intro to JavaScript will start, and while I'm almost sick of writing about it, it's been constantly on my mind. I've talked a little here about some of the changes I'll be making--primarily I'll be teaching jQuery first--but there's another big experiment I'm going to try: using test-driven development tools to give students feedback on assignments and quizzes throughout the class. I did this in a limited way during the Winter quarter, with an assignment and an exam built using unit tests, but now I want to make it much more widespread. Why use testing this way? Two reasons: to force students to code, and to encourage them to think more critically about their mistakes.

The first reason may seem odd--they're in a programming class, shouldn't that force them to code already?--but it is really an extension of drill learning. I am not good at teaching via drills: I appreciate the value of repetition when I'm learning myself, but I hate standing in class and going through the same exercise again and again. I'd much rather give lectures about theory and application than do drills. But it's clear from last quarter that many students do not get enough practice from lectures. They may be too busy to write code at home, and assignments don't usually force them to use all the constructs of the language.

Drill and repetition are important because, just as with a spoken language, it's not enough to learn the rules of grammar. You need to go out and practice, so that the mechanics become natural and you can concentrate on expressing yourself. That's the motivation behind Zed Shaw's Learn Python the Hard Way series, and I tend to agree. Take the humble for loop, for example. I've written loops for so long I could do them in my sleep, but if you're just starting out there are a lot of moving parts that need to be grasped: for (var i = 0; i < list.length; i++) { //loop goes here } We've written a keyword (for), a series of semicolon-delimited statements inside parentheses, a block (represented by the curly braces), and we haven't even gotten to the actual loop. Just within this boilerplate, we're initializing a counter with the arbitrary name of "i" and giving it an initial value, testing it, and incrementing it. All of this is expressed in a peculiar syntax (the test and increment are tossed in after the setup statement, not placed in the order where they actually get executed), with a lot of very particular punctuation, plus a lot of bookkeeping to track mentally.

Writing for loops (or conditionals, or functions, or almost anything else) is therefore as much about learning a very specific arrangement of characters as it is about the overarching concepts. Languages can make this slightly easier to grasp (as with Python's comprehension-style loops and no-braces indentation), but I don't think you ever get away from it entirely. It is my goal this quarter to make students write as many of any given construct as possible in class, so that the low-level syntax becomes automatic, while using tests to keep the high-level goals in sight.

Which brings us to the second reason for using constant testing: students need to learn how to find mistakes by thinking critically about their output. Part of that process involves teaching them to use the developer tools: how to read a Firebug log, understand an interpreter error, and examine values in the debugger. But it also means they need to learn to check their code constantly, and to have thought about what they expect to get out of any given block. While I'll be providing tests on quizzes and homework assignments, so that students can immediately see if their answer was correct, the ultimate goal is to have them write their own simple tests to check their own code.

It's my belief that most of my students know what they want to do at the very highest level, but they're not able to break those goals into smaller pieces, and they don't know how to figure out when the smaller pieces stop working. My hypothesis is that writing tests may not be enough to create critical thinking where none exists, but it will serve as a good starting place for discussions of problem-solving strategy: what caused this test to fail? What were our expectations, and how were the results different? Was this test actually a good test to begin with, even if it "passes?" These are questions that good programmers ask, perhaps naturally, but I refuse to believe that they can't be encouraged in any student.

March 7, 2012

Filed under: tech»coding

jQuery as a Second Language

At this point, with the winter quarter at SCCC drawing to a close, I'd like to add onto my earlier comments on teaching JavaScript. I would still argue that it's not a great first language--too much reliance on other technology stacks (the browser, HTML, CSS), among other things. But John Resig's original post also talks about working around that complexity with libraries, and that speaks to a deeper divide in the JavaScript community: should newcomers learn "JavaScript," or "jQuery?" In this, I'm increasingly on Resig's side.

It's easy to understand why many JavaScript programmers are so adamant that newcomers should learn the language first. Sites like Stack Overflow are full of people whose answer to every JavaScript question is "use jQuery," even in response to simple problems that should be solved more directly. There are a lot of people out there whose idea of being a JavaScript programmer means "I can include jQuery and a few plugins on my page." Not that there's anything wrong with that.

But is the solution to teach people the DOM? That seems like cruel and unusual punishment--a kind of Protestant work ethic: real programmers suffer. The browser document object model ranks up there in the history of terrible API design: it's overly-verbose, inconsistent, and tedious. If you use it for any amount of time, you will end up recreating at least the traversal parts of jQuery--searching up and down the tree for related elements--if not the animation and CSS methods as well. Traversal isn't a hard problem per se, but as with anything involving recursion, it's kind of a lot to throw at a beginner.

In fact, it probably gets in the way of doing any real, productive learning. This quarter, caught up in the desire to do things "the right way," I taught JavaScript from the DOM methods up. As a result, we didn't get to jQuery until the sixth week--halfway through our time!--and several students asked "why would we ever use this?" The jump from the DOM to actually being effective is just too far for beginners, and without that reinforcement, they're likely to throw up their hands and give up.

The other argument for JavaScript-first education is that learning jQuery will leave you with blind spots in the underlying language. That sounds reasonable, but is it true? I myself didn't learn the DOM methods first--I started with jQuery, and then I picked up the lower-level functions as I went along, usually as a way to write more efficient event listeners or work with XML. I would have a hard time pointing out any ways in which learning jQuery held me back from learning the language.

I suspect the opposite is actually true: jQuery serves as a better introduction to modern JavaScript than the DOM methods do. Over the last few years, the way people write JavaScript has evolved rapidly: greater use of closures and functional coding, creative abuse of objects as hashes, and a variety of inheritance methods. Using the DOM will not help you learn these common patterns, while jQuery relies on them heavily. Teaching the DOM methods first means less time to cover the many uses of first-class functions, meaning that students who go on to use underscore.js or d3.js (among others) are likely to be completely baffled by the code style they see there.

Ultimately the call to teach "real JavaScript" first is exactly the kind of macho posturing that's far too common throughout tech culture, like saying that only assembly programmers are "real programmers" and only people who build their own computers "really understand" them. It's ridiculous there, and it's still ridiculous in the browser. No-one would suggest starting a basic programming class by introducing the Win32 COM API, but we're supposed to force JavaScript programmers to learn at the lowest, least-useful level of abstraction? Most tellingly, if you asked people who are advocating for DOM methods, I have no doubt that they're all using jQuery (or another library) when it comes time to get work done.

As with any subject, foundation is important. But that doesn't always make it the right starting place. When teaching bass, for example, we wouldn't make someone master chord theory before we taught them a simple scale. Instead, we can start at one level of abstraction and work both up and down. It's the same when coding: by starting people with jQuery, I suspect it's easier to get them into the DOM later. We can still teach the fundamental patterns of the language without being caught up in the implementation of a bad API.

Future - Present - Past