this space intentionally left blank

February 20, 2023

Filed under: tech»web

Build Less

When it comes to web development, I'm actually fairly traditional. By virtue of the kinds of apps I make (either bespoke visualizations for work or single-serving toys for personal use), I'm largely isolated from a lot of the pain of modern front-end web development. I don't use React, I don't need to scale servers, and I render my HTML the old-fashioned way, from string templates. Even so, my projects are usually built on top of a few build tools, including Rollup, Less, and various SDKs for moving data between different cloud providers.

However, for internal utilities and personal projects over the last few years, I've been experimenting with removing tools, and relying solely on the modern browser. So instead of bundling JS, I'm just loading modules with import statements. I write one CSS file for my light DOM, but custom properties have largely eliminated what I need a preprocessor to do (and the upcoming support for nesting will cover the rest). Add something like the Eleventy dev server for live reload, and it's actually a really pleasant experience.

It's one thing to go minimalist for a single-serving hobby app, or for people in the Chalkbeat newsroom who can reach me directly for support. It's another to do it for a general audience, where the developer/user ratio starts to tilt and your scale becomes more amibitious. But could we develop a real, public-facing web app that doesn't rely on a brittle and slow compilation step? Is a no-build deployment feasible?

While I'm optimistic, I have enough self-awareness to know that things are rarely as simple as I want them to be. I wasn't always a precious snowflake, and I've seen first-hand that national (or international) scale applications have support infrastructure for a reason. To that end, here's a non-exhaustive list of potential hurdles I believe developers will need to jump to get to that tooling-free future.

Caching and coherency

In theory, HTTP2 (which reuses connections and parallelizes transfers) means that we don't pay a penalty for deploying our JavaScript as individual modules instead of a single bundled file. But it raises a new issue that we didn't have with those big bundles: what happens when we make a breaking change in part of the application, and someone visits it with a partially-primed cache, so they have some old files still hanging around? How do we make sure that we can take advantage of caching appropriately, while still keeping our code coherent for a given deployment?

Imagine we have a page that loads module A, which loads B and C, and is styled using CSS file D. I update file B, and changes to D are required for the new components. Different files may be evicted from the browser cache in unpredictable ways, though. Ideally, A and C should be loaded from the cache, and B and D should be fresh requests. If everything comes from the cache, users won't see new features, but ideally nothing should be immediately broken. It would be wasteful, but not disastrous, if all files are loaded fresh. The real problem comes if only one of B or D comes from the cache, so that we either get new code without the matching style changes, or styles without the new code.

As Jake Archibald notes, there are two working (and compatible) strategies for caching interrelated code: either long cache times with unique URLs, or no-cache headers and a shorter lifetime. I lean toward the latter strategy for now, probably using ETag hash-based headers for each file. Individual requests would be a little slower, since the browser would always check the server for individual files, but you'd only actually transfer new code, which is the expensive part (cache hits would return 304 Not Modified). Based on my experience with a similar system for election data updates, I think this would probably scale pretty well, but you'd need to test to be sure.

Once import maps are supported in all evergreen browsers, the hashed URL solution becomes the simpler of the two. Use short identifiers for all your import statements (say, based from the project root), and then hash their contents and generate a JSON mapping between the original path and the mangled filename for production deployments. Now the initial page load can be revalidated on every load, but the scripts that go with that particular page version will be immutable, guaranteeing that any change means a new URL and no cache conflicts. Here's hoping Safari ships import maps to users soon.

Vendor code

Personally, the whole point of developing things in a no-build environment is that I don't need to learn, manage, and optimize around third-party libraries. The web platform is far from perfect, but it's fast and accessible, and there's an undeniable pleasure in writing every line of code. I'm lucky that I have that opportunity.

Most teams are not lucky, and need to load libraries written by other people. Package managers mean we have a wealth of code at our fingertips. But at the same time, the import patterns that work well for Node (lots of modules in a big, deep folder hierarchy) have proven a clumsy match for the front-end. Importing files from node_modules is clumsy and painful, especially if you're also loading stylesheets and other non-JavaScript assets. In fact, much of the tooling explosion (including innovations like tree-shaking and transpilation) comes from trying to have our cake from npm and eat it too.

So loading from the same package manager as the server-side code is frustrating, and using a CDN requires us to trust a remote host completely (plus introducing another DNS/TCP handshake) into our performance waterfall. The ideal would be a shallow set of third-party modules that are colocated with our front-end code, similar to how Bower (RIP) used to handle libraries. Sadly, there are few tools or code conventions that I'm aware of now specifically for that niche anymore.

One approach that I'm intrigued by is Deno's bundle command, which generates an importable module file from an URL, including all its dependencies. Using a tool like this, you could pretty easily zip up vendor code into a single file in the equivalent of src/bower_components. You'd also have a lot more visibility into just how big those third-party libraries are when they're packed up into self-contained (absolute) units, which might provoke a little reflection. Maybe you don't need 3MB of time zone data after all.

That said, one secret weapon for managing those chunky libraries is asynchronous import(). Whereas code-splitting in a bundler is a complicated and niche process, when we use ES modules natively our code is effectively pre-split, and the browser gives us a mechanism to only request libraries when we need them. This means the cost equation for vendor code can change somewhat: maybe it's not great that a given component is multiple megabytes of script, but if users only pay the cost for that transfer and compilation when they're actually going to use it, that's a substantial improvement over the current state of affairs.

CSS imports

I've worked on some large projects where we had a single, unprocessed CSS file for the product. It was hard to stay disciplined. Without nesting or external constraints, we'd end up duplicating styles in different parts of the document and worrying about breakages if we needed to change something. The team tried to keep things well-structured, but you know how it is: if you've got six programmers, you have 12 different ideas about how the site should be organized.

CSS has @import for natively splitting styles into multiple files, but historically it hasn't been considered good for performance. Imports block the renderer and parser, meaning that you may be halting page load for the header while you wait for footer styles. We still want multiple small files on HTTP2, so the best practice is still to generate lots of <link> tags for CSS, possibly using tricks to unblock the parser. Luckily, at least, CSS is not load-order dependent the way that JavaScript is, and the @layer rule gives us ways to manage the cascade. But manually appending a tag for every stylesheet doesn't feel very ergonomic.

I don't have a good solution here. It's possible this is not as serious a problem as I think it is — certainly on my own projects, I'm able to move localized styles into shadow DOM and load them as a part of the component registration, so it tends to solve itself for anything that's heavily interactive or component-based. But I wish @import had the kind of ergonomics and care that its JavaScript counterpart did, and I suspect teams will find PostCSS easier to use than the no-build alternative.

HTML partials and templating

What's the ultimate point of eschewing build tools? Sure, on some level it's to avoid ever touching webpack.config.js (a.k.a. the Lament Configuration) ever again. But it's also about trying to claw our way back from a front-end culture that has neglected the majority of users. And the best way to address an audience on typical devices (read: an Android phone with meager single-core performance and a spotty network connection, or a desktop PC from 2016) is to send less JavaScript and more HTML and CSS.

Last week, I loaded a page from a local news outlet for work, which included data on a subset of Chicago schools. There was no dynamic content, although it did have an autocomplete search at the top. I noticed the browser tab was stuttering on load, so I looked in the dev tools: each of the 600+ schools was being individually templated and appended to a queried element from a JSON fetch. On a fairly new desktop PC, it froze the UI thread for more than half a second. On a phone, that was more like 7 seconds, even with ads blocked, and any news dev will tell you that the absolute easiest way to boost your story's load performance is to remove the ads from it.

If that page had been built as static HTML, it would parse and load almost instantly by comparison. Indeed, in the newsroom projects that I maintain, the most important feature is the ability to pull in data from a variety of sources (Google Docs, Sheets, local text and CSV, JSON, remote APIs) and merge that easily with the HTML template. The build scripts do other things, like bundling and CSS processing and deployment. But those things could be replaced, or reduced, or moved into other tools without radically changing the experience. HTML generation is irreplaceable.

At a bare minimum, let's say I want to be able to include partial templates (for sharing headers and snippets between pages), loop through some data, and inject my import map or my stylesheet collection into the page. Here's a list of the tools that let me do that easily, on most Linux servers, without installing a bunch of extra crap:

  • PHP

Listen, no shade on PHP, but I don't want to write it for a living anymore even if I wasn't working off of static file storage. It's a hard sell, especially in the context of "a modern web stack."

HTML templating is where the rubber really meets the road. We do not have capabilities for meta-processing in the language itself, and any solution that involves JavaScript (including the late, lamented HTML imports) is a non-starter. I'd kill for an <include> tag, especially if there were a way to use it without blocking the parser, similar to the way that declarative shadow DOM provides declarative support for component subtrees.

What I'm not interested in doing is stripping the build toolchain down if it means a worse experience for users. And once I need some kind of infrastructure to assemble my HTML, it's not actually that much more work to bolt on a script bundler and a stylesheet preprocessor, and reap the benefits from those ecosystems. I'm all-in on the web platform, but I'm not a masochist.

Let's build

This is by no means an exhaustive list of challenges, but Nano tells me I'm well past 200 lines in this text document, so let's wrap it up.

The good news, as I see it, is that the browser is in a healthier place than ever for hobbyists, students, and small project developers. You can open index.html, import Lit or Vue from a CDN, and have a reasonably performant front-end environment that can be grow more complex to fit your needs and skills. You can also write a lot less JavaScript than in years past, because CSS has gotten so much better for layout and interaction.

I'd say we're within reach of a significantly less complicated front-end technical culture. I would not be surprised to see companies start to experiment with serving JavaScript or CSS directly, using tooling to smooth off the rough edges (e.g., producing import maps or automating stylesheet inclusion) rather than leaning hard into full, slow-moving compilation steps. The ergonomics of these approaches are going to be better than a lot of people expect. Some front-end teams that have specialized in tooling-intensive ecosystems are going to either eat a lot of crow or get very angry for a while.

All that said: we're not going back to the days when all you needed was notepad.exe and some moxy to make a "real" website. Perhaps it's naive to think we ever were. But making a good web app is hard, I would argue harder than many other kinds of programming. It's the code you write in a trio of languages, but also the network between you and the user, the management of distributed state, and a vast range of devices, inputs, and outputs. The least we can do is make it less wearying to get started.

Past - Present