HomeRefactoring
Or, making the site more efficient
Back in March, I wrote about making my post publishing process on this blog a bit simpler. Well; that was really just a side effect. The main point of that post, and the process behind it, was to find a simple and cheap way to move this site onto HTTPS-based hosting, which I accomplished with an Azure Static Web App. The side effect was that the official way to deploy an Azure Static Web App is via Microsoft Oryx, run from a GitHub Action. So now, when I write a new post, I have a fairly ordinary workflow similar to what I’d use (and do use!) in a multi-developer team. I create my changes in a Git branch, create a GitHub pull request, merge that pull request, and the act of doing a merge kicks off a GitHub Action pipeline that fires up Oryx, runs Wintersmith, and produces a site image which Oryx then uploads to Azure. Don’t be scared of all the different names of all the steps: for me, it’s just a couple of buttons that sets off a whole Heath Robinson chain of events. If I was doing this in a multi-person team, the only real difference would be to get someone else to review the change before I merge it, just to make sure I haven’t said something completely stupid.
You, on the other hand, are getting me unfiltered.
I mentioned in that previous post that Oryx would often give me a very vague Failure during content distribution error if the content distribution step—the step that actually uploads the finished site to Azure—hit a five-minute timeout. I tried to address this, at least partially, by cutting the size and number of the tag pages; and it did address it, partially. Not all of the time though. After an evening of trying to deploy a new post for an hour or so, hitting the timeout each time and trying again, I decided I had to come up with a better approach. What I came up with, again, has another rather nice side effect.
A little bit of digging around what other people facing the Failure during content distribution error had written, unearthed a useful tidbit of info. That timeout doesn’t just happen when the site is large in size. It also is more likely to happen when an upload contains a lot of changes.
Now, every page on this site has a whole bunch of menus. If you’re on desktop, they’re over to your right; if you’re on mobile, they’re down at the bottom of the page somewhere. There’s articles filed by category and articles filed by date. There’s the cloud of all the tags I’ve used the most, and there’s links to other sites—I really should give that a refresh. Those blocks are on every page. The ones which link to other pages include a count of articles in that category or month, so you know what you’re letting yourself in for. The tag cloud’s contents shift about occasionally, depending on what I’ve written. The end result is that, when I was adding a new post to the site, every single page already on the blog had to be rewritten. For example, this is the first (and only) post from May 2024, so every single page already on the site (all 4,706 of them), had to be rewritten to add a “May 2024 (1)” line at the top of the “Previously…” section. That’s about 84% of the files on the blog, changing, to add one new post.
However…it doesn’t have to be like that.
The whole site doesn’t have to be completely static. It can still remain server-side static, if you’re willing to rely on a bit of client-side JavaScript; most people, the vast majority of people, have JavaScript available. Instead of including those menus in every page, I thought, why not render those menus once, and have a wee piece of JavaScript on each page that pulls in those blocks on demand?
It wasn’t that hard to do. Rendering the files just needed me to pull those blocks out of the main layout template and into their own files. The JavaScript code to load them is all of 11 lines long, and that’s if you write it neatly; it really just initiates a HTTP GET call and when the results come back, inserts them onto the right place on the page. There’s a sensible fallback option that lets you click through to a truly-static version of each menu, just in case you’re having problems—those largely already existed, but weren’t really being used. Now, adding one new post needs, at the moment, just over a hundred files to change. Most of those are the hundred-ish files that make up the “main sequence” of posts, as when you add one at the top, another drops off the bottom and on to the next page, and so on all the way down. There are also the affected category and month pages. Even so, you’re going from changing ~84% of the pages on the site, to changing somewhere around 2-5%. That’s a massive difference. It also reduces the size of the site quite a lot too: those menus are over 12kb of code, all together. Not very much by modern standards, just once; but repeated on every page of the site, that was using up about 58Mb of space which has now been clawed back.
Naturally, the first deployment of the new system took a few goes to work, because it was still changing every page on the site. Since, though, deployments have gone completely smoothly, and the problem hasn’t come back once. Hopefully, things will stay that way.
This isn’t the only improvement I’ve been working on, by the way. There is, upcoming, another big change to how the site is published. It isn’t quite ready to go live yet, though. I’ll be blogging about it when it reaches production, when I find enough free time to get it finished. It’s something I’m really pleased with, even though if I didn’t tell you, you wouldn’t actually notice a thing. You’ll just have to wait for the next meta-blog post about engineering on the site, to find out what I’m working on.