I run a small hosting service called Stelae. Every customer gets a private WordPress instance, and Stelae turns that instance into static HTML and publishes the result to a host the customer owns.
There’s one operator, and that’s me. No team, no on-call rotation, nobody to escalate to at three in the morning (and I like sleeping). A lot of the technical decisions follow from that, and this post is about what it looks like in the code.
The short version: the system assumes nobody will be there when something breaks, and tries to be boring enough for that to be fine.
Of course a lot of work goes into not having outages in the first place, and most of what I change in a given month is some version of making one less likely. But everything breaks eventually, no matter how much effort went in, and the failures that cost you are the ones nobody notices for hours. That second case is what most of this post is about.
The published site does not live here
The most important decision is that Stelae never hosts the published site.
The editor runs on my infrastructure, the result doesn’t. When a customer publishes, the static output gets pushed to an account they own, a git repository, Cloudflare or statichost.eu. I hold the editing environment and the backups, somebody else holds the thing the public actually visits.
That’s unusual for a hosting product and it costs me the obvious business model. Reselling hosting is where the recurring revenue in this category normally comes from, and I gave that up on purpose.
What I get in return is that my reliability stops being the customer’s problem. If Stelae goes down, published sites stay up, they were never running here. If I stop operating the service entirely, those sites are still where the customer put them. They only need me for the editor, which they use now and then and can restore from a standard WordPress backup.
I also prefer that to an uptime promise. If a one-person service guarantees availability, you’re trusting a claim. If my availability can’t affect your live site, there’s a lot less to trust.
Reconciliation instead of orchestration
The core operational idea is that nothing is expected to stay correct on its own. The usual alternative, orchestration, runs every change as a sequence of steps that has to finish in the right order, and when one fails halfway somebody has to work out where it stopped, which with one operator means me, possibly days later.
Every fifteen minutes a reconciler wakes up and works through the passes below. Each pass compares one pair of things that can drift apart and repairs the difference:
1 container state vs. actual Docker reality
2 subscription state vs. the payment provider
3 in-memory state vs. what is written in SQLite
4 stale pending-payment rows and orphaned records
5 Docker containers and networks nothing owns
6 365-day inactive site cleanup
7 orphaned export, backup and outbox directories
8 paid payments with no invoice generated yet
9 10-year retention purge (German bookkeeping law)
10 SQLite integrity check, once a day
11 free disk space
12 database reachability
13 containers pegged at their CPU or memory ceiling
14 captured WordPress mail waiting to be forwarded
15 container images vs. the latest upstream build, once a weekFor each kind of state there’s one source I trust: Docker for whether a container is running, the payment provider for whether a subscription is alive, and the state machines for what the application believes. When they disagree, something has already gone wrong, and I’m less interested in preventing that than in limiting how long it stays wrong.
I picked fifteen minutes because it’s short enough that no customer notices and long enough that I don’t have to build anything that reacts in real time.
The passes are capped: at most 50 invoices per pass, 20 TTL deletions, 20 orphan archive removals. If the invoice pass ever found a thousand missing invoices, it would take twenty ticks instead of saturating the machine in one. This has never mattered in production, it’s there so that the worst case is slow instead of loud.
Three state machines
Every site has three finite state machines: site, subscription and container.
site pending_payment → provisioning → active ⇄ inactive
(plus provision_failed, deleted)
subscription none → pending → activating → active → cancelled/expired
(plus recurring_failed)
container none → created_disabled → stopped ⇄ running
(plus suspend_failed, unsuspend_failed, start_failed)They’re separate because they fail separately. A subscription can lapse while a container is happily running, a container can crash while the subscription is fine. One combined status field would need states like “active but container broken”, and that list grows until nobody can enumerate it anymore.
Transitions that move several machines at once go through compound operations with names like PaymentConfirmed and SubscriptionExpired, so the logic lives in one place instead of being re-derived at every call site.
While the server runs, the state machines are the source of truth and SQLite is the durable record written alongside them. If the two drift apart, pass 3 copies the state machines’ view into SQLite, and after a restart it goes the other way, the state machines are rebuilt from what SQLite has.
How I find out
Everything so far is about the system repairing itself. Some things it can’t repair, and for those there has to be a way to reach me that doesn’t depend on me looking.
Alerts are push notifications to my phone, posted to an ntfy-compatible endpoint. If that POST fails, the same message goes out as an email through the transactional mail relay. The alert path has its own fallback because an alerting system that fails silently is worse than none, it lets you believe you would have heard.
There are two kinds of message.
The first is “a human is needed”. Each state machine marks some of its states as needing intervention: provisioning failed, a container that wouldn’t suspend, a recurring payment that didn’t go through. When a site lands in one of those, the registry opens an intervention record. A checker on its own one-minute ticker (deliberately independent of the reconciler, so a slow pass can’t delay it) sends the record and re-sends it every hour until the state changes. After a restart, the first thing the server does is send a summary of every open intervention, so a reboot can’t swallow a problem.
The second kind is activity: things happening on the platform that don’t need anything from me, but that I still want to hear about. Each type can be switched off from the admin panel.
Alerts get dampened where the loud version would just train me to ignore them. The invoice pass talks to the payment provider every tick, and a single failed call is usually DNS or a transient error on their side that fixes itself. So the pass alerts after four consecutive failures, which at the fifteen minute tick is about an hour: long enough to skip the blips, short enough that a real outage is on my phone the same day.
Then there’s the case where nothing on the machine can tell me anything, because the machine is gone. For that case silence is the signal. The reconciler posts a heartbeat after every pass, and a host-level monitor outside the container stack posts one every five minutes. Both go to a small checker running somewhere else, which knows nothing about the platform beyond “this source usually talks to me and has stopped”. It’s the only part of the monitoring that would survive the main machine disappearing, and deliberately the simplest.
If the machine is gone for good, what’s left is the backups. The databases and every site’s content are copied to a separate server on a schedule, so they don’t disappear with this one. A backup isn’t considered good just because the copy finished either, each SQLite snapshot gets an integrity check first, and if one fails the backup is abandoned and my phone gets an urgent alert instead of a corrupt archive being kept.
SQLite, and checking its settings at startup
The application database is SQLite. One file, WAL mode, foreign keys on, a five second busy timeout, and a maximum of one open connection.
With only one connection, I don’t have to deal with concurrent writes. For a service whose whole customer base fits in a spreadsheet, giving up write parallelism costs nothing. WAL is there for the one other process that does touch the file, the backup, which reads it while the server keeps writing.
The part I actually want to point at is a startup check that verifies those settings were applied.
Go’s SQLite drivers are configured through the connection string, and they silently ignore parameters they don’t recognise. Those two facts combine badly. A typo, or a snippet copied from a different driver’s documentation, gives you a database that opens successfully with none of the settings you asked for. No error, the service just runs without WAL, without foreign keys and without a busy timeout, and nothing tells you.
So on every startup the code reads the pragmas back from the live connection and refuses to start unless they match what was requested.
This actually happened once, before launch, with a different driver than the one I use now, and the check caught it. When I later swapped drivers (and the parameter syntax changed with it), the check was the thing that would have caught it again.
Containers that sleep
Each WordPress instance is a real, persistent container with its own network, its own database, and the site content bind-mounted from the host. It’s not ephemeral: the container itself gets recreated when a new image comes out, but the site lives on the host and survives that. Cancelling doesn’t vaporise anything either, a site is kept for a year after its subscription runs out.
But it does sleep. After 60 minutes without access the container stops, and the next request wakes it. There’s also a 20 hour per day uptime allowance per site, because an editor that never sleeps is indistinguishable from a server I’m paying to run for nobody.
WordPress has a heartbeat API, the admin interface posts to the server every minute or so for as long as a tab is open (a bit less often in the background, but it never stops on its own). For a long time the proxy counted every one of those posts as activity, so a tab left open overnight kept the container running until the daily allowance was used up, and after that you got the daily-limit page instead of your editor. And when the idle timer did manage to stop a container, the next heartbeat woke it straight back up.
Heartbeats are now recognised by what’s in the request rather than where it’s going (the same endpoint carries autosave and a lot of plugin work, so ignoring all of it would stop somebody mid-edit, which is worse than the original bug). They don’t reset the idle timer anymore and they can’t wake a sleeping container, but they still go through while it’s running, so autosave notices and post locking keep working.
I mention it because most bugs in this system look like that one. Nothing crashed and nothing alerted, the proxy was doing exactly what I’d told it to, and the only symptom was somebody locked out of their own editor by a limit they didn’t know they were using.
Turning WordPress into files
Publishing runs in a throwaway container next to the site. If a run dies halfway, the container goes with it and the reconciler sweeps up whatever archive it left behind (pass 7 above).
The export asks WordPress for the list of URLs through wp-cli instead of guessing from links, crawls those pages in parallel, and walks the DOM of each page exactly once, rewriting as it goes. Absolute links to the editing host become root-relative. External assets that would otherwise leak (avatars, for example) are downloaded and remapped to local paths. A 404 page is captured deliberately, because a crawl never finds one on its own.
Then it puts some of the absolute URLs back. Structured data, Open Graph tags and sitemap entries have to be absolute against the final public address, because social and search parsers reject relative URLs there. So: strip everything, then re-absolutise the specific things that need it. That second pass isn’t elegant, but the alternative is a set of special cases threaded through the whole rewrite.
The result is a zip, pushed to one of three destinations: a git repository over SSH, Cloudflare, or statichost.eu. In each case, the account belongs to the customer.
The trade-off
Anything that needs WordPress at request time doesn’t survive the export: native comments, WooCommerce checkout, membership areas, anything behind a login, and those are real losses.
What you get in exchange is a public site with no PHP, no database and no login form facing the internet, plus a private editor that’s only reachable through a capability URL and spends most of its life stopped.
That sentence got tested the week I was finishing this post, when two unauthenticated remote code execution bugs turned up in PHP itself. The other post going up today, Mostly PHP, and mostly unaffected, is what that was like from here.
For a blog, a portfolio, a small business site or documentation, that’s a good trade, and for a shop it isn’t.
Where this fits
None of the individual pieces here are new. Reconciliation loops, finite state machines, SQLite, static site generation, all of it is ordinary.
Most of the sections come down to the central question: what happens to this system when nobody is looking at it?
I don’t think this is the right architecture for a company with an operations team. It’s the architecture I can actually run, and I’d rather have a system I understand completely than one that’s theoretically better and practically opaque.
