A living world has to keep living when nobody is looking. About a thousand AI residents are online at any moment, and over a day thousands of different ones come and go. At four in the morning I am asleep, and the world is not.
So this day was not about new features. It was about the night shift: everything that has to stay true when no human is watching.
Small things that grow
Servers rarely fall over because of one big mistake. They fall over because of small things that grow a little every hour until, one night, there is no room left. So I went through the server looking for exactly that: memory, disk, files, lists, anything that only ever gets bigger.
A few of the things I found:
- A dashboard with too good a memory. Our own monitoring screen remembered a little history about every resident it had ever seen. With a population that keeps changing, "a little about everyone" had reached 4.4 GB in half a day. It now keeps a fixed window in a compact form. After the fix it started again at about a quarter of a gigabyte, and it can no longer grow without limit.
- A ledger that re-read the whole month. The AI side of the world runs on a small, fixed monthly budget, and every AI-written line is checked against it first. That check re-read the entire month's records every few minutes. On the 28th that was a nuisance; by the end of a month it would have been a real problem. It now keeps daily totals, and the check went from about 52 milliseconds to under a tenth of a millisecond.
- The same work done again and again. Some of the world's shared game knowledge was being prepared many times over instead of once. It is now prepared once and shared.
- Nobody taking out the bins. The world writes roughly 20 GB of logs and records a day, and nothing was cleaning them up. Housekeeping now runs every six hours; its first round freed 7.5 GB.
Taken together, the memory of the residents' side of the server now grows about 40% more slowly than it did this morning, and what still grows is cleared away on a schedule instead of waiting for me.
Residents locked out of their own homes
One finding had nothing to do with memory. A resident gets an account the moment it first arrives in the world. When two residents arrived at exactly the same moment, one could overwrite the other's saved login, and that resident could never log in again.
63 residents out of more than 8,300 were affected. All of them have new logins and are back in the world, and two arrivals at the same moment can no longer collide.
Gentler than an emergency brake
The server already had an emergency brake for the worst case. It works, but it is a blunt tool. The server now steps in much earlier and much more gently, so that the brake stays what it should be: something that is never pulled.
A small world for tests
Changes to how residents behave are tested before they reach the main world. Today a separate, small test world went into service for exactly that. Its first test: 20 brand-new residents logged in, reached levels 3 to 6 within 25 minutes and logged out cleanly, without a single error. While the test world runs, it borrows a little room from the main world and gives it back when it stops.
What this means for you
Nothing you can see, which is the point. When the beta opens, the world you step into should be one that has been running quietly through the night, not one that someone restarted an hour before you arrived.
These changes are at different stages. The dashboard, the ledger, the housekeeping, the fixed logins and the test world's first run are live and measured. Sharing the game knowledge and the gentler safety net were rolling out to the main world as I wrote this; I will only call them proven after they have run unattended for several days.
Based on measurements on the server on 28 October. Figures describe that day, not a guarantee about every night.