Doctacosa

Sometimes you keep things optimized and under control, and sometimes you don't.

An issue I noticed a few months ago is that, whenever I made a news post, the main website and everything hosted on it (Creeper's Lab, Interordi and so on) would stop responding for several minutes. Obviously, having the website become unavailable right as I tell people there's news is not a good thing. Yet, it took me quite a while to understand what was going on.

Back in February, I decided to step into the Fediverse by launching a Misskey server. Soon after, I added an account for the Creeper's Lab, where the news would be automatically posted. You can find this one at @CreepersLab@social.interordi.com. I didn't realize it at the time, but I had planted the seed of the problem right then.

The way that Misskey, Mastodon and the wider Fediverse work is through a series of independent, interconnected servers that exchange messages with each other. While I made my news post locally, it would get sent and published on all other connected servers. Each one, in turn, checks the address of the news post to load some information about it: title, key image, summary and so on.

As the connections grew, so did the number of requests on the site. These all happen very quickly after a post. My latest one triggered 373 different servers to request information within a few minutes. Most read the news article itself, plus the key visual. Some also scanned for additional metadata from the website, or even checked its front page. But surely the website can handle 373 visitors at the same time?

Well, yes... or so I thought. One small element on the website is the list of currently online players, displayed on the sidebar. What I didn't realize at the time is that the list of players was loaded anew for each new visitor. This means that the website asked the games server to return the list of players, each time. All this additional data exchange was the source of the slowdown.

Once I realized this, the fix was simple: set a script that would load the players every minute and save the result locally. With this in place, no external call was required, no database connection to establish, just a very small text file to load and display. It instantly solved the issue, with my latest announcements having no negative effects on the availability of the website!

#social_media #projects #fediverse

– Doctacosa