How do you test that your server infrastructure scales correctly and can handle the real world? Let’s look at how to do it properly.
When testing your infrastructure, you need a different mindset than when you test your game. It’s like the difference between testing an ice-cream flavour and testing an elevator. They just have very different criteria and approaches.
This can cause game developers a lot of problems when they make multiplayer games. So let’s look at what you need to know to test your infrastructure. And how to avoid the main pitfalls.
In a game, your primary concern is whether it’s fun. But with infrastructure, you’re testing whether it’s safe and reliable. Anybody who tests infrastructure knows that everything breaks eventually. The key is finding out when it breaks. And what happens when it does?
Here are the three most popular infrastructure tests:
All these tests apply to your network infrastructure, too. How many sessions can you run before your servers collapse? How long can you run your sessions before any memory leaks become a problem? What happens if you have a surge of players?
Using these tests you can figure out how stable your infrastructure is and come up with fixes ahead of time.
Make sure that you’re thoroughly planning out your tests. You want to come up with all the different possible scenarios. Testing infrastructure is all about considering the most extreme situations and planning accordingly. Trust us, those extreme situations will happen eventually. So ask yourself four questions:
Non-functional requirements are the metrics that affect your players beyond basic system stats. The system might technically work, but is it a good experience? For example, it could be how long it takes to join a match, how quickly you can spin up a new session, or your in-game latency.
You can only get insights from the data you track. So make sure you consider what metrics would affect your player’s experience before you start testing.
Every third-party tool you use is going to affect your overall infrastructure in some way. For example, your matchmaker, your orchestrator and your player database. Ideally, you’d test everything all together. But that’s not always possible.
For example, it might be too expensive or complicated to run every component at once. In those cases, separate out your tests into more manageable chunks and try to use mock or stub components to make things easier.
Saying that, it’s best to try and budget for an environment that mimics your final release as much as possible, even if it’s a smaller scale. That way, you can be pretty sure things are going to work when you finally release.
Observibility is key. You want to make sure you’re monitoring and logging key metrics, but also that you have a way to understand the data. Set up dashboards to view the information in a more manageable way.
Having a clear way to analyze your data will make it much easier to spot bottlenecks and the root cause of any problems.
There’s no right way to set up your testing environment. But if you want to make the process easier, we’d recommend using one of these open-source tools:
Which tool you choose depends on your architecture and what types of scenarios you want to run.
If you’re looking to build your own server infrastructure, we can help you with the overall architecture, set up our orchestrator to manage it automatically, and help test it works. Just get in touch.
