One of the worst bugs your game can encounter is a race condition. It happens when multiple threads try to access the data and usually is deceptively elusive – game can run fine for a hour just to crash without any clear reason.
When players begun to report occasional random crashes, I was certain that race condition is to blame. Usually you find such bugs by attaching debugger to the game and triggering in – but how do you do that if there is no clear way to replicate the crash? You just need to play for hours and hours, until you hit it.
After couple of tries I came to idea to rig the NPC AI to players ship. It should work just fine, since AI is not cheating in any way. And it worked!
The AI-controlled ship performed multiple full dives, flying, digging and actually making profit. And it did find all the bugs while doing that. It took a full day, but it required minimal effort on my part – I just sold the output when the ship was in the base and performed repairs – and waited.
Turns out all my problems were due to a bug present in Godot 3.1.1 audio implementation. It’s already fixed in 3.2, but it’s not out yet – so I settled for a workaround. And tested it extensively with the AI.
I always wanted to have automated tests for my game, but never could figure out a good way to perform it. Using in-game AI opens a lot possibilities – this is true end-to-end test, with all components of the game participating. Now I’m trying to figure a way to adapt this method to test other aspects of the game – and to integrate automated testing into my build pipeline.