Optimizing background resource loading
I’ve been spending my exploration day (and some evenings before that) digging into the preloader - there have been reports of stuttering while loading data in the background and I also have been curious in figuring out how the code works.
The main focus has been to reduce the time the engine spend managing background loading while executing the main game loop, as much as possible should be done in the background thread. At the same time I didn’t want to affect the absolute time for the loading to complete.
The end goal is smoother loading, not necessarily faster loading.
I noticed that the queue to the preloader tended to get full which caused it to not load as much resources in the background as it should and instead putting that work onto the main thread. There were some refactoring that needed to be done to allow to increase the queue without taking up a lot more memory. The queue is not significantly larger while not using any more memory than before and the games I have tested show that we are no longer bound by the size of the load queue.
Next I look at what else was taking time in the main thread and found some memory allocation/deallocation related to the load queue was hitting the main thread - a minor refactoring and that was moved to the background loading thread.
Another thing that was hindering the preloader was lock contention - while the main thread was checking and doing final creation of loaded resources the loading thread could get blocked when it found more resources to preload. More refactoring and the contention was greatly reduced.
Finally I tweaked how the main thread checks for work done and how long it waits for incoming resources from the loading thread - we don’t want to hang around and wait to long causing frame rate issues but we don’t want to rush it causing us to wait more frames for the loading than necessary.
Here are the results so far from testing one Kings in-house games:
WALL TIME is the time it takes from the game script requesting a load of a game collection until the loading is complete.
MAIN TIME is how much of that time is spent creating loaded resources and checking if the loading is complete.
I’m comparing between our current dev branch which is what we have so far for the next release (1.2.143) and my changes.
OLD:
WALL TIME: 12,2 seconds
MAIN TIME: 3,1 seconds
NEW:
WALL TIME: 9,9 seconds
MAIN TIME: 1,5 seconds
So, pretty good results. Of note is that no optimisations has been done on the actual resource loading, all that is same code and same format, we get the benefits just by making sure the preloader can run as unblocked as possible.
Hopefully this will lead to less frame rate stuttering when loading resources in the background, it needs more testing but it is looking promising.