I’m seeing close to 15 minute build times for Android on my game now, and roughly 10 minutes for a PC build. Rebuilding while testing fortunately usually doesn’t take more than 30 seconds if I don’t need to add any new textures or graphics. I’ve disabled texture compression for the Windows build but that only helps slightly improve the times. I require it on Android to avoid OpenGL out of memory issues.
Are you using bob scripts yet for making bundles?
I really need to set aside time and get to grips with bob, but I find it quite intimidating. Poor excuse, I know! I am aware there is documentation here, but if anyone has some readily available examples that would be much appreciated!
For example, something I’d use a lot would be a script that cranks out a bundle for each of the available PC platforms (Win64, Win32, Mac, Linux) into separate folders in a specified location.
2 Likes
Here is the sh script I use for making Steam builds. There’s more on the Steam side. 1 click to get builds bundled uploaded to Steam.
#!/bin/bash
title=$(less game.project | grep "^title = " | cut -d "=" -f2 | sed -e 's/^[[:space:]]*//')
title_no_space=$(echo -e "${title}" | tr -d '[[:space:]]')
contentroot=C:/steamworks/sdk/tools/ContentBuilder/content/
echo "Project: ${title}"
java -jar bob.jar --version
if [ ! -f bob.jar ]
then
echo "Unable to find bob.jar. Download it from d.defold.com."
exit 1
fi
bob() {
java -Xmx4g -jar bob.jar $@
}
build() {
platform=$1
shift 1
rm -rf build
bob --archive clean resolve build --build-report-html "build-report-${platform}.html" --texture-compression true --variant "release" --platform "${platform}" -e a@a.com -u fake_token --with-symbols
}
bundle() {
platform=$1
echo "${platform}"
shift 1
bob --platform ${platform} --bundle-output build/${platform} $@ bundle
}
move_to_steam() {
platform=$1
if [ "${platform}" == "x86_64-darwin" ]
then
echo "${title_no_space}_${platform}"
echo "Removing old ${contentroot}${title_no_space}_${platform}"
rm -rf "${contentroot}${title_no_space}_${platform}"
if ! cd build/${platform} ; then read -p "Build folder ${platform} not found! - Press Enter"; exit 1; fi
mkdir -p "${contentroot}${title_no_space}_${platform}"
cp -r * "${contentroot}${title_no_space}_${platform}"
cd ../..
else
echo "${title_no_space}_${platform}"
echo "Removing old ${contentroot}${title_no_space}_${platform}"
rm -rf "${contentroot}${title_no_space}_${platform}"
if ! cd "build/${platform}/${title}" ; then read -p "Build folder ${platform} not found! - Press Enter"; exit 1; fi
mkdir -p "${contentroot}${title_no_space}_${platform}"
cp -r * "${contentroot}${title_no_space}_${platform}"
cd ../../..
fi
}
do_platform() {
platform=$1
shift 1
echo -e "\n[Building ${platform}]"
build ${platform}
echo -e "\n[Bundling ${platform}]"
bundle ${platform}
echo -e "\n[Moving to Steam ${platform}]"
move_to_steam ${platform}
echo -e "\n[${platform} Done]"
}
# build / bundle / archive platforms
do_platform x86_64-win32
do_platform x86_64-linux
do_platform x86_64-darwin
# play some music to let us known it's done
ffplay fanfare.mp3 -autoexit -nodisp
echo -e "\n[All Done!! - You can now run upload_steam]"
The reason for mentioning bob in this case is that you can disable some of the steps that the editor makes mandatory, which may lower time to getting a bundle. We still do need a cache system for textures based on hash of file + build + target system + texture settings.
1 Like
Wow, that is really long. I assume you have native extensions? Which ones? Uploading extension content and getting a custom engine back will increase build times. BUT if you do not change extensions this should be a one time thing.
Applying texture compression is probably the problem in your case. How many atlases do you have? And what size?
@JCash We should look into some way to have bob measure the time of the individual build steps and generate a report at the end of a build.
bob will show more clearly where the time is being spent as it prints out the % and does the steps, if you add timers to the above script it could give step time estimates. The biggest time in our game bundles is always texture compression / processing. Lots of large atlases. Depending on compression level it can take dramatically longer, sharing texture profile for Android would be good to see too.
I think that’s my case as well. I’m only using Rendercam and Monarch in my project as far as extensions go. I disabled texture compression however and builds still take awhile… Do you know if models/textures not included in the Atlases are compressed or is there some detection system to make sure they are included? I have some large assets not included so I may have to find them all and clean them up.
Can you confirm / test doing a bundle with bob and disable some of the options?
Do you have a texture profile set in your project? Try setting its values to allow very large atlases and have fastest webp compression.
There’s an issue in bob that makes it compile assets even if they’re not actually used in game. Removing those assets would help a bit.
@Pkeod If you compile the folder twice with bob, and not cleaning in between, the old (compiled) textures should still be untouched during the build. Is that not the case for you?
E:g. run find . -iname "*.texturec" | xargs ls -la to print all the time stamps.
1 Like
In my ever-growing project I’m experiencing build times of around 30 minutes for iOS at the moment, slightly less for Android.
Three atlases at 2048x2048 or less, less than 100 model textures that come in at around 3.5 MB in total.
Dependencies:
https://github.com/rgrams/rendercam/archive/1.0.3.zip,https://github.com/britzl/monarch/archive/3.2.2.zip,https://github.com/britzl/gooey/archive/9.1.0.zip,https://github.com/AGulev/set_screen_position/archive/master.zip,https://github.com/indiesoftby/defold-hyper-trails/archive/1.0.0.zip,https://github.com/defold/extension-gpgs/archive/2.2.0.zip,https://github.com/animonger/extension-gamekit/archive/master.zip,https://github.com/AGulev/DefVideoAds/archive/refs/tags/3.7.1.zip,https://github.com/defold/extension-firebase-analytics/archive/1.4.2.zip,https://github.com/defold/extension-iap/archive/3.0.1.zip,https://github.com/GameAnalytics/defold-openssl/archive/1.0.0.zip,https://github.com/GameAnalytics/GA-SDK-DEFOLD/archive/3.3.10.zip,https://github.com/defold/extension-facebook/archive/refs/tags/6.2.0.zip
Is this to be expected, or may there be a bottleneck somewhere?
Can you post your texture profiles file?
Unless the dependencies change they do not matter. The first build will take a bit longer because you also need to build and download a custom engine. Subsequent builds will not have to do this.
My guess is that the texture compression is slowing you down. Have you tried disabling it? Or run it with fastest settings?
1 Like
Building with just those dependencies takes <30 seconds for me (using a live server).
So, I’d suspect the texture compression.
Although it does sound a bit long with 30 minutes for compressing these textures.
Are you using texture compression for this?
How long does it take when you don’t use compression?
Although the basis universal compressor uses multiple cores, the rest of the pipeline is single threaded, so if each texture itself has a “startup cost” when compressing, then 100+ textures will start to add up. But again, I’d think no more than a few seconds per texture.
2 Likes
I do daily builds and the first one when starting a project takes no time at all, then gradually the time increases as assets are added. The build times I’m seeing now is the longest I’ve seen in any project, hence the question.
Okay, then it’s not the dependencies causing the long build time. They change rarely, as you can expect.
I’m using the Default that comes with Defold v1.2.181:
path_settings {
path: "**"
profile: "Default"
}
profiles {
name: "Default"
platforms {
os: OS_ID_GENERIC
formats {
format: TEXTURE_FORMAT_RGBA
compression_level: BEST
}
mipmaps: true
}
}
A fresh export to iOS without texture compression took just over 20 minutes.
Could it have something to do with my internet connection? For some reason my area of London has pitiful upload speeds, maxing out at around 10-20Mbit/s.
Well, sure, it probably doesn’t help, but it’s affecting the time it takes to upload the source code/libraries, and then download the binary.
You can time a new clean project with only your dependencies. That way we’ll know how much time is spent building the executable, versus building the content.
The content building is always done locally.
Also if you’ve already built the game once, the custom engine is cached and will not build the next time (unless that source changes).
1 Like
I bundled the same project again, using texture compression, to get a more precise bundling time, and to see what the difference was. It took the same time as without actually, 21 minutes. The 30 minutes was a guesstimate before. At least now we know it’s unlikely to be the texture compression.
A new project, containing only the dependencies listed above, took 6 minutes to bundle.
I should mention I bundle as debug, too, on iOS. I tend to use debug so I can easily… debug stuff during development. Since Google Play doesn’t accept debug builds, those are always bundled as release.
1 Like
So 15 minutes (21-6) to build with uncompressed textures seems quite a lot, especially given the info you gave earlier.
If possible, I’d love to take a look at the project (or a repro case) for further investigation.
1 Like
Sent PM.
This might be what’s causing the extra time. Try testing it lower than best (like fast) and see if your times are lower.
Also why is BEST set as the default atm!?
1 Like
