How to improve build times?

#1

I’m seeing close to 15 minute build times for Android on my game now, and roughly 10 minutes for a PC build. Rebuilding while testing fortunately usually doesn’t take more than 30 seconds if I don’t need to add any new textures or graphics. I’ve disabled texture compression for the Windows build but that only helps slightly improve the times. I require it on Android to avoid OpenGL out of memory issues.

0 Likes

#2

Are you using bob scripts yet for making bundles?

0 Likes

#3

I really need to set aside time and get to grips with bob, but I find it quite intimidating. Poor excuse, I know! I am aware there is documentation here, but if anyone has some readily available examples that would be much appreciated!

For example, something I’d use a lot would be a script that cranks out a bundle for each of the available PC platforms (Win64, Win32, Mac, Linux) into separate folders in a specified location.

1 Like

#4

Here is the sh script I use for making Steam builds. There’s more on the Steam side. 1 click to get builds bundled uploaded to Steam.

#!/bin/bash
title=$(less game.project | grep "^title = " | cut -d "=" -f2 | sed -e 's/^[[:space:]]*//')
title_no_space=$(echo -e "${title}" | tr -d '[[:space:]]')
contentroot=C:/steamworks/sdk/tools/ContentBuilder/content/

echo "Project: ${title}"

java -jar bob.jar --version

if [ ! -f bob.jar ]
	then
		echo "Unable to find bob.jar. Download it from d.defold.com."
		exit 1
fi

bob() {
	java -Xmx4g -jar bob.jar $@
}

build() {
	platform=$1
	shift 1
	rm -rf build
	bob --archive clean resolve build --build-report-html "build-report-${platform}.html" --texture-compression true --variant "release"  --platform "${platform}" -e a@a.com -u fake_token --with-symbols
}


bundle() {
	platform=$1
	echo "${platform}"
	shift 1
	bob --platform ${platform} --bundle-output build/${platform} $@ bundle
}

move_to_steam() {
	platform=$1
	if [ "${platform}" == "x86_64-darwin" ]
		then
			echo "${title_no_space}_${platform}"
			echo "Removing old ${contentroot}${title_no_space}_${platform}"
			rm -rf "${contentroot}${title_no_space}_${platform}"
			if ! cd build/${platform} ; then read -p "Build folder ${platform} not found! - Press Enter"; exit 1; fi
			mkdir -p "${contentroot}${title_no_space}_${platform}"
			cp -r * "${contentroot}${title_no_space}_${platform}"
			cd ../..	
	else
		echo "${title_no_space}_${platform}"
		echo "Removing old ${contentroot}${title_no_space}_${platform}"
		rm -rf "${contentroot}${title_no_space}_${platform}"
		if ! cd "build/${platform}/${title}" ; then read -p "Build folder ${platform} not found! - Press Enter"; exit 1; fi
		mkdir -p "${contentroot}${title_no_space}_${platform}"
		cp -r * "${contentroot}${title_no_space}_${platform}"
		cd ../../..
	fi
}

do_platform() {
	platform=$1
	shift 1
	echo -e "\n[Building ${platform}]"
	build ${platform}
	echo -e "\n[Bundling ${platform}]"
	bundle ${platform}
	echo -e "\n[Moving to Steam ${platform}]"
	move_to_steam ${platform}
	echo -e "\n[${platform} Done]"
}

# build / bundle / archive platforms

do_platform x86_64-win32
do_platform x86_64-linux
do_platform x86_64-darwin

# play some music to let us known it's done

ffplay fanfare.mp3 -autoexit -nodisp

echo -e "\n[All Done!! - You can now run upload_steam]"

The reason for mentioning bob in this case is that you can disable some of the steps that the editor makes mandatory, which may lower time to getting a bundle. We still do need a cache system for textures based on hash of file + build + target system + texture settings.

1 Like

#5

Wow, that is really long. I assume you have native extensions? Which ones? Uploading extension content and getting a custom engine back will increase build times. BUT if you do not change extensions this should be a one time thing.

Applying texture compression is probably the problem in your case. How many atlases do you have? And what size?

@JCash We should look into some way to have bob measure the time of the individual build steps and generate a report at the end of a build.

0 Likes

#6

bob will show more clearly where the time is being spent as it prints out the % and does the steps, if you add timers to the above script it could give step time estimates. The biggest time in our game bundles is always texture compression / processing. Lots of large atlases. Depending on compression level it can take dramatically longer, sharing texture profile for Android would be good to see too.

0 Likes

#7

I think that’s my case as well. I’m only using Rendercam and Monarch in my project as far as extensions go. I disabled texture compression however and builds still take awhile… Do you know if models/textures not included in the Atlases are compressed or is there some detection system to make sure they are included? I have some large assets not included so I may have to find them all and clean them up.

0 Likes

#8

Can you confirm / test doing a bundle with bob and disable some of the options?

Do you have a texture profile set in your project? Try setting its values to allow very large atlases and have fastest webp compression.

0 Likes

#9

There’s an issue in bob that makes it compile assets even if they’re not actually used in game. Removing those assets would help a bit.

@Pkeod If you compile the folder twice with bob, and not cleaning in between, the old (compiled) textures should still be untouched during the build. Is that not the case for you?
E:g. run find . -iname "*.texturec" | xargs ls -la to print all the time stamps.

1 Like