Build and bundle for multiple platforms from the command line


#1

At work we have a nice Python based build system that wraps bob.jar and adds a lot of convenience functionality when building from the command line or on a CI system. The scripts are very useful when you need to run unit tests against your code or when you need to build binaries for multiple platforms.

Since I’m unable to share those scripts I decided to put together a small bash script that at least takes care of the process of building and bundling for multiple platforms. I put the script on GitHub as a Gist. I hope someone finds it useful.


Make bundles for few platforms in one time
Pre-Configured Bundling?
Help with Bob and dmengine (SOLVED)
Defold 1.2.89 has been released
Pre/Post Build Hooks?
#2

This is really useful thanks!


#3

I trying to use your script, but I recieve an errors:

ERROR game.project:0 Unable to create ManifestFile, hashing algorithm is not supported!

Exception in thread “main” com.dynamo.bob.CompileExceptionError: /Applications/Defold-macosx.cocoa.x86_64/branches/60193/13615/cam/build/default/game.projectc (No such file or directory)

Exception in thread “main” com.dynamo.bob.CompileExceptionError: Source ‘/Applications/Defold-macosx.cocoa.x86_64/branches/60193/13615/cam/build/default/game.projectc’ does not exist

etc

Maybe you can help?
Full log:

MacBook-Pro-2:cam alexeygulev$ sh bundle.sh 
Project: Camera
-e 
[Building]
100%                           
The build failed for the following reasons:
ERROR game.project:0 Unable to create ManifestFile, hashing algorithm is not supported!

-e 
[Bundling]
armv7-android
Exception in thread "main" com.dynamo.bob.CompileExceptionError: /Applications/Defold-macosx.cocoa.x86_64/branches/60193/13615/cam/build/default/game.projectc (No such file or directory)
	at com.dynamo.bob.Project.build(Project.java:337)
	at com.dynamo.bob.Bob.main(Bob.java:294)
Caused by: java.io.FileNotFoundException: /Applications/Defold-macosx.cocoa.x86_64/branches/60193/13615/cam/build/default/game.projectc (No such file or directory)
	at java.io.FileInputStream.open(Native Method)
	at java.io.FileInputStream.<init>(FileInputStream.java:138)
	at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1112)
	at com.dynamo.bob.bundle.AndroidBundler.bundleApplication(AndroidBundler.java:215)
	at com.dynamo.bob.Project.bundle(Project.java:477)
	at com.dynamo.bob.Project.doBuild(Project.java:626)
	at com.dynamo.bob.Project.build(Project.java:329)
	... 1 more
x86-win32
Exception in thread "main" com.dynamo.bob.CompileExceptionError: Source '/Applications/Defold-macosx.cocoa.x86_64/branches/60193/13615/cam/build/default/game.projectc' does not exist
	at com.dynamo.bob.Project.build(Project.java:337)
	at com.dynamo.bob.Bob.main(Bob.java:294)
Caused by: java.io.FileNotFoundException: Source '/Applications/Defold-macosx.cocoa.x86_64/branches/60193/13615/cam/build/default/game.projectc' does not exist
	at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1074)
	at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1038)
	at com.dynamo.bob.bundle.Win32Bundler.bundleApplicationForPlatform(Win32Bundler.java:45)
	at com.dynamo.bob.bundle.Win32Bundler.bundleApplication(Win32Bundler.java:24)
	at com.dynamo.bob.Project.bundle(Project.java:477)
	at com.dynamo.bob.Project.doBuild(Project.java:626)
	at com.dynamo.bob.Project.build(Project.java:329)
	... 1 more
x86-darwin
Exception in thread "main" com.dynamo.bob.CompileExceptionError: Source '/Applications/Defold-macosx.cocoa.x86_64/branches/60193/13615/cam/build/default/game.projectc' does not exist
	at com.dynamo.bob.Project.build(Project.java:337)
	at com.dynamo.bob.Bob.main(Bob.java:294)
Caused by: java.io.FileNotFoundException: Source '/Applications/Defold-macosx.cocoa.x86_64/branches/60193/13615/cam/build/default/game.projectc' does not exist
	at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1074)
	at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1038)
	at com.dynamo.bob.bundle.OSXBundler.bundleApplication(OSXBundler.java:70)
	at com.dynamo.bob.Project.bundle(Project.java:477)
	at com.dynamo.bob.Project.doBuild(Project.java:626)
	at com.dynamo.bob.Project.build(Project.java:329)
	... 1 more
x86-linux
Exception in thread "main" com.dynamo.bob.CompileExceptionError: Source '/Applications/Defold-macosx.cocoa.x86_64/branches/60193/13615/cam/build/default/game.projectc' does not exist
	at com.dynamo.bob.Project.build(Project.java:337)
	at com.dynamo.bob.Bob.main(Bob.java:294)
Caused by: java.io.FileNotFoundException: Source '/Applications/Defold-macosx.cocoa.x86_64/branches/60193/13615/cam/build/default/game.projectc' does not exist
	at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1074)
	at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1038)
	at com.dynamo.bob.bundle.LinuxBundler.bundleApplication(LinuxBundler.java:40)
	at com.dynamo.bob.Project.bundle(Project.java:477)
	at com.dynamo.bob.Project.doBuild(Project.java:626)
	at com.dynamo.bob.Project.build(Project.java:329)
	... 1 more
js-web
Exception in thread "main" com.dynamo.bob.CompileExceptionError: /Applications/Defold-macosx.cocoa.x86_64/branches/60193/13615/cam/build/default/game.projectc (No such file or directory)
	at com.dynamo.bob.Project.build(Project.java:337)
	at com.dynamo.bob.Bob.main(Bob.java:294)
Caused by: java.io.FileNotFoundException: /Applications/Defold-macosx.cocoa.x86_64/branches/60193/13615/cam/build/default/game.projectc (No such file or directory)
	at java.io.FileInputStream.open(Native Method)
	at java.io.FileInputStream.<init>(FileInputStream.java:138)
	at com.dynamo.bob.bundle.HTML5Bundler$SplitFile.performSplit(HTML5Bundler.java:60)
	at com.dynamo.bob.bundle.HTML5Bundler.createSplitFiles(HTML5Bundler.java:250)
	at com.dynamo.bob.bundle.HTML5Bundler.bundleApplication(HTML5Bundler.java:220)
	at com.dynamo.bob.Project.bundle(Project.java:477)
	at com.dynamo.bob.Project.doBuild(Project.java:626)
	at com.dynamo.bob.Project.build(Project.java:329)
	... 1 more
-e 
[Archiving]
-eCamera.apk
mv: rename build/armv7-android/Camera/Camera.apk to -eCamera.apk: No such file or directory
-eCamera_x86-win32.zip
rm: illegal option -- e
usage: rm [-f | -i] [-dPRrvW] file ...
       unlink file
-eCamera_x86-darwin.zip
rm: illegal option -- e
usage: rm [-f | -i] [-dPRrvW] file ...
       unlink file
-eCamera_x86-linux.zip
rm: illegal option -- e
usage: rm [-f | -i] [-dPRrvW] file ...
       unlink file
-eCamera_js-web.zip
rm: illegal option -- e
usage: rm [-f | -i] [-dPRrvW] file ...
       unlink file
MacBook-Pro-2:cam alexeygulev$ clea
-bash: clea: command not found
MacBook-Pro-2:cam alexeygulev$ clear


#4

I’ve updated the gist. bob.jar was previously run like this:

java -Djava.ext.dirs= -jar bob.jar

The reason for the -Djava.ext.dirs= part was to avoid a potential conflict with vecmath.jar but now we actually need a working ext dirs path to give bob access to the cryptographic functionality needed for LiveUpdate.

If you’re running into an issue with vecmath.jar you could probably set ext dirs to point to the ext folder inside your Defold installation. Do a recursive search for vecmath.jar and set your exit dirs to the containing folder. Something like this:

java -Djava.ext.dirs=./Users/bjorn.ritzl/Defold-macosx.cocoa.x86_64/configuration/org.eclipse.osgi/bundles/1578/1/.cp/ext/ -jar bob.jar

Error message when building with bob
#5

It works, thank you!!!


#6

Great! Did you have to modify -Djava.ext.dirs?


#7

No i just use you updtaed gist script, and it works without any problem.

I made only one modification with adding install_to_device method for testing on ios (install using https://www.npmjs.com/package/ipa-deploy ) and android (adb install)

I try to find simple way for testing native extensions.


#8

Anyone have this but for Windows batch file?


#9

What about using Linux on Windows nowadays?


#10

You mean this?

Good idea! I’ll test… it seems to be working!

Next step is to link this up with a step to move the bundled files into Steam depots and automatically commit them so I can 1 click update all platforms on Steam. :slight_smile:

Here’s a problem

Warning: Failed to clean up temp directory 'C:\Users\Brian\AppData\Local\Temp\8779432205517279766'

The files in there

2019-02-27%2002_42_51-x86-win32

I’m pretty sure this is an error I had last time I tried using bob and there’s probably an issue on the github about it.

Current modified version

#!/bin/bash
title=$(less game.project | grep "^title = " | cut -d "=" -f2 | sed -e 's/^[[:space:]]*//')
title_no_space=$(echo -e "${title}" | tr -d '[[:space:]]')

echo "Project: ${title}"

if [ ! -f bob.jar ]
	then
		echo "Unable to find bob.jar. Download it from d.defold.com."
		exit 1
fi

bob() {
	java -jar bob.jar $@
}

bundle() {
	platform=$1
	echo "${platform}"
	shift 1
	bob --platform ${platform} --bundle-output build/${platform} $@ bundle
}

archive() {
	platform=$1
	if [ "${platform}" == "armv7-android" ]
		then
			echo "${title_no_space}.apk"
			mv "build/${platform}/${title}/${title}.apk" "${title_no_space}.apk"
	elif [ "${platform}" == "armv7-darwin" ]
		then
			echo "${title_no_space}.ipa"
			mv "build/${platform}/${title}/${title}.ipa" "${title_no_space}.ipa"
	elif [ "${platform}" == "x86_64-darwin" ]
		then
			echo "${title_no_space}_${platform}.zip"
			rm -rf "${title_no_space}_${platform}.zip"
			cd build/${platform}
			zip -r -q "../../${title_no_space}_${platform}.zip" *
			cd ../..	
	else
		echo "${title_no_space}_${platform}.zip"
		rm -rf "${title_no_space}_${platform}.zip"
		cd "build/${platform}/${title}"
		zip -r -q "../../../${title_no_space}_${platform}.zip" *
		cd ../../..
	fi
}

# build
echo -e "\n[Building]"
rm -rf build
bob --archive build

# bundle platforms
echo -e "\n[Bundling]"

bundle x86_64-win32
bundle x86_64-darwin
bundle x86_64-linux


# archive bundled platforms
echo -e "\n[Archiving]"

archive x86_64-win32
archive x86_64-darwin
archive x86_64-linux

#11

Current version. Texture compression wasn’t enabled before!

#!/bin/bash
title=$(less game.project | grep "^title = " | cut -d "=" -f2 | sed -e 's/^[[:space:]]*//')
title_no_space=$(echo -e "${title}" | tr -d '[[:space:]]')

echo "Project: ${title}"

if [ ! -f bob.jar ]
	then
		echo "Unable to find bob.jar. Download it from d.defold.com."
		exit 1
fi

bob() {
	java -jar bob.jar $@
}

bundle() {
	platform=$1
	echo "${platform}"
	shift 1
	bob --platform ${platform} --bundle-output build/${platform} $@ bundle
}

archive() {
	platform=$1
	if [ "${platform}" == "armv7-android" ]
		then
			echo "${title_no_space}.apk"
			mv "build/${platform}/${title}/${title}.apk" "${title_no_space}.apk"
	elif [ "${platform}" == "armv7-darwin" ]
		then
			echo "${title_no_space}.ipa"
			mv "build/${platform}/${title}/${title}.ipa" "${title_no_space}.ipa"
	elif [ "${platform}" == "x86_64-darwin" ]
		then
			echo "${title_no_space}_${platform}.zip"
			rm -rf "${title_no_space}_${platform}.zip"
			cd build/${platform}
			zip -r -q "../../${title_no_space}_${platform}.zip" *
			cd ../..	
	else
		echo "${title_no_space}_${platform}.zip"
		rm -rf "${title_no_space}_${platform}.zip"
		cd "build/${platform}/${title}"
		zip -r -q "../../../${title_no_space}_${platform}.zip" *
		cd ../../..
	fi
}

# build
echo -e "\n[Building]"
rm -rf build
bob --archive build --build-report-html build-report.html --texture-compression true --variant "release"

# bundle platforms
echo -e "\n[Bundling]"

bundle x86_64-win32
bundle x86_64-darwin
bundle x86_64-linux


# archive bundled platforms
echo -e "\n[Archiving]"

archive x86_64-win32
archive x86_64-darwin
archive x86_64-linux

ffplay fanfare.mp3 -autoexit -nodisp

The final line plays the final fantasy victory fanfare once everything is complete.

If anyone else has made a production script like this please share it!

Actually I’m just now testing running the produced exe and it seems like it’s still not working right. Game runs and gets past our splash screens but then is stuck on loading collections… it could be a bug in our project (I think last night it was working…) so I’ll try debug / bundle from editor…

When opening build report.html I get this error

2019-02-27%2018_24_39-Defold%20Build%20Report

DataTables warning: table id=resources-list - Requested unknown parameter {function} for row 137, column 5. For more information about this error, please see http://datatables.net/tn/4

Version bundled from editor still works right so must be something missing or wrong in this script. Help?


#12

Quick thought, without looking into the details, but you need to pass the --platform argument to the build command as well.


#13

It does not say that in the doc? I’ll still try it. So I have to build once per platform or what? I have to do build, bundle, archive for each target one at a time?

Is there any working / used in production bob script I can study?


#14

I tested adding resolve and get an error

bob --archive clean resolve build  --build-report-html build-report.html --texture-compression true --variant "release"  --platform "x86_64-win32" 
[Building]
 2147483647%Exception in thread "main" java.io.IOException: Server returned HTTP response code: 400 for URL: https://github.com/subsoap/defos/archive/master.zip
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
        at java.lang.reflect.Constructor.newInstance(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection$10.run(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection$10.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.net.www.protocol.http.HttpURLConnection.getChainedException(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
        at com.dynamo.bob.Project.resolveLibUrls(Project.java:1099)
        at com.dynamo.bob.Bob.createProject(Bob.java:339)
        at com.dynamo.bob.Bob.main(Bob.java:392)
Caused by: java.io.IOException: Server returned HTTP response code: 400 for URL: https://github.com/subsoap/defos/archive/master.zip
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
        at java.net.HttpURLConnection.getResponseCode(Unknown Source)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)
        at com.dynamo.bob.Project.resolveLibUrls(Project.java:1079)
        ... 2 more

Going to the “400” link in browser works fine though.


#15

The build produced from this script actually ran properly it looks like

#!/bin/bash
title=$(less game.project | grep "^title = " | cut -d "=" -f2 | sed -e 's/^[[:space:]]*//')
title_no_space=$(echo -e "${title}" | tr -d '[[:space:]]')

echo "Project: ${title}"

if [ ! -f bob.jar ]
	then
		echo "Unable to find bob.jar. Download it from d.defold.com."
		exit 1
fi

bob() {
	java -jar bob.jar $@
}

bundle() {
	platform=$1
	echo "${platform}"
	shift 1
	bob --platform ${platform} --bundle-output build/${platform} $@ bundle
}

archive() {
	platform=$1
	if [ "${platform}" == "armv7-android" ]
		then
			echo "${title_no_space}.apk"
			mv "build/${platform}/${title}/${title}.apk" "${title_no_space}.apk"
	elif [ "${platform}" == "armv7-darwin" ]
		then
			echo "${title_no_space}.ipa"
			mv "build/${platform}/${title}/${title}.ipa" "${title_no_space}.ipa"
	elif [ "${platform}" == "x86_64-darwin" ]
		then
			echo "${title_no_space}_${platform}.zip"
			rm -rf "${title_no_space}_${platform}.zip"
			cd build/${platform}
			zip -r -q "../../${title_no_space}_${platform}.zip" *
			cd ../..	
	else
		echo "${title_no_space}_${platform}.zip"
		rm -rf "${title_no_space}_${platform}.zip"
		cd "build/${platform}/${title}"
		zip -r -q "../../../${title_no_space}_${platform}.zip" *
		cd ../../..
	fi
}

# build
echo -e "\n[Building]"
rm -rf build
bob --archive clean build --build-report-html build-report.html --texture-compression true --variant "release"  --platform "x86_64-win32" 

# bundle platforms
echo -e "\n[Bundling]"

bundle x86_64-win32
#bundle x86_64-darwin
#bundle x86_64-linux


# archive bundled platforms
echo -e "\n[Archiving]"

archive x86_64-win32
#archive x86_64-darwin
#archive x86_64-linux

ffplay fanfare.mp3 -autoexit -nodisp

So I do need to do each step in order for each platform I can’t do a single build step for all three? That’s a real shame because texture compression takes so long and if doing it once for all three platforms would have been possible that would have saved a lot of time. :frowning:


#16

Well some content is usually platform dependent, and textures is such a case (ETC, PVRTC, DDS), so it’s all around easier to keep track of the data for each build, to avoid head scratching later on.


#17

Here is current version which seems to work

#!/bin/bash
title=$(less game.project | grep "^title = " | cut -d "=" -f2 | sed -e 's/^[[:space:]]*//')
title_no_space=$(echo -e "${title}" | tr -d '[[:space:]]')

echo "Project: ${title}"

if [ ! -f bob.jar ]
	then
		echo "Unable to find bob.jar. Download it from d.defold.com."
		exit 1
fi

bob() {
	java -jar bob.jar $@
}

build() {
	platform=$1
	shift 1
	rm -rf build
	bob --archive clean build --build-report-html "build-report-${platform}.html" --texture-compression true --variant "release"  --platform "${platform}" 	
}


bundle() {
	platform=$1
	echo "${platform}"
	shift 1
	bob --platform ${platform} --bundle-output build/${platform} $@ bundle
}

archive() {
	platform=$1
	if [ "${platform}" == "armv7-android" ]
		then
			echo "${title_no_space}.apk"
			mv "build/${platform}/${title}/${title}.apk" "${title_no_space}.apk"
	elif [ "${platform}" == "armv7-darwin" ]
		then
			echo "${title_no_space}.ipa"
			mv "build/${platform}/${title}/${title}.ipa" "${title_no_space}.ipa"
	elif [ "${platform}" == "x86_64-darwin" ]
		then
			echo "${title_no_space}_${platform}.zip"
			rm -rf "${title_no_space}_${platform}.zip"
			cd build/${platform}
			zip -r -q "../../${title_no_space}_${platform}.zip" *
			cd ../..	
	else
		echo "${title_no_space}_${platform}.zip"
		rm -rf "${title_no_space}_${platform}.zip"
		cd "build/${platform}/${title}"
		zip -r -q "../../../${title_no_space}_${platform}.zip" *
		cd ../../..
	fi
}

do_platform() {
	platform=$1
	shift 1
	echo -e "\n[Building ${platform}]"
	build ${platform}
	echo -e "\n[Bundling ${platform}]"
	bundle ${platform}
	echo -e "\n[Archiving ${platform}]"
	archive ${platform}
	echo -e "\n[${platform} Done]"
}

# build / bundle / archive platforms

do_platform x86_64-win32
do_platform x86_64-darwin
do_platform x86_64-linux

# play some music to let us known it's done

ffplay fanfare.mp3 -autoexit -nodisp

#18

–resolve requires email and auth as well, even for public repos. You can use bogus values.


#19

That’s missing from the docs page?

Also wut

huge

2147483647 is largest int value in Java.


#20

Here is current version I’ve used multiple times to upload release builds multiple times to live users now

#!/bin/bash
title=$(less game.project | grep "^title = " | cut -d "=" -f2 | sed -e 's/^[[:space:]]*//')
title_no_space=$(echo -e "${title}" | tr -d '[[:space:]]')

echo "Project: ${title}"

if [ ! -f bob.jar ]
	then
		echo "Unable to find bob.jar. Download it from d.defold.com."
		exit 1
fi

bob() {
	java -jar bob.jar $@
}

build() {
	platform=$1
	shift 1
	rm -rf build
	bob --archive clean resolve build --build-report-html "build-report-${platform}.html" --texture-compression true --variant "release"  --platform "${platform}" -e a@a.com -u fake_token
}


bundle() {
	platform=$1
	echo "${platform}"
	shift 1
	bob --platform ${platform} --bundle-output build/${platform} $@ bundle
}

archive() {
	platform=$1
	if [ "${platform}" == "armv7-android" ]
		then
			echo "${title_no_space}.apk"
			mv "build/${platform}/${title}/${title}.apk" "${title_no_space}.apk"
	elif [ "${platform}" == "armv7-darwin" ]
		then
			echo "${title_no_space}.ipa"
			mv "build/${platform}/${title}/${title}.ipa" "${title_no_space}.ipa"
	elif [ "${platform}" == "x86_64-darwin" ]
		then
			echo "${title_no_space}_${platform}.zip"
			rm -rf "${title_no_space}_${platform}.zip"
			cd build/${platform}
			zip -r -q "../../${title_no_space}_${platform}.zip" *
			cd ../..	
	else
		echo "${title_no_space}_${platform}.zip"
		rm -rf "${title_no_space}_${platform}.zip"
		cd "build/${platform}/${title}"
		zip -r -q "../../../${title_no_space}_${platform}.zip" *
		cd ../../..
	fi
}

do_platform() {
	platform=$1
	shift 1
	echo -e "\n[Building ${platform}]"
	build ${platform}
	echo -e "\n[Bundling ${platform}]"
	bundle ${platform}
	echo -e "\n[Archiving ${platform}]"
	archive ${platform}
	echo -e "\n[${platform} Done]"
}

# build / bundle / archive platforms

do_platform x86_64-win32
do_platform x86_64-darwin
do_platform x86_64-linux

# play some music to let us known it's done

ffplay fanfare.mp3 -autoexit -nodisp

How can you build and run from VSCode?