Unable to build for HTML5 when HTML5 native extension is included (DEF-2815)

Bugs
#1

Hi,

I am not able to make any HTML5 builds when there is a HTML5 native extension included. When building from the editor I get a “504 Gateway Time-out” error message. If I try to build using bob i get this error message:

Exception in thread “main” com.dynamo.bob.MultipleCompileException: Build error
at com.dynamo.bob.bundle.BundleHelper.buildEngineRemote(BundleHelper.java:298)
at com.dynamo.bob.Project.buildEngine(Project.java:643)
at com.dynamo.bob.Project.doBuild(Project.java:738)
at com.dynamo.bob.Project.build(Project.java:335)
at com.dynamo.bob.Bob.main(Bob.java:320)
Caused by: com.defold.extender.client.ExtenderClientException: Failed to build source.
at com.defold.extender.client.ExtenderClient.build(ExtenderClient.java:78)
at com.dynamo.bob.bundle.BundleHelper.buildEngineRemote(BundleHelper.java:290)
… 4 more

If I remove the HTML5 native extension from the project I am able to build. This has also been verified using this project: https://github.com/defold/extension-html5

Hope this can be fixed soon.

Thanks,
Martin
Lead SDK Developer @ GameAnalytics

2 Likes
Native extension - build error
#2

@Mathias_Westerdahl, do you know what’s wrong?

#3

It seems that the build request times out for some reason
We’re investigating why that is.

1 Like
#4

We’ve tweaked the server timeouts now, please try again!

1 Like
#5

Ok thanks. I don’t get that error anymore now, but I get:

ReferenceError: navigator is not defined

Is it not supported or am I doing something wrong? We use ‘navigator’ in our own js library to collect some metadata to send with our events.

#6

I think navigator should be supported, we use it to determine languages and userAgent.
Does the error message contain any more info?

#7

I get this under errors console:
Errors running builder ‘Content Builder’ on project ‘Test’.

org.eclipse.core.runtime.AssertionFailedException: assertion failed: Marker property value is too long: Build server output: em++ -c -O3 -g -DDM_PLATFORM_HTML5 -DGL_ES_VERSION_2_0 -fno-exceptions -s EXPORTED_FUNCTIONS=["_JSWriteDump","_main"] -Iupload/gameanalytics/include/ -I/var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//include -I/var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//sdk/include upload/gameanalytics/src/gameanalytics.cpp -o build/gameanalytics.cpp_0.o

em++ -c -O3 -g -DDM_PLATFORM_HTML5 -DGL_ES_VERSION_2_0 -fno-exceptions -s EXPORTED_FUNCTIONS=["_JSWriteDump","_main"] -Iupload/gameanalytics/include/ -I/var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//include -I/var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//sdk/include upload/gameanalytics/src/GameAnalyticsDefold.cpp -o build/GameAnalyticsDefold.cpp_1.o

em++ -c -O3 -g -DDM_PLATFORM_HTML5 -DGL_ES_VERSION_2_0 -fno-exceptions -s EXPORTED_FUNCTIONS=["_JSWriteDump","_main"] -Iupload/gameanalytics/include/ -I/var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//include -I/var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//sdk/include upload/gameanalytics/src/android/GameAnalyticsJNI.cpp -o build/GameAnalyticsJNI.cpp_2.o

emar rcs /tmp/job807670116296703591/build/liba2034de7-f1ab-4d2d-b0fa-265e8451712c.a build/gameanalytics.cpp_0.o build/GameAnalyticsDefold.cpp_1.o build/GameAnalyticsJNI.cpp_2.o

em++ -c -O3 -g -DDM_PLATFORM_HTML5 -DGL_ES_VERSION_2_0 -fno-exceptions -s EXPORTED_FUNCTIONS=["_JSWriteDump","_main"] -I/var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//include -I/var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//sdk/include build/main.cpp -o build/main_tmp652270ad-81e7-4bfc-ba82-3017a3dc1cbb.o

em++ build/main_tmp652270ad-81e7-4bfc-ba82-3017a3dc1cbb.o -o build/dmengine.js -O3 --pre-js /var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//share/js-web-pre.js --llvm-lto 0 --memory-init-file 0 -s PRECISE_F32=2 -s AGGRESSIVE_VARIABLE_ELIMINATION=1 -s TOTAL_MEMORY=268435456 -s DISABLE_EXCEPTION_CATCHING=1 -s EXPORTED_FUNCTIONS=["_JSWriteDump","_main"] -L/tmp/job807670116296703591/build -L/tmp/job807670116296703591/upload/gameanalytics/lib/js-web -L/var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//lib/js-web -L/var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//ext/lib/js-web -la2034de7-f1ab-4d2d-b0fa-265e8451712c -lengine -lwebviewext -ladtruthext -lprofilerext -lfacebookext -liapext -lpushext -liacext -lrecord_null -lgameobject -lddf -lresource -lgamesys -lgraphics -lphysics -lBulletDynamics -lBulletCollision -lLinearMath -lBox2D -lrender -lscript -llua -lextension -lhid -linput -lparticle -lrig -ldlib -ldmglfw -lgui -ltracking -lcrashext -lliveupdate -lsound --js-library /tmp/job807670116296703591/upload/gameanalytics/lib/js-web/GameAnalytics.js --js-library /tmp/job807670116296703591/upload/gameanalytics/lib/js-web/GameAnalyticsDefold.js --js-library /var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//lib/js-web/js/library_glfw.js --js-library /var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//lib/js-web/js/library_sys.js --js-library /var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//lib/js-web/js/library_script.js --js-library /var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//lib/js-web/js/library_facebook.js --js-library /var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//lib/js-web/js/library_facebook_iap.js --js-library /var/extender/sdk/8c2083883fdf321a3fa2aa1a1a5d319a82635c10/defoldsdk//lib/js-web/js/library_sound.js
WARNING:root:e[33mgenerating system library: libcxx_noexcept.a…e[0m
WARNING:root:e[33m oke[0m
WARNING:root:e[33mgenerating system library: libcxxabi.bc…e[0m
WARNING:root:e[33m oke[0m
WARNING:root:e[33mgenerating system library: libc.bc…e[0m
WARNING:root:e[33m oke[0m
WARNING:root:e[33mgenerating system library: dlmalloc.bc…e[0m
WARNING:root:e[33m oke[0m
error: failure to execute js library “/tmp/job807670116296703591/upload/gameanalytics/lib/js-web/GameAnalytics.js”: ReferenceError: navigator is not defined,ReferenceError: navigator is not defined
at e.device.e.device (eval at (eval at globalEval (/opt/emsdk/bin/emsdk_portable/emscripten/master/src/compiler.js:105:8)), :2:21757)
at gameanalytics (eval at (eval at globalEval (/opt/emsdk/bin/emsdk_portable/emscripten/master/src/compiler.js:105:8)), :2:22424)
at eval (eval at (eval at globalEval (/opt/emsdk/bin/emsdk_portable/emscripten/master/src/compiler.js:105:8)), :2:22450)
at eval (eval at (eval at globalEval (/opt/emsdk/bin/emsdk_portable/emscripten/master/src/compiler.js:105:8)), :7:3)
at Object.LibraryManager.load (eval at globalEval (/opt/emsdk/bin/emsdk_portable/emscripten/master/src/compiler.js:105:8), :146:9)
at JSify (eval at globalEval (/opt/emsdk/bin/emsdk_portable/emscripten/master/src/compiler.js:105:8), :61:20)
at /tmp/job807670116296703591/upload/gameanalytics/lib/js-web/GameAnalytics.js (/opt/emsdk/bin/emsdk_portable/emscripten/master/src/compiler.js:208:3)
at Module._compile (module.js:434:26)
at Object.Module._extensions…js (module.js:452:10)
at Module.load (module.js:355:32)
preprocessed source (you can run a js engine on this to get a clearer error message sometimes):

(function(scope){
var CryptoJS=CryptoJS||function(e,n){var t={},i=t.lib={},r=function(){},s=i.Base={extend:function(e){r.prototype=this;var n=new r;return e&&n.mixIn(e),n.hasOwnProperty(“init”)||(n.init=function(){n.$super.init.apply(this,arguments)}),n.init.prototype=n,n.$super=this,n},create:function(){var e=this.extend();return e.init.apply(e,arguments),e},init:function(){},mixIn:function(e){for(var n in e)e.hasOwnProperty(n)&&(this[n]=e[n]);e.hasOwnProperty(“toString”)&&(this.toString=e.toString)},clone:function(){return this.init.prototype.extend(this)}},o=i.WordArray=s.extend({init:function(e,n){e=this.words=e||[],this.sigBytes=void 0!=n?n:4e.length},toString:function(e){return(e||u).stringify(this)},concat:function(e){var n=this.words,t=e.words,i=this.sigBytes;if(e=e.sigBytes,this.clamp(),i%4)for(var r=0;r<e;r++)n[i+r>>>2]|=(t[r>>>2]>>>24-r%48&255)<<24-(i+r)%48;else if(65535<t.length)for(r=0;r<e;r+=4)n[i+r>>>2]=t[r>>>2];else n.push.apply(n,t);return this.sigBytes+=e,this},clamp:function(){var n=this.words,t=this.sigBytes;n[t>>>2]&=4294967295<<32-t%48,n.length=e.ceil(t/4)},clone:function(){var e=s.clone.call(this);return e.words=this.words.slice(0),e},random:function(n){for(var t=[],i=0;i<n;i+=4)t.push(4294967296e.random()|0);return new o.init(t,n)}}),a=t.enc={},u=a.Hex={stringify:function(e){var n=e.words;e=e.sigBytes;for(var t=[],i=0;i<e;i++){var r=n[i>>>2]>>>24-i%48&255;t.push((r>>>4).toString(16)),t.push((15&r).toString(16))}return t.join("")},parse:function(e){for(var n=e.length,t=[],i=0;i<n;i+=2)t[i>>>3]|=parseInt(e.substr(i,2),16)<<24-i%84;return new o.init(t,n/2)}},c=a.Latin1={stringify:function(e){var n=e.words;e=e.sigBytes;for(var t=[],i=0;i<e;i++)t.push(String.fromCharCode(n[i>>>2]>>>24-i%48&255));return t.join("")},parse:function(e){for(var n=e.length,t=[],i=0;i<n;i++)t[i>>>2]|=(255&e.charCodeAt(i))<<24-i%48;return new o.init(t,n)}},d=a.Utf8={stringify:function(e){try{return decodeURIComponent(escape(c.stringify(e)))}catch(e){throw Error(“Malformed UTF-8 data”)}},parse:function(e){return c.parse(unescape(encodeURIComponent(e)))}},l=i.BufferedBlockAlgorithm=s.extend({reset:function(){this._data=new o.init,this._nDataBytes=0},_append:function(e){“string”==typeof e&&(e=d.parse(e)),this._data.concat(e),this._nDataBytes+=e.sigBytes},_process:function(n){var t=this._data,i=t.words,r=t.sigBytes,s=this.blockSize,a=r/(4s),a=n?e.ceil(a):e.max((0|a)-this._minBufferSize,0);if(n=as,r=e.min(4n,r),n){for(var u=0;u<n;u+=s)this._doProcessBlock(i,u);u=i.splice(0,n),t.sigBytes-=r}return new o.init(u,r)},clone:function(){var e=s.clone.call(this);return e._data=this._data.clone(),e},_minBufferSize:0});i.Hasher=l.extend({cfg:s.extend(),init:function(e){this.cfg=this.cfg.extend(e),this.reset()},reset:function(){l.reset.call(this),this._doReset()},update:function(e){return this._append(e),this._process(),this},finalize:function(e){return e&&this._append(e),this._doFinalize()},blockSize:16,_createHelper:function(e){return function(n,t){return new e.init(t).finalize(n)}},_createHmacHelper:function(e){return function(n,t){return new f.HMAC.init(e,t).finalize(n)}}});var f=t.algo={};return t}(Math);!function(e){for(var n=CryptoJS,t=n.lib,i=t.WordArray,r=t.Hasher,t=n.algo,s=[],o=[],a=function(e){return 4294967296*(e-(0|e))|0},u=2,c=0;64>c;){var d;e:{d=u;for(var l=e.sqrt(d),f=2;f<=l;f++)if(!(d%f)){d=!1;break e}d=!0}d&&(8>c&&(s[c]=a(e.pow(u,.5))),o[c]=a(e.pow(u,1/3)),c++),u++}var v=[],t=t.SHA256=r.extend({_doReset:function(){this._hash=new i.init(s.slice(0))},_doProcessBlock:function(e,n){for(var t=this._hash.words,i=t[0],r=t[1],s=t[2],a=t[3],u=t[4],c=t[5],d=t[6],l=t[7],f=0;64>f;f++){if(16>f)v[f]=0|e[n+f];else{var g=v[f-15],m=v[f-2];v[f]=((g<<25|g>>>7)^(g<<14|g>>>18)^g>>>3)+v[f-7]+((m<<15|m>>>17)^(m<<13|m>>>19)^m>>>10)+v[f-16]}g=l+((u<<26|u>>>6)^(u<<21|u>>>11)^(u<<7|u>>>25))+(u&c^~u&d)+o[f]+v[f],m=((i<<30|i>>>2)^(i<<19|i>>>13)^(i<<10|i>>>22))+(i&r^i&s^r&s),l=d,d=c,c=u,u=a+g|0,a=s,s=r,r=i,i=g+m|0}t[0]=t[0]+i|0,t[1]=t[1]+r|0,t[2]=t[2]+s|0,t[3]=t[3]+a|0,t[4]=t[4]+u|0,t[5]=t[5]+c|0,t[6]=t[6]+d|0,t[7]=t[7]+l|0},_doFinalize:function(){var n=this._data,t=n.words,i=8this._nDataBytes,r=8n.sigBytes;return t[r>>>5]|=128<<24-r%32,t[14+(r+64>>>9<<4)]=e.floor(i/4294967296),t[15+(r+64>>>9<<4)]=i,n.sigBytes=4*t.length,this._process(),this._hash},
at org.eclipse.core.runtime.Assert.isTrue(Assert.java:110)
at org.eclipse.core.internal.resources.MarkerInfo.checkValidAttribute(MarkerInfo.java:64)
at org.eclipse.core.internal.resources.MarkerInfo.setAttribute(MarkerInfo.java:148)
at org.eclipse.core.internal.resources.Marker.setAttribute(Marker.java:243)
at com.dynamo.cr.editor.builders.ContentBuilder.buildLocal(ContentBuilder.java:202)
at com.dynamo.cr.editor.builders.ContentBuilder.build(ContentBuilder.java:81)
at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:726)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:199)
at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:321)
at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:396)
at org.eclipse.core.internal.resources.Project$1.run(Project.java:618)
at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2344)
at org.eclipse.core.internal.resources.Project.internalBuild(Project.java:597)
at org.eclipse.core.internal.resources.Project.build(Project.java:124)
at com.dynamo.cr.editor.handlers.LaunchHtmlHandler.buildProject(LaunchHtmlHandler.java:202)
at com.dynamo.cr.editor.handlers.LaunchHtmlHandler.access$1(LaunchHtmlHandler.java:168)
at com.dynamo.cr.editor.handlers.LaunchHtmlHandler$1.run(LaunchHtmlHandler.java:123)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)

#8

Ah, thx for reporting. I’ve added the ticket DEF-2815 for this.

2 Likes
#9

I think I understand the issue here. We support emscripten “JavaScript libraries”, which are specially constructed JavaScript files. These are hooked in at compile time with the compiler option “–js-library”, which in turn means that it’s executed in order to obtain information about variables etc.

To my knowledge, you cannot hook in “plain” .js-files into the emscripten build.

As I see it, you have two options:

  1. Create a Emscripten “JavaScript library” out of your file:
    Documentation: [https://kripken.github.io/emscripten-site/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.html]
    Example: library_browser.js (more)

  2. Put the .js file in the /res folder, to add it to your bundle.
    Then interact with that file, either via Native Extensions or Lua

Clarification, the DEF-2815 I created was regarding the "org.eclipse.core.runtime.AssertionFailedException: assertion failed: Marker property value is too long: ". I currently don’t think the “failure to execute js library” is something we can solve at this point.

1 Like
#10

Thanks for looking into this.

Unity and Unreal they make it possible to compile with Emscripten’s --pre-js (https://kripken.github.io/emscripten-site/docs/tools_reference/emcc.html#emcc-pre-js) if you rename the file extension to .jspre as a workaround for this problem.

So for now, if I move our javascript GameAnalytics library to gameanalytics/res/js-web (where ‘gameanalytics’ is the native extension folder) as you mention in option 2, will it when work or does the user need to add somthing to resulting .html file?

#11

That’s a good solution, I’ve added ticket DEF-2816 for supporting .jspre files

For option 2, you will need to add your script to the .html file, then the user should be able to interact with it using html5.run('gameanalytics.test_func()') from Lua (or some variation on this, see the example link in my previous post)

3 Likes