Native Extensions

selimanac · March 13, 2018, 10:58am

Similar problem with Defos NE after 1.2.123 update. Looks like there is a breaking changes on MacOSX10.13 sdk

clang++ -c -arch x86_64 -target x86_64-apple-darwin12 -isysroot /opt/MacOSX10.13.sdk/ -m64 -O2 -g -mmacosx-version-min=10.7 -DDM_PLATFORM_OSX -DLUA_BYTECODE_ENABLE   -Iupload/defos/include   -I/var/extender/sdk/46d8b7e63d7e0f0f4acd545c46d25ca2b227a806/defoldsdk//include -I/var/extender/sdk/46d8b7e63d7e0f0f4acd545c46d25ca2b227a806/defoldsdk//sdk/include  upload/defos/src/defos_mac.mm -obuild/defos_mac.mm_3.o
In file included from upload/defos/src/defos_mac.mm:7:
In file included from /opt/MacOSX10.13.sdk//System/Library/Frameworks/AppKit.framework/Headers/AppKit.h:10:
In file included from /opt/MacOSX10.13.sdk//System/Library/Frameworks/Foundation.framework/Headers/Foundation.h:128:
/opt/MacOSX10.13.sdk//System/Library/Frameworks/Foundation.framework/Headers/NSUUID.h:26:49: error: nullability specifier '_Nullable' cannot be applied to non-pointer type 'uuid_t' (aka 'unsigned char [16]')
- (instancetype)initWithUUIDBytes:(const uuid_t _Nullable)bytes;
                                                ^
/opt/MacOSX10.13.sdk//System/Library/Frameworks/Foundation.framework/Headers/NSUUID.h:29:30: error: nullability specifier '_Nonnull' cannot be applied to non-pointer type 'uuid_t' (aka 'unsigned char [16]')
- (void)getUUIDBytes:(uuid_t _Nonnull)uuid;
                             ^
2 errors generated.

Mathias_Westerdahl · March 13, 2018, 11:11am

I’m looking into this. Not sure why this fails

Current workarounds:

Build with bob.jar with “–defoldsdk fce7921da858a71876773c75920b74310ca7ac1f”
Use an 1.2.122 editor:
- http://d.defold.com/editor2/fce7921da858a71876773c75920b74310ca7ac1f/editor2/Defold-x86_64-darwin.dmg
- http://d.defold.com/editor2/fce7921da858a71876773c75920b74310ca7ac1f/editor2/Defold-x86_64-win32.zip

tacklemcclean · March 13, 2018, 11:20am

I heard there’s a Native Extension build server that Defold provides. I’m trying to understand the big picture concept of NE’s and building them.

Why is a build server needed? Is it always used, if you have a NE in your project?

For example, the FMOD extension by dapetcu21 - it is using bridge libraries that are pre-built in Visual Studio. Are the build servers not required in this case? But still used? Or not?

Are there multiple build steps to consider?

I’m lacking C++ experience but think it’s probably good to know what happens when.

Mathias_Westerdahl · March 13, 2018, 11:23am

Native extensions allows you, the user, to extend the engine with extra functionality. It can be a sound library, or an ads library, or something that interacts with the OS.

If your project contains a native extension, or has a dependency on one, we will build a custom engine for you, on our cloud server.

AGulev · March 13, 2018, 11:27am

I am not sure but the old version of clang can be the reason of this bug (some info here and here)

dapetcu21 · March 13, 2018, 1:59pm

I was initially compiling my own libs to remove the need to bundle the FMOD headers with the dependency (as I was not authorized to distribute them). Now that I am authorized to distribute FMOD, I decided to still keep this structure because the Lua bindings have a lot of glue code (with lots of templates and macros) that is slow to compile and I’d like not to have to make tea each time I build the game.

The build server is perfectly capable of compiling all of that stuff (weeeelll… eeeexcept on Windows, where I managed to make that particular version of the MSVC compiler crash), but I’m free to link against libraries I compiled on my own.

The build server is still needed to link the whole binary together with the Defold engine.

Also, the rationale behind having a build server is so that everybody can bundle for any platform no matter the OS they’re bundling from and without having to set up a complicated cross-compiler toolchain.

tacklemcclean · March 13, 2018, 2:11pm

Thanks, great explanation!

Mathias_Westerdahl · March 13, 2018, 3:49pm

@AGulev @selimanac @joshua.winters The server has been updated with a bug fix, and it now “works for me”. Could any of you verify the fix as well?

AGulev · March 13, 2018, 4:02pm

I checked DefUnityAds - it works!

joshua.winters · March 14, 2018, 1:08am

Works for me too

AGulev · March 28, 2018, 11:50am

Hi, can you share roadmap for NE. This is one of my favorite Defold features and I looking forward for next improvements.

Mathias_Westerdahl · March 28, 2018, 12:41pm

Hi!

Off the top of my head, there are a few bigger things that we wish to work on before we leave the alpha stage of the NE feature. The first ones are related to improving compiling speeds/latency for the builds. E.g., we want to cache code/extensions for some time, in order to reduce upload times, and also build times if an extension was already built by someone else (with correct version etc). And also, we’d like to investigate speeding up the windows builds too, since they’re currently quite long.

After that, I think improving the debug workflow should help a lot. At the same time, we’re also working on making sure we don’t get any unwanted downtime for the server, e.g. adding better alarms etc.

After that, I think it’s more down to requested features, and perhaps we could have a similar system as the editor2-issues for NE issues as well. I’ll mention this to the team and see what they think of it.

dapetcu21 · March 28, 2018, 6:49pm

Linking against shared libraries? pretty please?
Release builds when release is checked?
Access to rendering primitives?
Access to the input system?

Insality · April 1, 2018, 1:02pm

Trying to build Appodeal defold sdk (currently unsupported)
On android build is ok, but on iOS i got next thing:

“status”:500, “error”:“Internal Server Error”,“exception”:“java.lang.OutOfMemoryError”,“message”:“java.lang.OutOfMemoryError: Java heap space”,“path”:“/build/arm64-ios/…”

As I know, native extensions building on Defold servers. Is it problems on it? Locally I have increased java heap memory, seems its remote problem

britzl · April 1, 2018, 4:20pm

Yes, that’s a remote (build server) problem. @Mathias_Westerdahl can probably take a look on Tuesday.

Mathias_Westerdahl · April 3, 2018, 8:22am

Hi @Insality!
Sorry to see you had this build issue. The server was fine in general, but for some reason it choked on your project.
I tried building the appodeal demo but that seems kind of broken. Do you have a test case I can use to reproduce this issue? And can you still reproduce it?

Insality · April 4, 2018, 6:31am

Yea, still reproducing. Here link for the test project with appodeal (building on ios with this error)
Appodeal size too big
https://drive.google.com/open?id=1hZlSOFZvLHUXNU7wOMiORnO6J41QOsab

Mathias_Westerdahl · April 4, 2018, 8:48am

We’ve located the issue and are working on how to best fix it.

On another note, the Appodeal libraries for iOS are ~400mb! (32+64 bit)
So, when I managed to build it on my machine, the executable went from 6mb (32+64bit) to 46mb!
That’s almost 8x the size! (and yes, both sizes are stripped)

Oleg_The_Evangelist · April 4, 2018, 8:53am

I am talking to Appodeal now abut their Defold plugin, will also bring the size concern in. Probably would not help though

Insality · April 4, 2018, 9:50am

Thanks for fast reply
Android build with appodeal its +5mb to apk. Not so much as on iOS, as you said, buy too many on my opinion
40mb… holy sh

Currently I am not required Appodeal plugin, just tried to test it (and it’s unsupported now)