Native Extensions

[Mathias_Westerdahl]

So, a clarification (or fix for my half-accurate explanation)

So the feature basically allows you to include all uploaded files. Through the “-Iupload” compile flag.
And the uploaded files will retain their relative paths (relative to the root folder of the project.
So in your case, your extension folder contains two extensions: exta and extb like so:

mathiaswesterdahl ~/work/projects/testinc  
$ tree -L 2 extension/
extension/
├── createext.sh
├── exta
│   ├── ext.manifest
│   ├── include
│   ├── lib
│   └── src
└── extb
    ├── ext.manifest
    ├── lib
    └── src

The include path needed to include a_func.h from extb then needs to be:

#include <extension/exta/include/a_func.h>

(Since I’m used to implementing extensions as standalone repositories, I’m used to seeing them on the “root level”. )

I will update the documentation with this info as well.

[gerrykwok7]

It works fine, thank you.BTW, need I explicitly set “-lupload” flag to ext.manifest?

[Mathias_Westerdahl]

You’re welcome!

There’s no need to set this flag. this flag is set automatically.

[Mathias_Westerdahl]

One thing we missed to communicate properly was the recent change to support iOS 12 by using the latest iPhoneOS12.1.sdk.

This SDK included changes that meant that we couldn’t use Clang 6 anymore, but instead had to revert to using Apple’s clang. And that version is <6 (It’s unclear what version they’re actually on).

The biggest change for your projects might be if you previously relied on C++11 being the default flavor, you might get build errors. Sorry about that.

The fix is then to enable C++11 for your extension:

x86_64-osx:
    context:
        flags: ["-std=c++11"]

[Mathias_Westerdahl]

Although we fairly recently added these documentation pages, it’s good to reiterate some of the best practices.

In this particular case, it’s about using std::string and knowing about the downsides to it.
Apart from added compile time, it also adds extra code bloat (as C++ templates does) and is slower than C strings.
Those are side effects you can notice yourself, but there’s another more sneaky part to it:

• ABI compatibility

An API is basically the “names” of structs and functions:

struct Foo;
Foo* Alloc();
void DoWork(Foo*);

And ABI is the Application Binary Interface, which means how the structs are actually implemented internally.

// library version 1
struct Foo {
    uint8_t data;
};
// library version 2
struct Foo {
    uint32_t data;
};

(This is a simplified case just to illustrate the point)

If two different libraries are built towards two different versions of the ABI, and you later want to use both libraries in your executable, you can get nasty issues. At best a linker error. Or you can get a runtime error or possibly a crash (because then you at least know about it!), or at worst, a hidden bug lingering on.

In this case, we didn’t get a linker error, but sure enough, the GCC version wasn’t supported on the platform in question (SteamOS):

symbol _ZTVNSt7__cxx1119basic_istringstreamIcSt11char_traitsIcESaIcEEE
version GLIBCXX_3.4.21 not defined in file libstdc++.so.6 with link time reference

In the Defold engine itself, we stay away from the problems by simply using C strings:

#include <string.h>
#include <stdio.h>
size_t strlen(const char *s)
int snprintf(char *str, size_t size, const char *format, …)
char *strncat(char *dest, const char *src, size_t n)

So, now you know a little more about the potential issues when using STL and std:: functions
and how to avoid them.

[selimanac]

Help us master! Do you have a simple code which you can share? @Mathias_Westerdahl

Defold 1.2.148 has been released

Simple example for dmJson (for parsing and accessing m_Nodes) would be very useful. Try->Build->Fail->Try-Build->Fail is a long processes for NEs

[Mathias_Westerdahl]

In it’s simplest form:

#include <dmsdk/dlib/json.h>
const char* data = ...; // the input data you've read from somewhere
dmJson::Document doc;
if (dmJson::RESULT_OK == dmJson::Parse(data, &doc)) {
    // do stuff... 
}

Internally, we use jsmn (Jasmine) and our API follows that as well.

I need to write a better example for the documentation, so this is very untested:
To traverse the DOM document:

int Traverse(dmJson::Document* doc, index) {
    if (index >= doc->m_NodeCount)
    	return -1;
    const dmJson::Node& n = doc->m_Nodes[index];
    const char* json = doc->m_Json; // the actual data
    uint32_t length = n.m_End - n.m_Start; // The range of indices that this token refers to

    switch (n.m_Type) {
    case dmJson::TYPE_PRIMITIVE:
    	if (length == 4 && memcmp(json + n.m_Start, "true", 4) == 0) {
            // ...
        }
        // more primitives
        return index + 1; // next token please!
    case dmJson::TYPE_STRING:
    	// a string ...
        return index + 1; // next token please!

    case dmJson::TYPE_ARRAY:
        ++index;
        for (int i = 0; i < n.m_Size; ++i) {
        	index = Traverse(doc, index);
            if (index < 0)
                return -1;
        }
        return index;
    case dmJson::TYPE_OBJECT: // a table
        if ((n.m_Size % 2) == 0) { // key+value
            ++index;
            for (int i = 0; i < n.m_Size; i += 2)
            {
                index = Traverse(doc, index);
                if (index < 0)
                    return -1;
                index = Traverse(doc, index);
                if (index < 0)
                    return -1;
            }
        	return index;
        }
    	// malformed
    	return -1;
    }
    return -1;
}

Edit: And this is how to use it:

if (Traverse(&doc, 0) < 0) { //index==0 is the root node
    //error
}

[selimanac]

more than enough, thank you.

[mts]

We have updated GameAnalytics SDK and removed std::string and it looks to work now with SteamOS as @Pkeod has also requested on our Github repo. The main change was to start using rapidjson instead of JsonCpp other changes was just to get used to using c strings instead of std::string.

[AGulev]

Current dependencies included in the engine (it could be useful for android NE developers):

    compile 'com.google.firebase:firebase-messaging:17.3.4'
    compile 'com.google.firebase:firebase-core:16.0.7'
    compile 'com.google.android.gms:play-services-base:16.0.1'
    compile 'com.android.support:support-v4:27.1.1'
    compile 'com.android.support:support-compat:27.1.1'
    compile 'com.android.support:support-core-utils:27.1.1'
    compile 'com.android.support:support-core-ui:27.1.1'
    compile 'com.android.support:support-media-compat:27.1.1'
    compile 'com.android.support:support-fragment:27.1.1'
    compile 'com.android.support:support-annotations:27.1.1'
    compile 'android.arch.core:common:1.1.0'
    compile 'android.arch.core:runtime:1.1.0'
    compile 'android.arch.lifecycle:common:1.1.1'
    compile 'android.arch.lifecycle:compiler:1.1.1'
    compile 'android.arch.lifecycle:extensions:1.1.1'
    compile 'android.arch.lifecycle:reactivestreams:1.1.1'
    compile 'android.arch.lifecycle:runtime:1.1.1'

[britzl]

We should add this to the native extension manual.

Also with pointing out is that we plan to move Firebase messages to it’s own extension. As well as Google Play Services.

[erlend_sh]

Might you also end up supporting the Rust language for native extensions in the future?

[Mathias_Westerdahl]

To be honest, I don’t think adding Rust will be on our radar anytime soon.
You can ofc precompile libraries offline, and add them to your extension