Compile C files with C++ compiler while building native extension (Issue-3743) (DEF-3084)

Questions
#1

I tried to create a native extension to access sqlite databases in lua module. I know there would be buildin sqlite support on some platforms (for example iOS / macOS) , just need to link the sqlite lib would be fine. But for other platforms, I have to include sqlite3.c in the src folder and compile it with the extension.

It seems Defold cloud build system compile .C files with C++ compiler, I got lots of errors when I built the sqlite3.c (Please see the attached screenshot for more detail)

Could please anyone help me out of this? Did I miss some build flags?

image
image.png1428Ă—1116 142 KB

How to create a Windows release build?
Permissive flag for native extension (DEF-3127) (SOLVED)
#2

Hmm, it seems that’s something to improve upon. I’ve added it as DEF-3084

In the meantime, I would suggest building your static libraries “offline”, and add those to the /libs/ folder.
No need really to build static libs all the time.

3 Likes
#3

Thanks @Mathias_Westerdahl, I’ll go with the static libs workaround. BTW, is HTML5 bundle support static libs? If the answer is YES, what kind of libs should I provide?

#4

BTW, it is amzing the forum is so active, glade to meet so many nice people here, thanks to all for your kind help.

4 Likes
#5

The HTML5 builds support regular c/c++ libraries, built with Emscripten.
Or you can have javascript “libraries”, modified to support Emscripten.

See our HTML5 example for an example, and also the Emscripten documentation for more info

1 Like
#6

Got it, thanks @Mathias_Westerdahl !

1 Like
#7

Don’t we have to use extern "C" { } for defining C functions in a C++ program ?

Long time since last time i’ve written C, but i would bet it has something to do with name mangling.

#8

Yes, I used extern “C” { } for the c functions declaration. This problem is not about the name mangling, it is caused by compiling C files with C++ compiler, which is more rigorous than C compiler, and think some C warnings as errors.

#9

Hi! Small bump to this. I’ve spent the past half an hour adding explicit casts to the codebase of tree-sitter (I’m writing a parser for an in-house DSL) and I’m still getting unexplicable linker errors (possibly related to mangling).

2 Likes
#10

We have not planned to do anything about this in the near future I’m afraid.

#11

What Björn said.
Also, in that link you posted, there’s a C++ version?

1 Like
#12

No. That’s just a grammar for C++ written in tree-sitter. Tree sitter is a parser generator. You give it a language grammar and it spits out a C file with your parser that you have to use in conjunction with the main tree-sitter lib, which is a C library itself.

Anyway, I managed to get it to work with some small changes, so this bug is not a blocker for me, but it’s a barrier to anyone that wants to use C libraries in their native extensions.

2 Likes
#13

And I think I found the blocker. On Linux, this compiles fine as a C file, with gcc, but fails when using g++ (works just fine on platforms that compile with clang):

Screenshot%20from%202019-08-16%2001-36-38
Screenshot from 2019-08-16 01-36-38.png3840Ă—2160 651 KB

If you look in parser.h, you’ll see that TSParseActionEntry is a very convoluted union.

parser.c can be found here, along with the accompanying tree_sitter/parser.h, which should both compile fine without any other dependencies, in case you want to reproduce: https://github.com/dapetcu21/tree-sitter-fuior/tree/master/src

As a workaround, I will probably disable this extension in production and pre-compile the DSL files in my build script.

#14

Well it’s certainly another case for this feature.

However, in cases like this, I would recommend building this library locally on your machine and putting the result in the extension’s /lib folder.

#15

True, it’s not completely unsurmountable, though making static libs for every platform is a logistical nightmare requiring 5 makefiles and at least two reboots or VMs every time I make a change. :disappointed_relieved:

#16

“when you make a change” ?
Isn’t this a 3rd party library? No need to change that unless they’ve updated it?
You don’t have to write C code, you can use C++?

#17

No, it’s not a third party library. It’s the parser code generated from my own grammar. Every time I make a change to the grammar, I have to re-generate that parser file.

It’s anyway easier and more optimal to pre-compile the DSL at build time in this particular case and omit the NE from release builds.

#18

Ok, I understand.

For the VM part, I’d recommend looking into using things like Docker.
Here’s a Ubuntu 18 Dockerfile that allows me to simply switch shell from osx to linux, and build the things I need, without any fuzz.

I don’t have a Windows equivalent ready for public use yet, but it’s the same principle, start a linux container, and cross compile to windows.

2 Likes
#19

Thanks! I’ll give Docker a go next time I’ll need to do this.

#20

Hi @Mathias_Westerdahl I’m going to bump this one too. When making native extensions with C libs (much easier than with C++ for lua binding) Im getting the same problems. Simple example: sqlite3. If you use the amalgamated C source to use as a native extension, it wont build. And the “-std=c99” thats normally available doesnt appear to be working on the clang-13 being used.
Like @dapetcu21 I dont really want to build a bunch of different platform libs, just to be able to call C functions that should drop in? Bit puzzled about this.
Are there any compile options I could use to fix this? I have a couple of other projects in the pipe using C libs, and now Im a little concerned I will be spending a big chunk of time “converting” the source (I did this for other C source libs before).