Packaging SDK's for Defold Extender

Questions
#1

Hello everybody, this is my first question for the Defold community. I am currently trying to create a native extension for a Lua framework, LÖVE, that I can put into my Defold project directory. I am running WSL 2 on Windows for context. As of right now, I have packaged the SDK’s needed for Android, Windows, HTML5, and Linux, but I am at an impasse for Apple’s XCode. With my “Xcode_15.0.1.xip” file in the local_sdk folder, I have tried unpacking using 7z, giving me header files and a large content file. Overall, my research has lead me to trying to use xar to unzip the .xip file, then PBZX v2 to process the Content file. Am I on the right track here? When I see the output after trying to run “./scripts/package/package_xcode_and_sdks.sh”:

./scripts/package/package_xcode_and_sdks.sh: line 66: pushd: /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/: No such file or directory

Would I be able to move the Xcode installation (assuming xar and PBZX v2 work out) into an “Application” directory that I would make through mkdir at the root, allowing the IOS packaging script to run?

P.S.

Also, getting a bit ahead of myself, but once all of the packages are done, in .tar.gz format, and the directory served on a python HTTP server, is that all that is needed to build the Docker image properly? Since I have not built the image and run the container for extender, I am also uncertain about the process of converting the source code for LÖVE into a native extension, I would love a quick overview of the steps after the container is running. Thanks for taking the time to look over this question, I really would love to make a native extension so I can keep using Defold. Please let me know if you need more context to answer the question.

#2

Unless you really want to, I would recommend just setting up the extender with the SDK for your platform and commenting out the setup for all the others in the Dockerfile (you’ll have to do that anyway unless you happen to have Switch and Playstation SDKs lying around).

Once it’s running, open up the native extension template project and put the URL of your new extender server in File -> Preferences -> Extensions -> Build Server. It’ll be blank by default, which means it’ll pick https://build.defold.com, or https://build-stage.defold.com if you’re on the beta branch.

After that, it depends on what exactly you want to do. The native extension template gives an example of how to register a new Lua function, so you can apply that to register some functions from LÖVE’s modules.

2 Likes
#3

After building it, it is a regular Docker container, and you should use docker to run it (not Python).
We have scripts to run it from bash/command prompt (see server/scripts/run-local.*)

Note that if you’re not using macOS/iOS, you don’t need those packages for the Docker container.
You can simply skip that step, as it’s not needed to build the container.

I’m curious, Is there a particular reason you’re not using the vanilla build server that we provide? (which we keep up-to-date and supports all platforms)

1 Like
#4

This Defold project is meant to be a mobile app for iOS, as well as Android in the future. So, in this case, I would need to unpack the Xcode xip in the way I mentioned above and serve it?

I also was unaware of the vanilla build server (@ build.defold.com, correct me if I’m wrong). I am running my development in Docker to sync my partners work environments and easily maintain our repo, with a Debian base image that I set up SSH for remote connection (searching about using an iOS base image, it seems to not be possible). In the Dockerfile, I install the Defold editor, bob.jar, LÖVE, and dependencies.

My perceived issue is that this dev container I made has no internet access, so I kept looking for ways to host extender in the dev container (Docker-in-Docker). The way I read the Native Extensions manual, one creates the extension source code manually (manifests, and other folders), then in the editor or through bob.jar, the build server URL is provided, which will allow the project to be bundled(?). Overall, my goal is to develop an iOS app on Windows (only one of my partners has a Mac), which would be trivial without importing modules, but I wanted to use some of the LÖVE functions.

This is what the relevant portions of my Dockerfile looks like, for reference.

...

# Install required libraries and dependencies
RUN apt-get update && apt-get install -o DPkg::Lock::Timeout=120 -y --no-install-recommends \
    ... \
    && rm -rf /var/lib/apt/lists/*

# Install Lua
RUN curl -L -O https://www.lua.org/ftp/lua-5.4.6.tar.gz && \
    tar -xzf lua-5.4.6.tar.gz && \
    cd lua-5.4.6 && \
    make linux test && \
    make install && \
    cd .. && \
    rm -rf lua-5.4.6 lua-5.4.6.tar.gz && \
    # Install LuaRocks
    curl -R -O http://luarocks.github.io/luarocks/releases/luarocks-3.11.0.tar.gz && \
    tar -xzf luarocks-3.11.0.tar.gz && \
    cd luarocks-3.11.0 && \
    ./configure && \
    make && \
    make install && \
    cd .. && \
    rm -rf luarocks-3.11.0 luarocks-3.11.0.tar.gz

# Download LÖVE source code
RUN git clone https://github.com/love2d/love && \
    cd love && \
    git checkout 11.5

# Build LÖVE
RUN cd love && \
    ./platform/unix/automagic && \
    ./configure && \
    make  

# Download the Defold zip file
RUN curl -L -o Defold-x86_64-linux.zip "https://github.com/defold/defold/releases/download/1.7.0/Defold-x86_64-linux.zip"

# Extract the Defold engine ZIP file
RUN 7z x Defold-x86_64-linux.zip -o/app && \
    chmod +x /app/Defold/Defold && \
    rm Defold-x86_64-linux.zip

# Download the bob.jar file
RUN curl -L -o /app/mobileproject/bob.jar "https://github.com/defold/defold/releases/download/1.7.0/bob.jar"

...

# Create the dev.sh script
RUN echo '#!/bin/bash' > /tmp/dev.sh && \
    echo 'set -e' >> /tmp/dev.sh && \
    echo 'echo "Starting dev.sh script..."' >> /tmp/dev.sh && \
    echo 'if [ ! -d "/app/mobileproject/loveextension/src/" ]; then' >> /tmp/dev.sh && \
    echo '    echo "Adding love src code to extension"' >> /tmp/dev.sh && \
    echo '    cp -R /app/love /app/mobileproject/loveextension/ || echo "Error adding love"' >> /tmp/dev.sh && \
    echo '    mv /app/mobileproject/loveextension/love/ /app/mobileproject/loveextension/src/' >> /tmp/dev.sh && \
    echo 'fi' >> /tmp/dev.sh && \
    echo 'echo "Running bob.jar..."' >> /tmp/dev.sh && \
    echo 'java -jar /app/mobileproject/bob.jar || echo "Error running bob.jar"' >> /tmp/dev.sh && \
    echo 'echo "Running Defold..."' >> /tmp/dev.sh && \
    echo '/usr/local/Defold/Defold || echo "Error running Defold"' >> /tmp/dev.sh && \
    echo 'echo "dev.sh script completed."' >> /tmp/dev.sh && \
    chmod +x /tmp/dev.sh && \
    chown appuser:appuser /tmp/dev.sh

...
#5

I think this is something completely different, and not really related to the extender server.

As for the extender server, we always recommend users use the built in server (already preset in the Defold editor, in the preference page under “Extensions”)
A reason for not using our vanilla server is company security policies.
I those cases, it’s relevant to build your own server.
Is this your case?

Yes, to build for macOS/iOS, you need to pack the XCode sdk + toolchain. We have the helper script, which we run on a macOS machine.
To build and run the macOS server, you won’t use Docker, but instead run it stand alone:

mkdir /usr/local/extender
./server/scripts/rebuild-and-run-standalone-local.sh /usr/local/extender

It will then provide a build server that listens at http://localhost:9010
(Read the scripts for details on how you can modify your setup)

For the other platforms, you need the Docker container.
You can comment out platforms in the Dockerfile as you desire, then build and run the container as usual, and it will listen on http://localhost:9000

But, unless you actually need to build a custom server (?), I’d recommend using our preset vanilla server (which is open source, and free to use).

1 Like