Pre-processing project files in lifecycle hook editor script

Hello! I’m trying to augment build pipeline with some project files pre-processing by third-party tool. Looks like the best place for this is lifecycle hook script. The problem is that I can’t get the paths of files in project directory since the hook script is being run in /opt/Defold directory, where my Defold installation resides; this is confirmed by the following simple hook script:

local M = {}
function M.on_build_started(opts)
    local r = io.popen("pwd")
    error(r:read("*a"))
end
return M

However, I’d need a full paths to (some) files in project in order to feed those to that third-party tool. How can I acheive that? I’ve looked at opts argument and even at editor built-in, but none of those contain the project path to build the file paths upon it.

I haven’t used our editor script extension system enough to give good support I’m afraid. I know @Insality and @Jerakin are frequent users though. Perhaps they can help?

1 Like

Not sure I am much help here either unfortunately. I have tried to get the “absolute” project path in the past but never succeeded, however I have found in most cases I don’t need to.

io.popen("pwd") reports something “unexpected” for me too however normally if I would get a file with

-- usually this is a resource not a file already, like your selection
local root = editor.get("/game.project", "path") 

local file_handle = io.open("." .. root, "r")

While the editor.get does return a relative path io.open is fine with that and will read/write to the expected location with it. So lua does have the project path internally, not 100% sure how to get that though.

In the past I have passed these paths a long to python and it worked fine doing that too.

Probably, for your help I can share example of editor script from Druid:

It’s works fine with files inside project root where editor script is started. As I remember, the relative paths is working.

I think hooks.editor_script should work with similar way. For example in my hook script I have the string like:

local file = io.open(".check_file", "r")

And it open file correctly from project root folder

1 Like

Thanks for helpful pointers guys! The io.open indeed does work for the files in project. The only question left, is there a way to programmatically enumerate all the files in the project in the lifecycle hook, possibly matching an extension, so I can feed those I need to that third-party preprocessor tool?

I don’t think that possible with a strict .editor_script. What I would probably do something that I did in the link I posted above (this commit but the removed code), I would get the path to game.project within the editor script then send that into python (as the link) then gather your files there and pass it along. Not pretty but would get the job done.

Got it, thanks!
I ended up using the regular editor script, since it kind of feets my needs a little bit better anyway. Thanks for the tips!

Hi, I want to share my solution since I am currently developing a python script to convert LDTK project to Defold, and I use an editor_script to call the python script from inside the editor.

I am on Defold v1.11.1 and you can use this function https://defold.com/ref/editor-lua/#editor.external_file_attributes:path to get path of files. This function resolves relative path with respect to the project root, so I have 2 functions:

-- get an absolute filesystem path from a path relative to defold project
local function get_absolute_path(local_path)
    if string.sub(local_path, 1,1)=='/' then 
        local_path = string.sub(local_path,2)
    end
    return editor.external_file_attributes(local_path).path
end

-- get absolute filesystem path of the defold project root
local function get_project_root()
    return editor.external_file_attributes('.').path
end

Then in the editor script I can call python like that (I am passing the absolute ldtk file path and the full defold project root path):

  run = function(opts)
                local ldtk_file = editor.get(opts.selection, "path")
                local ldtk_path = get_absolute_path(ldtk_file)
                local project_root = get_project_root()
                                
               --  local config_file = editor.prefs.get("ldtkpy.config_file")  -- TODO: handle config file 
                            -- TODO: allow config to python command
                local python_bin = 'python' 
                local python_script = project_root .. '/ldtk_py_defold/ldtk_py_defold.py'
                pprint('Run python:')
                pprint(python_script, ldtk_path, project_root)
                editor.execute(python_bin, python_script, ldtk_path, project_root)
                pprint('conversion OK')

                return {}
            end

I hope this will help

2 Likes

Hi,
I have to share again that my solution above works for python scripts… but only if used inside the project with the source code, not if you use it as a dependency.

I was willing to keep my python source code besides my editor script, so that the editor script version just execute the source and the versions are always in synch, without relying on a pip install.

But I want to use this script as a library in other projects → that’s when thing doesnt work anymore.

As I understand it, when you import a library, the editor just download the zip file of the library, but never unpack it as true files inside your project. So when you call python ./myLib/path/to/script.py, the python executable doesnt find any files to execute!

A work around would be

  1. to copy, effectively, the lib folder inside the project. But then this folder is shadowing the lib folder and you cannot load a new version anymore. More over you need to commit theses files into you project.
  2. to package my python code and push it to pip, then tell people to pip install, or do some pip install from inside the editor_script. Then I could call a module directly, not a python file.

Is there other solutions? Can we tell python to look inside the zip file? Would imported python modules work?

The editor can unpack the files for you:

Mmmh, are you talking about this part of the doc??

“”"
Also note that although dependencies are shown in Assets view, they do not exist as files (they are entries in a zip archive). It’s possible to make the editor extract some files from the dependencies into build/plugins/ folder. To do it, you need to create ext.manifest file in your library folder, and then create plugins/bin/${platform} folder in the same folder where the ext.manifest file is located. Files in that folder will be automatically extracted to /build/plugins/${extension-path}/plugins/bin/${platform} folder, so your editor scripts can reference them.
“”"

I am sorry but the doc is really really scarce and not understandable… what’s in an ext.manifest file?
What’s that ${platform} variable, and ${extension-path} variable?

Could you provide just an example please?, I can figure it out from that (and maybe update the doc, can we as a user contribute to it? I like to fix docs right when I am not satisfied with it :hugs: )

The ext.manifest file is a configuration/definition file for an extension. It is mostly used when an extension contains native code to provide additional compiler flags etc but it is also a general identifier for our tools that a folder should be treated as an extension.

When an ext.manifest file is encountered the editor will also unpack any zip files in the plugins folder. This can be both platform specific files arm64-macos.zip and common files common.zip. The files will be unpacked to your build output folder where ${extension-path} will be replaced by the folder name of the extension and ${platform} by one of the predefined supported platform identifiers (arm64-macos, wasm-web etc).

An example can be seen in defold-pbr: