Building a Sync Tool for Blender to Defold

anon95708182 · April 12, 2022, 8:16am

OK, will post text next time and google a bit of how to find the library (still not very familiar with macos)

astrochili · April 12, 2022, 9:25am

Yes, /Users/[username]/Library/Application Support/Bforartists/3.2/scripts/addons/sync_tool/textures exists and is empty.
Yes, I can paste any files inside without a password.

One thing I noticed that the path in logs is Pixel/tile_16.png but my local path to the images is textures/Pixel/tile_16.png. Is it legal?

Tried to run sync_tool/defoldsync/luajit/darwin/luajit and everything ok, it runs because previously I run chmod +x with this file by the instruction. Otherwise there will be error about permissions and it’s normal. By the way, my problem with img.save() runtime error is the same, regardless of the executability of luajit.
Tried to run {sync_tool path}/defoldsync/luajit/darwin/luajit {sync_tool path}/defoldsync/main.lua {sync_tool path} and got this output. Stack traceback is the same with @anon95708182 but the error is different.

$ /Users/astro/Library/Application\ Support/Bforartists/3.2/scripts/addons/sync_tool/defoldsync/luajit/darwin/luajit /Users/astro/Library/Application\ Support/Bforartists/3.2/scripts/addons/sync_tool/defoldsync/main.lua /Users/astro/Library/Application\ Support/Bforartists/3.2/scripts/addons/sync_tool
Lua generation started...
/Users/astro/Library/Application Support/Bforartists/3.2/scripts/addons/sync_tool/defoldsync/luajit/darwin/luajit: ...rartists/3.2/scripts/addons/sync_tool/defoldsync/png.lua:13: dlopen(libpng.dylib, 5): image not found
stack traceback:
	[C]: in function 'load'
	...rartists/3.2/scripts/addons/sync_tool/defoldsync/png.lua:13: in main chunk
	[C]: in function 'require'
	...cripts/addons/sync_tool/defoldsync/material-textures.lua:41: in main chunk
	[C]: in function 'require'
	...ts/3.2/scripts/addons/sync_tool/defoldsync/generator.lua:28: in main chunk
	[C]: in function 'require'
	...artists/3.2/scripts/addons/sync_tool/defoldsync/main.lua:32: in main chunk
	[C]: ?

Hi @anon95708182,
I’m not sure that I understand what images I need to remove. Original png files, textures in Blender or something else? Sorry I’m not so good in Blender.

You can find the location of the brew package and check the architecture by following example. This is my output:

$ brew --prefix libpng
/opt/homebrew/opt/libpng

$ file /opt/homebrew/opt/libpng/lib/libpng.dylib
/opt/homebrew/opt/libpng/lib/libpng.dylib: Mach-O 64-bit dynamically linked shared library arm64

$ file /Users/astro/Library/Application\ Support/Bforartists/3.2/scripts/addons/sync_tool/defoldsync/luajit/darwin/luajit
/Users/astro/Library/Application Support/Bforartists/3.2/scripts/addons/sync_tool/defoldsync/luajit/darwin/luajit: Mach-O universal binary with 2 architectures: [i386:Mach-O executable i386] [x86_64:Mach-O 64-bit executable x86_64]
/Users/astro/Library/Application Support/Bforartists/3.2/scripts/addons/sync_tool/defoldsync/luajit/darwin/luajit (for architecture i386):	Mach-O executable i386
/Users/astro/Library/Application Support/Bforartists/3.2/scripts/addons/sync_tool/defoldsync/luajit/darwin/luajit (for architecture x86_64):	Mach-O 64-bit executable x86_64

I know that homebrew has changed the path for storing packages to /opt/homebrew for M1. This information may be helpful.

dlannan · April 12, 2022, 9:27am

@Mathias_Westerdahl Thanks for helping out. Have been having dinner
Yes, @anon95708182 looks like you have a libpng problem, which is the brew install related stuff. I’m going to try to remove that dependency tomorrow and have it included in the folder (like Win10 has). This shouldnt be too hard to sort out

@astrochili however, thats looking like the same sort of issue. I’ll post a release tomorrow hopefully addressing this. Thanks for everyones help and debug info, this is very useful and allows me to diagnose things very quickly. I will add it to a log generator tomorrow so you can just send me a log

Again, appreciate everyones time. I might get to some of this tonight hopefully, so it might only be a few hours away.

anon95708182 · April 12, 2022, 9:37am

To see what would happen without images, I removed the image textures from the materials in Blender and switched off “use nodes”.
Thank you for showing me how to find the package and check the architecture.
I get the same output as you do.

astrochili · April 12, 2022, 9:45am

@anon95708182 Thanks, I removed images from textures and switched off nodes. The result is the same as you have.

dlannan · April 12, 2022, 11:20am

I’ll do a little explanation as to whats happening so everyone understand why this is happening

When you sync to build the folder of data for the project, files need to be generated. Originally I did some of this in python, but it was a pain to maintain and debug. I already had a generator written in my gltf loader project in lua and it was quite stable, so I decided to go with this method - there are a couple of other nice advantages of this too which I wont detail.

So the process looks like:
Sync Tool button pressed → Convert object tree into a useful json data set (in temp folder) → Convert meshes to a json in temp → Convert anims to a json + dae data set.
This is where the python blender script ends.
Then, the lua script does the json → go files, mesh files, model files, textures and collection files.

When fetching the textures during the python stage, I force the textures to be written out to the temp file (using blender’s tools). This also forces them to output as png. Since this is just easier to manage for the later step.

When processing the data in the lua scripts, I need the textures to be in a specific format to minimise their foot print (this will happen more) when using PBR materials. The albedo, metallic and roughness textures are merged from three textures into one. This uses this libpng library to do so.

While libpng is not really necessary, it is convenient . Normally when I use these sorts of tools I make sure I have a dynamic library in the same folder as the luajit executable (so the search paths dont need modifying, and so you dont need to install libpng), however the libpng I had for OSX, was very old and not working. Hence the annoying homebrew process that I added to the readme this week.

I have my own very simple lib png library (I only need the loading, and direct access to the binary data) which I may use, or I will just include a prebuilt libpng. This is what Im looking at right now.

I hope this explains what has happened, and how its quite easily fixed but I want to make sure the fix is suitable for multiple OSX versions and is relatively consistent with the other OS’s I have support for.
Watch this space. I hope this is all resolved soon

astrochili · April 12, 2022, 5:36pm

Thank for the explonation, waiting for the update to test it )

dlannan · April 13, 2022, 1:41am

Ok. Good progress. It turns out the image paths are also a problem. On the python side, Blender is trying to save the texture with a path (which it shouldnt be doing). I need to strip the paths to make sure this doesnt happen.

The other issue is being solved by having a local libpng. This should support everything needed.
Should have a release in around an hour. All things going well

dlannan · April 13, 2022, 3:08am

Ok. I have a working solution on OSX. Its much simplified (no more homebrew).
Im testing the other platforms at the moment. I hope to have this release up very soon.

dlannan · April 13, 2022, 9:23am

@anon95708182 and @astrochili a new release us up that should solve the issues above.

Ive been testing on win10 and Linux as well, and all look good now. If you find any problems please let me know. I didnt get the logging in due to life getting in the way today I hope to get that in tonight sometime so people can report more easily.

anon95708182 · April 13, 2022, 2:28pm

@dlannan - you did it!!!
Look:
Blender scene:

Defold:

The image rotation didn’t come out right (normals ok and transformation is applied in Blender), but that’s not a prob.
Well done you and thank you very, very much for this tool!

Addendum:
all good with the rotation, my fault, I forgot to flip a normal.

dlannan · April 13, 2022, 10:33am

Fabulous!!! The scene rotation is something I dont like (would prefer complete transforms) its on my todo list

Thanks again for the help. I hope its useful enough. I’ll raise the rotation as an issue so I get to it sometime soon

anon95708182 · April 13, 2022, 10:39am

Give me a sec, maybe it’s a mistake I made, I’ll try with another mapping type (the dachshunds are imported with the right rotation) …
Nope, I sent you the project if you would like to have a look

dlannan · April 13, 2022, 10:47am

No this isnt an error on your part. There is a root rotation to get the Blender scene “close” to a Defold scene, rather than transform all the verts, objects, normals, animations etc - which can get a little messy.

So this is kind of expected. You can cheat by swapping the UV’s But I dont like that idea. I do want to transform everything properly, but as noted above, there are quite a few complications

I will possibly get a look at it over Easter. Will see. So, to be clear, this is a “known” issue (there are a few) and it is important to get it working better than it is atm. One thing to check though is to make sure you have applied all transforms before using the sync tool. Un applied transforms are not used and this could also cause this problem.

I’ll have a look at your scene and let you know.

dlannan · April 13, 2022, 11:01am

Ok. Ignore all my drivel above. It seems you have a different issue. The object within the collection has the wrong rotation - this is weird. Looking into that now, I have had all sorts of objects with all sorts of rotations on them, and I havent seen this. Will let you know what I find.

anon95708182 · April 13, 2022, 11:06am

I am sooo sorry, the image came in with rotation at 0, but was facing backwards and so there was nothing to see - so it was I who rotated it in Defold.

dlannan · April 13, 2022, 11:09am

Oh no dont be sorry. It should export exactly what you see in blender space. The problem is getting Blender XYZ to match Defold XYZ - you want it to be consistent.

As you mentioned, the normals might be the cause of this. You see I dont transform the normals, and so it might be the fact that flipping the normals, on a plane wont be correct by just rotating the root node to get into the same space (I think).

Im doing some tests now. Will let you know.

anon95708182 · April 13, 2022, 11:30am

@dlannan, you can stop your test. I am an idiot and checked the normal once more - and it was flipped. So sorry for the false alarm!
Btw, I just exported a massive scene with about 200 meshes and it took only some seconds. Wonderful!

dlannan · April 13, 2022, 11:38am

Hehe. I have been writing up the same thing. Was adding some extra info and screenies

When you need to run the app, you can use the camera in the scene as your camera source.
Also, in your main collection file, I usually use a Collection file - and then assign the sync tool collection you want to test/view into the scene. Its a little like referencing the asset. I recommend this, because then you can have one “project” collection with scripts etc, and you just add collection files as needed. There are some gotchas to this method, but its good for testing.

The github project includes a sample project and an orbit camera script to easily look at and test your generated assets.

Code to use:

github.com

dlannan/defold-blender-export/blob/600d5448d0df732cdc390a558ffc6248e5ea8344/defold/main/main.script#L4-L6

    
      
          	msg.post(".", "acquire_input_focus")
          	msg.post("@render:", "use_camera_projection")
          	msg.post("/main#camera", "acquire_camera_focus")

I often use:
msg.post("/Collection/Camera#camera", "acquire_camera_focus")

dlannan · April 13, 2022, 11:40am

The main thing to remember when you open the sync tool collection file, the camera will by default be looking down the -Z axis. So its a good idea to do the same thing in Blender (with a render mode on like cycles or evee) and see from that position to check it is the same - it should be similar (camera properties might be different).

Oh yea. Watch out for normals This did bite me badly on the f18 animated landing gear, so many inside out polys. It can mess with your mind