DEFRAME - Defold Rendering Simplified - Idea

[Pawel]

Intoduction

I start this blog, just to keep me motivated on making a tool

I wish to built a tool/editor to simplify making stuff in Defold. The most tedious and complex for me is rendering pipeline - render script is so powerful, but so difficult to tame and understand. It’s very easy to make a mistake in a render script that is very hard to debug. Yet, if you work with it for a longer period, you open up amazing opportunities and also recognize repeatable patterns.

---

Visual Render Script Concept

The idea is to build a visual scripting tool to simplify making render scripts. I believe render script can be presented in a more interesting way and simulataneously allow to simplify making render pipelines in it.
Remember, that idea behind visual scripting isn’t being an answer to “how to write less code”, but “how to make better interface for programming”.

First sketches were drawn on paper, where I just wanted to understand available render scripts. Then I moved it computer to draw how such a tool can look like and what should be its elements.

Imagine that the whole update function of the render script is represented as a table with rows being code lines.

A very basic and most easiest render script would just draw stuff on screen with some default settings:

But more complex could look like this:

Some of the “tiles”/“bricks”/“rows” in this visual scripting tool would be 1:1 render API function calls, some would be simplified functions/wrappers, that will utilise a helper lua module, however those are stateless and preferably should not change the given data in parameters - parameters should be read only, those could then eventually return new values (kind of setters, constructors).

It should generate a whole render script based on the setup in the above visual scripting tool that represents in a form of a list - an update function of the render script. And it’s a main panel of such a tool.

Additional panels to be added are:

• A list of all available functions (bricks) to include here - a drag and drop solution to add bricks to this main view.
• A view of all the allocated/created stuff - predicates, render targets, variables, matrices, cameras, etc. This view represents what is created and how it is initialized in init of the render script. It’s a kind of a configurator of the render script. Some of the stuff should be read from “.render” file - the associated materials.
• A special view/(tab?/popup?) that represents on_message. What functions are handled and in what way. It’s similar to update, but no render API should be used here. On message callbacks should allow to change certain data or receive data from e.g. camera component, but it shouldn’t render anything. I don’t know yet how to do it precisely, most probably a lot of “Custom Lua line” bricks will be added here, because rest should be simple setters of allocated/created data/variables.

A concept view of the window with 3 of the above panes (without on_message pane):

Or it could be horizontal, more of a configurator, yet linear, so that each panel is actually convertible to lines of code in generated render script, something like:

---

Rules for the tool

1. Allocate everything upfront. Don’t allocate in update. Allocate everything upfront. Even if not all render scripts made in Defold are following this rule to not create/allocate anything new in the update(), I will let this rule be my number one. It works perfectly with second rule.
2. Only created stuff can be used in update. If everything is created in init, update function consists only of functions that can modify values or state of the renderer (through render API). When you add a brick/row that should use some data and you don’t have it yet defined, you should have a possibility in that particular place to quickly add this data and initialize it by default, that could be later changed.
3. Only correct code is generated. This might be very hard to accomplish, but it doesn’t mean it creates a correct pipeline - only the syntax is correct, usage is correct, correct data is used etc. It means that when running the code it should not assert and produce ERRORs. This will not apply to “Custom Lua code” brick that has to be added in the visual script for all the stuff that is not available (yet or never) among the pre-defined bricks.
4. Rules might change. I’m only thinking/prototyping.

---

Execution

I initially wanted and many times started creating such a tool in Defold, with barebone GUI, then with Gooey, then with Druid, but as you know - none of them are finished. I wanted a seamless integration with Defold. I think, maybe, just MAYBE, the best will be to do it with the amazing stuff that will be soon available in Defold as presented by @vlaaad:

https://twitter.com/v1aaad/status/1821850332537749642

Just posted it now to check it out with you, what you think about it, do you have some ideas/insights/tips on something like this?

Order restrictions?
For example, some things are need to be done in render script in particular order for it to work. If user just randomly put functions in it, it doesn’t mean the code will not compile if the syntax is correct and data is correct (Rule 3), but the output of such render pipeline will be not as user might think. Should I somehow restrict order of things? What are possible corner cases here? The tool imho shouldn’t be limiting, e.g. you should be able to write any rendering pipeline. But for beginners it’s hard without guidance.

Reverse generation?
Other thing is that - I could make it so that I can parse actually existing script to create its version in such tool, but if there would be any Lua modules used - I would pass. Too complex I think. What are other corner cases here? Strange Lua lines of code? E.g. for loops, ifs, etc? Not aligned with the rules above? I would most probably put them in a “Custom Lua code” bricks, if I couldn’t parse such lines. It would be on user to the either refactor it or recreate it with predefined blocks instead in that tool.

P.S. Yes, obviously name was one of the first thing that I came up with, now years ago
P.S.2. Yeah, years, the doc with those sketches (moved to PC form paper) was created in January 2021…

P.S.3. I prototyped already a strange parser, that extrudes all data from a Lua script: all named variables (even from tables or function parameters (if there are annotations) or functions themselves and it knows a lot of stuff from Defold API) and its data type (if determinable):

→

[Pawel]

@vlaaad I have a very important question : Will it be possible to add icons in such popups in Editor? If so, where the icons used currently in Editor? In 1.9.1 new icons were added for properties, this stimulates imagination Could we enhance the set of current icons with new icons eventually?

[britzl]

@Pawel

Could we enhance the set of current icons with new icons eventually?

It seems reasonable to think that custom icons will be possible to use, probably loaded from disk.

[Vow]

I think this tool will be mostly useless for most Defold users.

You mention wanting to build a tool that simplifies the render script, but I fail to see how these mockups simplify anything. The terminology and function ordering used in the tool is almost identical to what we see in the render script, which means the user would have to already understand how the render script is working in order to understand how this tool is working.

Additionally, some of the terminology here is arguably worse and more arbitrary, without any hint of what’s actually happening. For example, what is meant by “2D Basic Blend?” There are many combinations of OpenGL blending arguments to achieve very specific effects. It’s fine if some of the most common ones are dubbed something like “2D Basic Blend,” however one of your goals is to allow the tool to do anything a render script can do, which necessarily means listing out all the possible arguments. Again, this goes against your goal of simplification.

I disagree that the render script is “so difficult to tame and understand.” It’s mostly just a bit of camera logic and a series of mostly OpenGL-equivalent function calls. It’s only particularly difficult to understand for users who aren’t familiar with graphics programming, which is a problem that a tool like this isn’t going to solve. I suppose this lack of knowledge could be masked though, which would require you to go against your goal of allowing the tool to do anything the render script can do, and instead restrict the user to highly abstracted “bricks” and parameters. I would argue this goes against Defold’s core philosophy of being relatively bare bones and highly configurable, but inexperienced users might not care when all they want to do is make very simple projects. But this doesn’t even justify the tool’s existence, because if users just want to get something simple up and running, then there are a few render pipeline libraries available that already solve that problem.

You’re correct to say that there are merits to visual scripting, however I don’t see those merits here. It seems there are too many contradictory goals for this tool, namely simplifying graphics programming while still allowing 1:1 mapping to the render script. Every abstraction comes with a flexibility cost. I would say that this tool might be better suited as a much more restrictive and beginner-friendly version of the render script, but as I outlined in the above paragraph, there are already libraries available for users who aren’t comfortable writing a render script from scratch.

Perhaps what you’re really trying to achieve here is a way of educating developers about how the render script is working, in terms of OpenGL calls and the necessary linear algebra behind camera functionality? If so, then I think building a visual scripting tool is the wrong solution, as it doesn’t help with that education. Maybe explanatory tutorials or links to existing graphics programming tutorials is the right substitute.

Regardless of my criticism, I’m interested in seeing how this progresses if you decide to keep going, and I’d be glad to continue the conversation if it is helpful.

[blisteredMind]

Coming from someone self taught on how to use this engine and game coding in general. I knew nothing about graphics programming starting out and now Defold has taught me so much because of the reason mentioned above. I am still far from anything good at graphics or opengl but I can stand toe to toe with current graduates trained on Unity and Unreal drop in rendering pipelines. The fact that Defold makes no assumptions has pushed me in the direction to learn what I am doing with graphics and how better to use them in games.

What HAS helped me is seeing examples done from open source projects on Github that have pushed the abilities of Defold. I believe creating a demo project that showcases abilities, and then well documented source code of the example are much more beneficial than a visual click here script type setup. Looking at what makes Defold… Defold… it’s that there is no bloat unless you put it there. It’s lean, it’s fast. It’s gotten me into the gaming industry.

Now taking all that and pushing to create a render script that handles many lights, different quality settings, shadows and is designed to optimal on mobile devices… I feel that would be a game changer for a lot of devs coming over from Unity/etc. Something like that maintained for new versions of Defold would be absolutely amazing.

I don’t know enough, I am trying to learn as much as possible so I can, because this game engine has truly changed my life and I would invest the time to up keep it.

Either way… Everyone here in the Defold community is amazing and total legends. Never would of thought I’d be here!! Thanks to you all.

[iotaHum]

Clearly you have put a lot of effort into this. Creating a visual scripting tool for this I think is ultimately really really difficult, I not sure it will ever have legs. What about going the other way, analysing code and creating a (dot) graph of connections? This might be more tractable and result in a tool useful for debugging. Ha, all I am doing is suggesting work for someone else to do:-)

[Wolfe2x7]

@Vow

I disagree that the render script is “so difficult to tame and understand.” It’s mostly just a bit of camera logic and a series of mostly OpenGL-equivalent function calls. It’s only particularly difficult to understand for users who aren’t familiar with graphics programming, which is a problem that a tool like this isn’t going to solve.

Having learned to write shaders first, and Lua & Defold second, the render script seemed more of a red carpet to me than a hurdle. Knowing some graphics programming, it’s a friendly open invitation to do anything I can imagine.

Heck, my first Defold project was a render script I released as an asset. I support efforts to help developers understand such a powerful file.

[Pawel]

@iotaHum

What about going the other way, analysing code and creating a (dot) graph of connections

There is something like Sourcetrail - for cpp projects, but not maintained anymore, public archive:

GitHub - CoatiSoftware/Sourcetrail: Sourcetrail - free and open-source...

The idea behind it is in my opinion great, I like to visualize connections and think viusally, but code is a foundation that will preserve anyway. I just think that code dependencies/connections or data flow is better visualized with links that in your memory by constantly reading and remembering code. It’s though complex to make good of course and perhaps that’s why it’s not so successful (or there aren’t as much visual thinkers as I think )

[mikatuo]

I totally agree with @Vow. For a total noob in rendering things like “G-Buffer Pass” and “Composite Pass” make no sense. I’m thinking that some sort of “render script constructor” that allows to pick and choose from common use cases could be a great start for beginners. In example, “add second GUI layer” or “add full screen shader effect”, etc. Something that makes common use cases easily achievable, and if it would generate a render script with comments it would be a good “learn by example” tool too.

[Pawel]

@Vow

there are already libraries available for users who aren’t comfortable writing a render script from scratch.

Also on this - yes, but those are not really flexible and not easy for beginners to modify to suit their needs. So as soon as they want to achieve some different effects, they start digging in and end up learning about whole complexity of rendering instead of making a game. So this:

@mikatuo

some sort of “render script constructor”

Might be a good way to go.

Now, that we have Editor Scripts and UI, it might be way easier to make some kind of tool like this, without tilting at windmills with a whole visual scripting editor. I will definitely rethink this

[iotaHum]

What I find helpful is examples that show how to write the code rather than an asset that kind-of works but not quite in my use case, I then have to working out the asset coding before I can work out what it is doing underneath. That said the assets are really good:-). I need to find the time to put in the effort to learn how to write vp and fp code in Defold.