Advanced rendering techniques - any ideas?

morgerion · February 11, 2022, 8:01am

made a small feature request:

Instant go.set_enable() for model management between render passes in one frame

opened 07:56AM - 11 Feb 22 UTC

feature request

**Is your feature request related to a problem? Please describe (REQUIRED):** T…he problem concerns the fact that i need to switch different meshes on and off dynamically within the same frame. This is best illustrated by rendering a reflection map. The reflection map (pass-1) should contain the entire world around the camera, while the final screen (pass-2) contains only those meshes that are caught in the viewport. Turning off part of the surrounding world for rendering in pass-2 must happen in a single frame. A detailed description of the problem can be found here: https://forum.defold.com/t/advanced-rendering-techniques-any-ideas/70326/12 **Describe the solution you'd like (REQUIRED):** It would be ideal if the mesh had a method like gui.set_enabled(). Something like go.set_enabled(), which does the same thing as msg.post(go_url, "enable"/"disable"), but does it instantly in the current frame. **Describe alternatives you've considered (REQUIRED):** go.set() executes instantly, so i could have two materials - “unvisible” and “visible” - and set them to the right meshes immediately via go.set("#mesh", “material”, material). Unfortunately, this option resets the meshes shader constants to 'default' values.

morgerion · February 12, 2022, 12:25am

Let’s go back to the MRT for a moment.

Whatever you decide regarding Vulcan, it does not remove the need for MRT. In order not to complicate your life with Vulcan’s “sub-pass logic”, it would be logical to think of some solution close to classic MRT (which is also possible in Vulcan).

dlannan · February 12, 2022, 1:17am

I think @Mathias_Westerdahl is trying to temper your expectations of how this type of feature would be able to be implemented - it effects a number of graphics platforms. With only a team of three (I think? Is that right?) they need to prioritize tasking based on common demand of the Defold engine.

You can read their roadmap to gauge how this would fit. From my understanding, Vulkan is going to be necessary purely due to the deprecation of Metal, and support for Nintendo systems. And one of the primary advantages Vulkan will bring is cross platform compatibility for the rendering pipeline, and speedups in a number of key graphics areas (like large numbers of sprites and so on).

Im not sure how MRT fits into the “scheme” of things, but if its a large amount of work for a small user-base, I suspect it might be put on the pile of “todos” … so… if you really want it, and imho in a few months time I may be looking at it for some of the world rendering Im doing, maybe a PR is the go

Pkeod · February 12, 2022, 9:34am

Will bring? You can already use Vulkan with Defold! You just need to enable it in the app manifest. Defold App Manifest generator

dlannan · February 12, 2022, 9:48am

Oh Sorry. I thought it was fairly restricted to a couple of platforms … I had some problems on Linux with it a while ago. Will have to try again! Great stuff!!!

Oh. Just re-read, this is for native extensions only?

Mathias_Westerdahl · February 12, 2022, 10:42am

I just tried to make the point that when discussing a feature, we should make sure it fits into all our graphics backends. We currently have OpenGL and VUlkan, and we’ll soon have a new one for Playstation 4.
I’m not sure what’s required for any of those api’s, and thus I just wanted to point that out. That is all.

@dlannan Yes, we’ve supported Vulkan for some time now, and currently, the only way to shift between the backends is to use an app manifest.
This will however change in the future (fairly soon I’d say) when we will move to Vulkan (via MoltenVK) as the default backend for the Darwin platforms (macOS/iPhoneOS).

morgerion · March 23, 2022, 1:15am

Question about textures - what’s the fate of etc1 and others?
I understand that now only 32-bit/16-bit RGB/RGBA can be sent to video memory?

For example, for android, i chose the format etc1 in the texture profile. Question - will my textures go into the video memory in this format? Or in RGB32?

The size of APK build has not decreased in any way - it means that in the build the textures go in RGBA32 format. Will they be converted to etc1 during loading?

Mathias_Westerdahl · March 23, 2022, 6:50am

Our current solution includes a generic way for all our platforms to be able to support GPU textures. We did limit the library we use (Basis Universal) to currently not include the ETC1S (their name) pipeline, which would give you support for the cTFETC1_RGB format. One reason we removed that part of the library was that it also grew the exe size a lot.

Personally, I’d like us to have a more custom pipeline for our textures, so that you could compress them how you’d like, and then also decompress them how you’d like it. I think this is the way forward on this front. However, it is not currently in our backlog.

The old texture format enums are converted to the new version. E.g for etc1.
The old enums are still there in order to not completely break old projects.

And at runtime, we test the capabilities of the GPU too see which format to transcode to in the end (source)

morgerion · March 23, 2022, 7:12am

Mmm, I still don’t understand your answer.
Can I throw a packed texture into the GPU or can’t I?
If I can, what should its setting look like in the texture profile?

Mathias_Westerdahl · March 23, 2022, 7:34am

Yes, the engine does this when you select compression type COMPRESSION_TYPE_BASIS_UASTC.

It then gets compressed into the interim format. At runtime it will be transcoded into the final format supported by the GPU. It tries the formats int this order

morgerion · August 11, 2022, 11:45am

It’s amazing how much this topic has become obsolete in six months.
We already have float render-targets, MRT, and more on the way.
There are already examples of deferred rendering and cluster lighting.
That’s impressive!!!

What else is missing?
I’d like a culling for meshes and true billboards, but it’s in progress for 1.3.6

jhonny.goransson · August 11, 2022, 12:02pm

Glad to see the support for our work!
There’s always room to participate in our design process (I think our design docs are public), or perhaps even contributing to the engine. Even just coming up with features that you need for your projects will help Defold grow the 3D support, if there’s a clear use case for a feature we can work on it and I’m always happy to build features for 3D

britzl · August 11, 2022, 4:08pm

Culling of meshes might make it into 1.3.6. Billboards will be pushed to the next release. We’ve had a lot of issues with 1.3.5 that has delayed other work.

masterneme · August 12, 2022, 9:16am

+1 to billboards with both cylindrical and spherical rotation. Simple to use decals would be nice too.