Defold 1.4.2 BETA

Release notes
#1

Defold 1.4.2

The latest beta is now released, and we invite those interested in beta testing the new features in the editor and engine to join now.

The beta period will be 2 weeks and the next planned stable release is two weeks from now.

Please report any engine issues in this thread or in issues using Help → Report Issue

Disclaimer

This is a BETA release, and it might have issues that could potentially be disruptive for you and your teams workflow. Use with caution. Use of source control for your projects is strongly recommended.

Access to the beta

Download the editor or bob.jar from Releases · defold/defold · GitHub

Set your build server to https://build-stage.defold.com

Defold 1.4.2

Summary

  • CHANGE: (#7234) Introduce VertexStreamDeclaration concept
  • CHANGE: (#7250) Use uniform scale for the particle fx in gui
  • NEW: (#7196) Create atlas resource in runtime
  • NEW: (#7206) resource.set_texture transcode support
  • NEW: (#7093) Added mesh frustum culling
  • NEW: (#5989) Improved on the error message when reaching the max number of possible draw calls
  • FIX: (#7171) Fix crash when max sound instances is 0
  • FIX: (#7251) Make sure to invert initial mouse y position on macOS
  • FIX: (#7257) Fix issues with missing default values when getting config values
  • FIX: (#7259) Vulkan bugfixes
  • FIX: (#6977) Track in profiler if thread was attached to JVM and wasn’t detached
  • FIX: (#7265) Use alpha propagation in GUI text nodes
  • FIX: (#7244) Conform ‘buffer is full’ error messages
  • FIX: (#7271) Cut down time Bob needed to create a build size report
  • FIX: (#7260) Get CPU usage on Android 8+ devices
  • FIX: (#7252) Do not run build hooks when attaching debugger or hot-reloading
  • FIX: (#7200) Make the search for resource links more efficient
  • FIX: (#7112) Non-editable resource variants
  • FIX: (#7254) Increase connection timeout during update
  • FIX: (#7262) Show informative error message on cyclic resource dependencies
  • FIX: (#7278) Show tile index in tilemap palette view

Engine

CHANGE: (#7234) Introduce VertexStreamDeclaration concept
Removed dmGraphics::VertexElement from dmSdk in favor of new API functions to create vertex declarations from.

NEW: (#7196) Create atlas resource in runtime
Added new function resource.create_atlas(path, args) to create an atlas resource in runtime. The resource is almost the same as an atlas generated during build time except for a few caveats. Please consult the API reference for how to use this feature.

NEW: (#7206) resource.set_texture transcode support
Added support for setting compressed and transcoded texture data in the resource.set_texture function. This can be used to decrease the runtime memory for dynamically created texture resources. The function now accepts binary data from .basis files and will pick a compressed GPU format for the client platform.

NEW: (#7093) Added mesh frustum culling
Frustum culling support for mesh component (sphere algorithm)

NEW: (#5989) Improved on the error message when reaching the max number of possible draw calls

FIX: (#7244) Conform ‘buffer is full’ error messages
When a component cannot be created we show an error message on where in the project that can be modified, but previously the messages had different wording. This change tries to conform most of these cases into the same format.

FIX: (#7171) Fix crash when max sound instances is 0
Fixed a crash when max sound instances is 0 in game.project

FIX: (#7251) Make sure to invert initial mouse y position on macOS
The initial mouse y position on macOS uses inverted y coordinates with 0 starting at the top of the screen. This fix inverts the y position of the initial mouse position.

FIX: (#7257) Fix issues with missing default values when getting config values
The recently added functionality to get config values as strings, ints and numbers takes an optional second argument as the default value if the config value does not exist. In the case of getting an int or a number the code incorrectly required a default value or nil, but not the complete lack of a second argument. In addition to this the API reference also states that in the absence of a default value that nil should be returned when in fact a 0 is returned. Both of these issues are now fixed and it is now possible to call the function without a second argument, in which case 0 is returned if the config value doesn’t exist.

Assuming a game.project with the following:

[test]
foo = 123

print(sys.get_config_int("test.foo")) --123
print(sys.get_config_int("test.bar")) -- 0
print(sys.get_config_int("test.bar", 345)) -- 345
print(sys.get_config_number("test.foo")) --123
print(sys.get_config_number("test.bar")) -- 0
print(sys.get_config_number("test.bar", 345.5)) -- 345.5

FIX: (#7259) Vulkan bugfixes
Fixed two issues with the Vulkan renderer:

  • Set binding base 1 for shader bindings (uniforms & samplers) - this means that we can detect unused bindings and still produce valid spir-v which previously did not work
  • Clearing render targets (also applies to backbuffer) always cleared color buffers as well, the clear flags should now be respected

FIX: (#6977) Track in profiler if thread was attached to JVM and wasn’t detached
It detects only cases when thread attached and wasn’t detached between frames.

FIX: (#7260) Get CPU usage on Android 8+ devices
Alternative CPU usage algorithm for android 8+. Fixes profiler.get_cpu_usage report as 0 on late android versions.

Editor

CHANGE: (#7250) Use uniform scale for the particle fx in gui
The editor preview of particle effects in gui scenes did not show an accurate representation of the particle effect at runtime. After some investigation it turns out that it was in fact the runtime which was inaccurate and picked the x component of the scale at all times instead of the minimum scale value from a non-uniform scale. This means that a particle effect with scale [5.0 1.0 1.0] used scale 5.0 while a pfx with scale [1.0 5.0 1.0] used scale 1.0. The correct scale should be 1.0 in both cases since the smallest scale value should always be chosen.

This is likely a regression from a change introduced in #2540 back in 2018. This is now fixed and particle effects both on game objects and in gui scenes will use the uniform minimum scale value in both the editor and engine.

FIX: (#7265) Use alpha propagation in GUI text nodes
Bugfix for GUI text nodes in editor when using inherit-alpha.

FIX: (#7252) Do not run build hooks when attaching debugger or hot-reloading
The pre- and post-build hooks were incorrectly run also when attaching the debugger and hot-reloading resources. This fix only runs the pre- and post-build hooks when building the project using the “Project->Build” and “Debug->Start with debugger” options.

FIX: (#7200) Make the search for resource links more efficient
On big projects with a lot of resources, it might take a lot of time to show output in the console because for every potential resource link we have to filter through all tracked resources. We now try to be smarter about it, both by limiting the candidates for search only to ellipsis-prefixed ones and by building a suffix map for the added lines batch so we have to go through all the resources less frequently.

FIX: (#7112) Non-editable resource variants
Added a Non-editable Directories field to Shared Editor Settings. After a restart, resources below these directories will be loaded in a non-editable mode that will be more efficient to load and manage, at the cost of not being editable. In addition, these resources will not be overwritten when saving in the editor.

Known limitations

  • It is not possible to override properties on objects inside non-editable collections or game objects from an editable collection.

FIX: (#7254) Increase connection timeout during update
The editor update sometimes fails due to connection timeouts when the update is downloaded on a slow or low quality network. This change increases the connection timeout to 5000ms and the read timeout to 10000ms.

FIX: (#7262) Show informative error message on cyclic resource dependencies
Cyclic resource dependencies in the editor did not show a user friendly error message. The message did in fact seem to indicate an error with the editor rather than with the project structure. This change detect cyclic dependencies and shows a popup with an improved error message to clearly indicate the problem to the developer.

23 Likes
#2
#3

Brilliant release!
I’m curious what is the use case scenario for this?

1 Like
#4
#5
#6

Nice! Was expecting this one for quite a while.

3 Likes
#7

Thank you!

2 Likes
#8

It’s for cases where a lot of content generated outside of the editor without needs to be able to edit it in the editor.

We see more and more tools that can export content into Defold format, for example Tiled, Blender scripts or TrenchBroom.

If after export you have ready-for-use content you don’t want to edit in the editor it will be efficient to mark such content as read-only. That will make the editor much more responsive especially in cases when you have a ton of content.

If you have more questions feel free to ask and tag @mats.gisselson , he knows more about it and can explain it better.

5 Likes
#9

We will add documentation about this soon!

5 Likes
#10

I have no idea why. But defos not build in 1.4.2 beta.
But it worked with 1.4.1. Any ideas?

line 6:#include <atlbase.h>
/native/defos/src/defos_win.cpp
	Line 6: In file included from upload/native/defos/src/defos_win.cpp:6:
use of undeclared identifier 'DispatchMessage'
                        DispatchMessage(&msg);
1 Like
#11

How to worked with mesh culling?
On my project in doesn’t worked:(

Same mat_frustum worked for sprites.

local mat_view = render3d.camera_view()
render.set_view(mat_view)
local mat_proj = self:get_perspective()
render.set_projection(mat_proj)
local mat_frustum = mat_proj * mat_view

--draw meshes
self.draw_opts.frustum = mat_frustum
render.draw(self.predicates.model, self.draw_opts) 
self.draw_opts.frustum = nil
#12

I got the same error for the clipboard extension on 1.4.2 BETA and Windows 10.

1 Like
#13

Sorry, the description wasn’t very informative. You need to manually set a bounding volume on the buffer I think. I’ll write up documentation and an example tomorrow.

#14

Thanks, wait for example to make tests in game:)
Ohh. So if i have a 3d level with a lot of meshes, it don’t worked out of the box?

2023-01-16_22-38-28
2023-01-16_22-38-281350×932 290 KB

#15

Correct. Given the dynamic nature of the buffers, we opted for letting the users set the bounding box manually since they already have that data.

1 Like
#16

This is strange. I don’t think we have vhanged that api. Perhaps we’ve accidentally changed some includes. We’ll have a look in the morning!

Just quickly looking at the includes, you could try moving the windows includes to the top, or before the ”dmsdk/sdk.h” in particular

1 Like
#17

I’ve now submitted a fix to the dmsdk/sdk.h where I removed thread.h (which I had added). A new beta build will show up within the hour.

3 Likes
#18

Some more clarifications on where the AABB+meshes feature is currently at:

With the latest addition of the frustum culling of meshes, it allows us to set meta data to a buffer at runtime. This is done with either Lua: buffer.set_metadata(), or C++ dmBuffer::SetMetaData(). There are also getters.

The mesh component looks for a AABB meta data with 6 floats: (minx, miny, minz), (maxx, maxy, maxz)
An example in Lua:

buffer.set_metadata(buf, hash("AABB"), {0, 0, 0, 1, 1, 1}, buffer.VALUE_TYPE_FLOAT32)

Reasoning:

Since the meshes are mainly for dynamic use at runtime, we first focused on the runtime part of the meta data + culling. We recommend when updating the buffer with new vertices, to also update the bounding box accordingly.

Since the buffer format is very generic (could be a sound or anything you want it to be), we cannot reliably calculate a bounding box for you. Especially if you consider that the vertex data may be packed in any way. E.g. in a terrain patch, the data may very well be represented as

uint8_t x, z;
uint16_t y;

where the x/z are in a different space from y.

Further improvements can be made, such as storing meta data in the buffer resource. However, it might also be pointed out that the model component will have similar functionality if not more, with multi materials. We’ll also add frustum culling soon for models. And we’re also looking to implement instancing for models.

But, for now, if you are using static pre-created mesh buffers, you need to iterate over your vertices and calculate the AABB manually.

7 Likes
#19

Try to worked with mesh. But it crashed:(
Sometimes i get buffer invalid. Sometimes i get crash.

Can’t dmBuffer::ValidateBuffer or dmBuffer::GetStream
In logs buffer look ok.
Any ideas?

DEBUG:SCRIPT: hash: [/scene_hub_4/meshes/office_6.004.bufferc]
DEBUG:SCRIPT: buffer.buffer(count = 2454, version = 0, { hash("position"), 

buffer.VALUE_TYPE_FLOAT32, 3 }, { hash("normal"), buffer.VALUE_TYPE_FLOAT32, 3 }, { hash("texcoord0"), buffer.VALUE_TYPE_FLOAT32, 2 })
INFO:CRASH: Successfully wrote Crashdump to file: C:\Users\user\AppData\Roaming\Defold/_crash
ERROR:CRASH: CALL STACK:

ERROR:CRASH:  0 0x7FF79504CFE0 dmCrash::GenerateCallstack D:\a\defold\defold\engine\crash\src\backtrace_win32.cpp:144

ERROR:CRASH:  1 0x7FF79538235C _seh_filter_exe /tmp/job3421627433228567764/minkernel/crts/ucrt/src/appcrt/misc/exception_filter.cpp:219

ERROR:CRASH:  2 0x7FF7953CFF98 `__scrt_common_main_seh'::`1'::filt$0 d:\agent\_work\4\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:304

ERROR:CRASH:  3 0x7FF795346840 __C_specific_handler d:\agent\_work\4\s\src\vctools\crt\vcruntime\src\eh\riscchandler.cpp:290

ERROR:CRASH:  4 0x7FFB21792300 __chkstk <unknown>:0

ERROR:CRASH:  5 0x7FFB21741070 RtlRaiseException <unknown>:0

ERROR:CRASH:  6 0x7FFB21790F20 KiUserExceptionDispatcher <unknown>:0

ERROR:CRASH:  7 0x7FF7950C3E10 dmBuffer::ValidateBuffer D:\a\defold\defold\engine\dlib\src\dlib\buffer.cpp:324

ERROR:CRASH: Lua Callstack:

ERROR:CRASH:   [C]:-1: in function mesh_set_aabb

ERROR:CRASH:   world/game/levels/level_creator.lua:389: in function init_meshes

ERROR:CRASH:   world/game/levels/level_creator.lua:258: in function <world/game/levels/level_creator.lua:150>

ERROR:CRASH: 

INFO:CRASH: Successfully wrote Crashdump to file: C:\Users\user\AppData\Roaming\Defold/_crash
ERROR:CRASH: CALL STACK:

ERROR:CRASH:  0 0x7FF7950C3E10 dmBuffer::ValidateBuffer D:\a\defold\defold\engine\dlib\src\dlib\buffer.cpp:324

ERROR:CRASH:  1 0x7FF794FC2110 MeshSetAABB /tmp/job3421627433228567764/upload/native/game/src/extension.cpp:305

ERROR:CRASH:  2 0x7FF794FD429E lj_BC_FUNCC <unknown>:0

ERROR:CRASH:  3 0x7FF794FD5053 lj_ff_coroutine_resume <unknown>:0

ERROR:CRASH:  4 0x7FF7950445C0 lua_pcall <unknown>:0

ERROR:CRASH:  5 0x7FF795076170 dmScript::PCallInternal D:\a\defold\defold\engine\script\src\script.cpp:1386

ERROR:CRASH:  6 0x7FF7950C2810 dmGameObject::RunScript D:\a\defold\defold\engine\gameobject\src\gameobject\comp_script.cpp:138

ERROR:CRASH:  7 0x7FF7950C2A00 dmGameObject::CompScriptUpdateInternal D:\a\defold\defold\engine\gameobject\src\gameobject\comp_script.cpp:236

ERROR:CRASH:  8 0x7FF79509FA60 dmGameObject::Update D:\a\defold\defold\engine\gameobject\src\gameobject\gameobject.cpp:2632

ERROR:CRASH:  9 0x7FF79515C2E0 dmGameSystem::CompCollectionProxyUpdate D:\a\defold\defold\engine\gamesys\src\gamesys\components\comp_collection_proxy.cpp:299

ERROR:CRASH: 10 0x7FF79509FA60 dmGameObject::Update D:\a\defold\defold\engine\gameobject\src\gameobject\gameobject.cpp:2572

ERROR:CRASH: 11 0x7FF795114540 dmEngine::StepFrame D:\a\defold\defold\engine\engine\src\engine.cpp:1570

ERROR:CRASH: 12 0x7FF79510F770 dmEngineUpdate D:\a\defold\defold\engine\engine\src\engine.cpp:2098

ERROR:CRASH: 13 0x7FF795114F60 dmEngine::RunLoop D:\a\defold\defold\engine\engine\src\engine_loop.cpp:83

ERROR:CRASH: 14 0x7FF79503AEB0 engine_main D:\a\defold\defold\engine\engine\src\engine_main.cpp:148

ERROR:CRASH: 15 0x7FF7953463F4 __scrt_common_main_seh d:\agent\_work\4\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288

ERROR:CRASH: 16 0x7FFB210F7600 BaseThreadInitThunk <unknown>:0

ERROR:CRASH: 17 0x7FFB21742680 RtlUserThreadStart <unknown>:0

ERROR:CRASH: Lua Callstack:

ERROR:CRASH:   [C]:-1: in function mesh_set_aabb

ERROR:CRASH:   world/game/levels/level_creator.lua:389: in function init_meshes

ERROR:CRASH:   world/game/levels/level_creator.lua:258: in function <world/game/levels/level_creator.lua:150>

ERROR:CRASH: 

INFO:CRASH: Successfully wrote MiniDump to file: C:\Users\user\AppData\Roaming\Defold/_crash.dmp

local res = go.get(mesh_id, "vertices")
local buffer = resource.get_buffer(res)
pprint(res)
pprint(buffer)
game.mesh_set_aabb(buffer)

static int MeshSetAABB(lua_State *L) {
    DM_LUA_STACK_CHECK(L, 0);
    d954masGameUtils::check_arg_count(L, 1);
    dmBuffer::HBuffer mesh = dmScript::CheckBuffer(L, 1)->m_Buffer;
    dmBuffer::Result validate = dmBuffer::ValidateBuffer(mesh);
    if(validate!=dmBuffer::RESULT_OK ){
        luaL_error(L,"buffer invalid");
    }

    float* positions = 0x0;
    uint32_t size = 0;
    uint32_t components = 0;
    uint32_t stride = 0;
    uint32_t count = 0;
    dmBuffer::Result r = dmBuffer::GetStream(mesh, dmHashString64("position"), (void**)&positions, &count, &components, &stride);


    if (r == dmBuffer::RESULT_OK) {
        //must have at least 1 point
        if(size>0 && false){
            float aabb[6];
            //min
            aabb[0] = positions[0];
            aabb[1] = positions[1];
            aabb[2] = positions[2];
            //max
            aabb[3] = positions[0];
            aabb[4] = positions[1];
            aabb[5] = positions[2];
            positions += stride;
            for (int i = 1; i < count; ++i){
                float x = positions[0];
                float y = positions[1];
                float z = positions[2];
                if(x<aabb[0]) aabb[0] = x;
                if(y<aabb[1]) aabb[1] = y;
                if(z<aabb[2]) aabb[2] = z;

                if(x>aabb[3]) aabb[3] = x;
                if(y>aabb[4]) aabb[4] = y;
                if(z>aabb[5]) aabb[5] = z;
                positions += stride;
            }
            dmBuffer::Result metaDataResult = dmBuffer::SetMetaData(mesh, dmHashString64("AABB"), &aabb, 6, dmBuffer::VALUE_TYPE_FLOAT32);
            if (metaDataResult != dmBuffer::RESULT_OK) {
                luaL_error(L,"dmBuffer can't set AABB metadata");
            }
        }
    } else {
        luaL_error(L,"dmBuffer can't get position.Error:%d",r);
    }
    return 0;
}
#20

If you get the buffer from a resource you can’t use the m_Buffer pointer as-is, you need to get it from the m_BufferRes. @JCash maybe we need to expose the UnpackLuaBuffer function to the dmsdk?

3 Likes