Dmengine: very slow loading proxy collection (SOLVED)

Editor: v1.2.106 mac
device: iPhone 7+
Dmengine:

loader.script in main.collection

function init(self)
  print("async start")
  msg.post(".", "acquire_input_focus")
  msg.post("go#map", "async_load")
end

local last_time = ""
local update_counter = 0
function update(self, dt)
    t = os.date("*t")
    update_counter = update_counter + 1
    
    time = t.hour ..":" .. t.min ..":" .. t.sec
    if (time ~= last_time) then
        print(time .. ": " .. update_counter)
        
        last_time = time
        update_counter = 0
    end
end

Editor console:

INFO:ENGINE: Defold Engine 1.2.106 (dc92d10)
INFO:ENGINE: Loading data from: http://192.168.100.12:8080/build/default
INFO:ENGINE: Initialised sound device 'default'

DEBUG:SCRIPT: async start
DEBUG:SCRIPT: 21:31:54: 1
ERROR:RESOURCE: Empty resource path
DEBUG:SCRIPT: 21:31:55: 24
DEBUG:SCRIPT: 21:31:56: 52
DEBUG:SCRIPT: 21:32:23: 43  <---- freeze 27 seconds
DEBUG:SCRIPT: 21:32:25: 40  <---- freeze 1 second
DEBUG:SCRIPT: 21:32:26: 39
DEBUG:SCRIPT: 21:32:27: 60
DEBUG:SCRIPT: 21:32:28: 58
DEBUG:SCRIPT: 21:32:29: 60
DEBUG:SCRIPT: 21:32:30: 60
DEBUG:SCRIPT: 21:32:31: 60
DEBUG:SCRIPT: 21:32:32: 60
DEBUG:SCRIPT: 21:32:33: 60
DEBUG:SCRIPT: 21:32:34: 60
DEBUG:SCRIPT: 21:32:35: 60
DEBUG:SCRIPT: 21:32:36: 60
DEBUG:SCRIPT: 21:32:37: 60
DEBUG:SCRIPT: 21:32:38: 60
DEBUG:SCRIPT: 21:32:39: 60
DEBUG:SCRIPT: 21:32:40: 60
DEBUG:SCRIPT: 21:32:41: 60
DEBUG:SCRIPT: 21:32:42: 60
DEBUG:SCRIPT: 21:32:43: 60
DEBUG:SCRIPT: 21:32:44: 58
DEBUG:SCRIPT: 21:32:45: 60
DEBUG:SCRIPT: 21:32:46: 60
DEBUG:SCRIPT: 21:32:47: 60
DEBUG:SCRIPT: 21:32:48: 60
DEBUG:SCRIPT: 21:32:49: 28
DEBUG:SCRIPT: 21:32:50: 9
DEBUG:SCRIPT: 21:32:51: 49
DEBUG:SCRIPT: 21:32:52: 60
DEBUG:SCRIPT: 21:32:53: 60
DEBUG:SCRIPT: 21:32:54: 57
DEBUG:SCRIPT: 21:32:55: 60
DEBUG:SCRIPT: 21:32:56: 60
DEBUG:SCRIPT: 21:32:57: 57
DEBUG:SCRIPT: 21:32:58: 48
DEBUG:SCRIPT: 21:32:59: 60
DEBUG:SCRIPT: 21:33:0: 60
DEBUG:SCRIPT: 21:33:1: 56
DEBUG:SCRIPT: 21:33:2: 48
DEBUG:SCRIPT: 21:33:3: 48
DEBUG:SCRIPT: 21:33:4: 60
DEBUG:SCRIPT: 21:33:5: 52
DEBUG:SCRIPT: 21:33:6: 57
DEBUG:SCRIPT: 21:33:7: 58
DEBUG:SCRIPT: 21:33:8: 53
DEBUG:SCRIPT: 21:33:9: 52
DEBUG:SCRIPT: 21:33:10: 54
DEBUG:SCRIPT: 21:33:11: 58
DEBUG:SCRIPT: 21:33:12: 56
DEBUG:SCRIPT: 21:33:13: 59
DEBUG:SCRIPT: 21:33:14: 57
DEBUG:SCRIPT: 21:33:15: 57
DEBUG:SCRIPT: 21:33:16: 60
DEBUG:SCRIPT: 21:33:17: 59
DEBUG:SCRIPT: 21:33:18: 60
DEBUG:SCRIPT: 21:33:19: 60
DEBUG:SCRIPT: 21:33:20: 59
DEBUG:SCRIPT: 21:33:21: 58
DEBUG:SCRIPT: 21:33:22: 60
DEBUG:SCRIPT: 21:33:23: 60
.... freeze again :(
  1. It takes more than 10 minutes to load collection.
  2. darc file size is ~75MB (upload to device via wifi takes 2-3 seconds)
  3. problem exists only in dmengine. IPA builded with bob.jar load collection for 2-3 seconds

When you run the empty dmengine apk or ipa it’s an empty engine without any of the content of your game. When you start the game the empty game will stream the content it needs from your workstation to the phone. This can potentially take a bit of time if there’s a lot of content. It should be faster the next time your build and run since content will be cached on the device. Does this make sense if you consider what you’ve been testing?

1 Like

I though that editor builds darc file, then upload it to dmengine. And lua code will be executed only after uploading complete.

This can potentially take a bit of time if there’s a lot of content.

10 minutes to upload 75MB via wifi? :slight_smile:

It should be faster the next time your build and run since content will be cached on the device

Build is faster, but it takes the same time to load proxy collection.

I will provide demo project on Monday.

1 Like

Nope, 10 minutes seems really crazy.

Please do and we’ll take a look!

1 Like

I clarify the problem. In our game we have about ~500 atlases (one image = one atlas). Those atlases are generated automatically. After we change generation rules and make 10 big atlases (2048x2048) - the problem is gone.

4 Likes

Wow, yeah, you should bundle images into as few atlases as possible for performance reasons. Good that you got it working!

1 Like