How textures manager works? (SOLVED)

Questions
#1

I try to understand how to unload texture.
How references are counted?
For example, I try to create collection that have GO with GUI component. GUI component has one atlas 2048x2048.

After start the app I’m waiting 5 seconds, then delete GO with GUI component. But memory usage didn’t change.

Then I thought that this is classic GC and it will free memory when something want to use this memory. I try to create few collection factory (every collection have it’s atlas). Memory use is increase, but first atlas did not removed.

I have next questions:

  1. Where is the counter of atlas? It depend on Component, GO or Collection?
  2. Is it possible to unload texture without unloading collection? (by removing GO, or some other way)

Would be great to have more information about texture management.

#2

Looks like my example works good on device (texture unloaded) and didn’t work by cmd+b (why it doesn’t work on mac test build?).

But if somebody who made textures management give more info (something like great post about draw calls for example) - it would be great for understand how it works.

UPD:
I Just tried the same thing in the main project and texture didn’t unloaded (it is preloader texture, and I try to remove GO with preloader GUI after main gameplay proxy was loaded)
I really don’t understand how it works.

#3

I have two collection:
loader.collection with GUI.go (contain gui component with loader.atlas)
game.collection with many game objects, but noone use loader.atlas

Two situations (test on android).

Situation 1.

  1. Load main loader.collection
  2. Wait 5 seconds
  3. Remove GUI.go
    Result: atlas removed from the memory

Situation 2.

  1. Load main loader.collection
  2. Load collection proxy
  3. waiting when collection proxy loaded
  4. Remove GUI.go
    Result: atlas didn’t remove.

What I’m doing wrong?

UPD: One more question: is it possible to receive list of atlases in memory?

#4

Hmm, this is a bit surprising. I would have guessed that all atlases are loaded into memory when the main collection is loaded (ie when the app starts).

What happens when you unload the collection? I would guess that the atlas is removed from memory.

Yes, use GAPID (Graphics API Debugger).

1 Like
#5

Ok. That mean that atlas linked with collection, not with object or gui component?
and what about this thread, how it works if atlas linked to collection?

It is hard to check. I’ll try it when understand how to check loaded textures.

Thank you, I’ll try. (but as I understand I need to have a root on the device)

#6

On android it should be enough to set android:debuggable="true" in the manifest file.

2 Likes
#7

Is it possible to use something like this on ios? (xcode tools?)

#8

How are you checking what resources are loaded and how much memory is used by the application? Through the built-in profiler or the web profiler?

In Situation 2, something certainly seems to be referencing the atlas. There should not be any difference between the two Situations.
What objects does the collection proxy’s collection contain. Have you double checked there is no possible reference to the atlas in any objects there, or by creating a gameobject or collection through factories (in scripts).

1 Like
#9

I’m absolutely sure that loaded collection do not use this atlas.
I check memory using profiler.get_memory_usage()
in situation one right after remove GO I see memory use is reduce
In situation two - a memory using is the same like before removing.

Now I try to check it using GAPID.

UPD:
ok, the memory reducing after remove GO do not related with texture. Texture still in memory. I think this is because of removing last GUI component.

#10

It’s worth noting again that this is information that is reported from the OS, nothing the engine keeps track of. It should be taken as an estimate, since the OS could still keep stuff in memory even if it is released by the app, depending on the access pattern etc.

4 Likes
#11

Thank you @sven, You right, this is bad idea to draw conclusions using this function.
I was wrong, and in both situation this atlas still in memory.

But question still open, how to a counter works (it depend of (linked to) collection, GO, factory or something else?).

I tried to make collection factory and exploit this bug, but texture still in memory after removing spawned GO. That one of the reason why I want to know how it works. (it is easier to use something when you know how it works)

1 Like
#12

When are you calling profiler.get_memory_usage?
The process of deleting GPU related resources is deferred to make sure they aren’t used by the GPU when being deleted. Are you checking memory usage in the same frame as the go is being deleted? If so, wait a few frames before checking / comparing the value.

You can’t remove an atlas resource. It will be removed when no resources other are referencing it anymore.
When a component (for example a sprite or a gui) is removed, it will dereference all it’s referenced resources, for example an atlas and when there are no more components referencing a resource (counter is zero) it will be deleted, or in the case of GPU resources, marked for delete (deferred).
This is common for all engine resources, so if deleting the one gameobject who’s components is referring the atlas isn’t releasing the atlas it might be a bug.

5 Likes
#13

in update method

I checked it twice using GAPID.
New situations (all information from GAPID trace):

Situation 3:
one collection, GO with gui component

  1. Load game and show gui
  2. Waiting 5 seconds
  3. Remove GO with gui.
    waiting 3-5 seconds
    Checking the GAPID trace in last frame : atlas still in memory.

Situation 4:
Collection with one collection proxy with many big atlases(ust for test) and one collelction factory with my GUI

  1. Load game
    (GUI atlas isn’t load to memory)
  2. Waiting 5 seconds
  3. Load collelction factory
    (now GUI atlas in memory)
  4. Waiting 5 seconds
  5. Delete GO from p.3 (with GUI)
  6. Waiting 5 seconds
    (atlas still in memory)
  7. Load collection proxy with many big atlases
    checking and my GUI atlas still in memory.

I don’t know how it works in Defold.
You tell me that it depend of direct link of some component (component referenced to atlas)

From @britzl post I understand that it’s depend of collection, not a component How textures manager works? (SOLVED) (collection referenced to atlas)

Here is information from @Ragnar_Svensson where he told that it have to depend of collection (also collection referenced to atlas) , but Defold have bug where you can load atlas using factory. But I do not understand how to unload this atlas . @sicher told that it’s impossible - that mean atlases referenced with collection or something else, but not with components.

I have to understand how it works in engine. Too many controversial information from different people.

#14

Yes, it easily gets confused when several people are expressing the same thing.
Bottom line is that when a resource, in this case your gui is referencing some other resource, in this case an atlas, the reference counter is decreased when the gui is destroyed (that is, when the gameobject containing it is deleted).
When you are destroying the gameobject containing the gui you say that the texture is still in memory. This shouldn’t be when the gui is the only resource referencing the atlas. In this case the gui component is responsible for dereferencing the atlas. If it doesn’t it’s a bug and would explain your problem.
I will look into this asap since it’s certainly not a good thing if gui’s aren’t releasing it’s textures.

4 Likes
#15

I take a break (I’m afraid my brain can explode), I’ll try to create clean/new project with example today later and recheck it one more time for GUI, GO in collection and using collection factory.

My current conclusion:
only components (not collection and so on) should referenced to the atlases. When last component (GO who contain this component) will be removed, atlas should be removed too. Right?

1 Like
#16

That is correct.

The profiler.get_memory_usage will report memory resident set size allocated by the OS for the application (https://en.wikipedia.org/wiki/Resident_set_size) on OSX.
See the documentation for profiler.get_memory_usage for more info and how it works on different platforms.

This needs to be elaborated in the documentation, we’ll look into this!

If you check the ‘Memory’ stats column in Activity Monitor (You mentioned you use OSX) you will see that memory consumption goes up and down when creating and destroying resources (loading/unloading).
If you add the ‘Real Mem’ tab you will see the same figures reported by profiler.get_memory_usage.

We do have an ongoing task/ticket to improve memory stats and other profiling related items, so we will likely re-iterate what memory consumption is reported and how. Until then, you will have to suffice with improved documentation.

3 Likes
#17

ok, thank you.

btw, In situation 3 and 4 I’m using GAPID for checking that texture is still in memory.

You mark theme like solved, but I want to make new example and recheck all test. I’ll give you know about results. As I understand from your words it is a bug that texture do not removed from the memory.

#18

I considered the main question to be the memory reported by profiler.get_memory_usage, and we sorted that out. If the engine wasn’t releasing the memory referenced by the gameobject, it would show up quite imminently.

I did overlook your question regarding your GAPID findings, so I’ll try to elaborate that.
If you run XCode Graphics Tools -> OpenGL Profiler, you’ll (likely) see that the texture is released from memory.
Using GAPID on Android, you’ll see that the textures are replaced by the same id (or likely so anyway).
This is because some drivers keeps the storage allocated so they can reuse it for future allocations instead of having to allocate new storage.
So, checking what is currently in driver memory can easily be mistaken for a memory leak. Drivers will handle this automatically and is nothing the application can control. Driver writers base their solution on what they consider is best for the hardware.

I verified this both on Mac and on Android, loading and unloading 16 unique collections sequentially each containing a 16MB RGBA textures, and in a loop.

5 Likes
#19

Collection proxy? - yes, it’s ok.
But I tell you about GO in main collection, or spawned using collection factory.

#20

I was using collection factories to spawn and delete from the main collection.
No proxy loading. Loading/Unloading as in I made sure the texture was loaded and unloaded.

2 Likes