Know your engine limitations!

#36

@dmitriy can you make a relative power consumption test? Like leaving it running for an hour or so and checking how much battery has been drained during the process.
You are doing great and very useful work, thank you!

1 Like
#37

Defold energy usage:

defold
defold.png1310×69 3.5 KB

min 1/20, max 3/20
Unity:
unity
unity.png1333×64 3.82 KB

min 1/20, max 12/20

According Instruments Help:
“The Energy Usage instrument indicates a level from 0 to 20, indicating how much energy your app is using at any given time. These numbers are subjective. If your app’s energy usage level is occasionally high, it doesn’t necessarily mean that your app has a problem. Your app may simply require more energy for some of the tasks it performs. For example, it may use the GPS while performing complex network operations. This is valid energy use. Look for spikes or areas of high energy use that are unexpected or that could be performed at more optimal times.”

Disclaimer: I have absolutely zero Unity experience.

9 Likes
#38

Defold is more consistent since it prepares most resources in advance while Unity does more real time management?

1 Like
#39

@dmitriy Thanks, insanely useful info

Unity does more real time management?

Unity has quite a lot of going on in background, but it’s not about resources, it’s about overall runtime. Resoures are loaded in advance in both cases, Unity just has too much of backend overhead - FMOD, Manager updates, Fixed rendering pipeline with 3D in mind (this is changing thanks to scriptable rendering pipeline in Unity, but it’s just a part of the whole runtime), all this stuff.

5 Likes
#40

@Pkeod is right, Unity does do a lot of at runtime deserializing, and prepping of content, it has huge trees of GO relationships to parse. They describe in detail in their asset bundle docs, and maybe other places. This is mainly why unity takes so long to start, there is just so much abstraction to deal with.

The new compiler tech and ECS(entity component system), unveiled last year, will likely change that, according to the lead dev. So they hope to have massive gains in performance and load times(which will better enable their webgl build, for instance).

I personally think Unity should not be seen as a competitor to Defold in terms of battery life and performance, Defold can compare it’s self to GMS and libgdx, these other lower end solutions solutions that are much more barebones.

As for the bunnymark, I didnt check your code, but if you used updates in every object you will loose performance. Ideally you should have a singleton pattern like Britzl suggested in my performance post, and poll/call->update the objs from there. Unity had a post about it a while back. But again whatever difference it makes it should really not be comparable to Defold.

On a final note,
I did a few battery tests with a friend a long while back, we tried Defold, Godot, GMS, and Unity.
Defold was the best in terms of battery life, GMS/Godot were 1.x-2x more demanding, depending.
Unity was, i think, around maybe 3x more consuming than Godot/gms, at max. So it did around 2-6 times Defold’s consumption. In a benchmark it’s one thing, but when you add lots of other objs, behaviors, UI, Unity will hog everything down, just look at the famous and big unity mobile games out there, they all have massive battery consumption.
(i don’t have these projects btw, don’t ask)

9 Likes
#41

I have just made a few performance / battery life tests on my Asus ME173X tablet (pretty low end device on which we usually test code and graphics performance) on a simple Defold logo ping-ponging from top to bottom using a Sine wave interpolation (Defold uses go.animate with internal tweening engine). I was conducting these tests for the last two days, made them at least twice per platform to ensure that results are consistent, and here are the results

unity
99% -> 1h -> 91%
99% -> 2h -> 79%
monogame
99% -> 1h -> 91%
99% -> 2h -> 82%
defold
99% -> 1h -> 93%
99% -> 2h -> 85%

Difference is quite apparent, especially after two hours, but it’s not nearly “3x more consuming”. I have also noticed that Unity and Monogame (especially Monogame) were less stable in terms of FPS (even on that small one ping-ponging logo demo), most likely occasionally missing vsync intervals, which resulted in stuttering. Defold, on the other hand, was consistent all the time. I have also noticed that Defold didn’t warm up the device while Unity and Monogame did (just a bit, it didn’t bring it to thermal throttling limits, but it was noticeable).

I will post a mini-postmortem on overall Defold usage “review” regarding our smaller 2D projects. We have worked with Unity for quite some time now (since 4.3, released a few games) and have decided to take a look at Defold as an alternative for creating more energy efficient and less development-heavy games.

13 Likes
#42

I wanted an excuse to mess around with Gideros, so I tested a quick bunnymark benchmark on my android phone (LG V20). https://github.com/lemon07r/gideros-bunnymark
Also tested with britzl’s https://github.com/britzl/defold-bunnymark

Results:
15000 Bunnies at 24 fps using go.animate() with Defold
15000 Bunnies at 29-33fps using particles with Gideros

*Fps was very erratic with the Gideros, jumping as high as 40 but didn’t seem to dip below 28.
**I’m not sure if this makes a difference but I exported to an Android Studio template from Gideros instead of to Android directly then compiled to an APK using android studio.

EDIT: Fixed HTML5 version. Try it here: https://lemon07r.github.io/gideros-bunnymark/

0 Likes
#43

Using particles with Gideros?

Try using ParticleFX with Defold with bunnies?

0 Likes
#44

I haven’t tried to port bunnymark to Defold (since britzl has already done it), will using ParticleFX be faster than britzl’s go.animate() method? I used particles in Gideros because that’s what was recommended for best performance, and I had thought go.animate() was the fasted method to draw bunnies on Defold.

0 Likes
#45

You’re comparing Apples to Oranges it looks like. If you try to get the same number of particles with ParticleFX it would be a better comparison to Gideros particles for FPS.

0 Likes
#46

Hmmm, food for the thought. I’ll try tomorrow unless someone else wants to do it. I dont have much experience with particles on Defold yet, so now I’ll have an excuse to learn. Learning to use gideros, I’ve read it’s good practice to use particles. So Im curious, why was go.animate() used over particles for bunnymark on defold?

0 Likes
#47

Particles are meant for certain kinds of effects such as the kinds you would want with many of the same objects moving together. You can use go.animate() for similar situations but there may still be better performance from using particles directly as there are other kind of overheads from the GO approach.

2 Likes
#48

I see that they are called particles but from the looks of the code you move them individually using code. Are there any limitations to particles in Gideros? Can you attach things to them? Can you change the looks of individual particles? Scale and rotation? Change the “sprite”?

1 Like
#49

Hey brtizl, I haven’t used gideros enough (literally only have a few hours on it) but I’ll try to answer your questions the best I can.

First and forth most, here is the reference: http://docs.giderosmobile.com/reference/gideros/Particles#Particles
This is where I’m answering most of your questions from.

Are there any limitations to particles in Gideros?

I haven’t used them enough to know yet! I’ll ask the Gideros community and see what they say.

Can you attach things to them?

Quickly skimming over the reference, the only form of “attaching things” I seem to see is a tagging system, but it looks like it’s still in development:
Particles:setParticleTag ()
Sets the tag associated to the given particle
and
(string) = Particles:getParticleTag(i)
Returns the tag associated to the given particle

Can you change the looks of individual particles?

Some of these are still in development from what I see, but you can change the angle, color, decay, position, size, speed, ttl, and tag using the particle index number.
It doesn’t look like you can change the texture individually with the particle index number, Particles:setTexture(texture) sets texture to all particles.

Scale and rotation?

Yes. Particles:setParticleSize(i, size) and Particles:setParticleAngle(i, angle)

Change the “sprite”?

I’m not sure what you exactly mean by change the sprite, but the documentation says particles inherit from sprites, so I’m assuming you would use the sprite class to change the sprite for things you cant do with the particle class:
http://docs.giderosmobile.com/reference/gideros/Sprite#Sprite
There seem to be a lot more transformation options using the sprite class.

1 Like
#50

Ok, so Gideros particles sound like Defold particles with the exception that Defold particles are controlled by the engine and spawned by emitters that are affected by modifiers. There is no programmatic way of controlling their movement pattern.

1 Like
#51

Would it even be feasible to draw bunnies with particlefx on defold? I took a quick look at the documentation for particlefx but it looked way more complicated than particles in gideros. If it’s doable I’d like to see how they compare using particles for a more apples to apples comparison.

PS I got a few different answers from the Gideros community if you are interested: http://giderosmobile.com/forum/discussion/7356/questions-about-particles-and-bunnymark-performance

0 Likes
#52

I added a particlefx test to my bunnymark project, but while testing I saw really weird spikes in the profiler so I’m not sure, but there could be a bug in the code when running with a high amount of particles (15k in my test). Anyway, it’s pushed to the repo if you want to do tests yourself, but keep in mind what I experienced while testing.

2 Likes
#53

Finally got around to testing it. Particlefx gets a stable 31fps for me on my LG V20 (go.animate() is still 24fps). The gideros bunnymark particles performance is very similar, slightly higher on average but wayyy more erratic.

A side note, the Defold APK size was less than 3mb, and the Gideros APK size with the single sprite and script is over 7mb. The more I use both engines the more mature Defold seems in comparison to Gideros, and I can honestly see Defold outpacing Gideros in development at this rate. I haven’t seen any major updates (or even plans) in a while from them but this all stands to question and could change in a heartbeat if things pick up over on their end.

Does finding those spikes in the profiler open up possibilities to make improvements?

0 Likes
#54

Yes, we know that we have things to improve, especially when it comes to game object performance, but probably also on particle effects.

2 Likes
#55

Would any of you be interested to see how other engines and frameworks compare in html5 performance to defold? I have a small compilation of bunnymark benchmarks of random framworks and engines I’ve been messing around with for fun. I was originally going to test for android but html5 has just been much quicker and easier to test.

4 Likes