I would say it’s definitely the fact that you are doing calculations so far away from the origin.
I have a list of “smallest steps” that can be represented here:
At 32km, the difference between each step is ~2cm!
And that will only compound as, of course, there are calculations made with these numbers.
f: 30000 next: 30000.001953125 diff: 0.00195312
Uggh. Life was so much easier when you could just write pixels yourself.
If we’re talking old school rendering, it is very likely that you would have the same problem with overflowing the fixed point math, so the case would still need to be handled.
The suggestion is to move the sensitive calculations back to origin.
E.g. for each sensitive object, translate them back close to origin, and place the camera accordingly and render.
In this case, it may perhaps suffice with moving the airplane back to origin.
Yeah thanks. I guess the camera (world) matrix has those float positions in them, and that kinda multiplies into the problem. Have seen this before on big sims at around the 20km mark.. but was hoping the vert rendering could be localised and the verts would be handled in the gpu as relative to camera. Clearly not
I mentioned pixel days because rendering this terrain and even really simple things like roads, and shapes.. is an utter pain with the GPU. With a sw renderer.. we often did really simple drawlines over the scene - you could do this with a render texture and sw engine (have done that) but there are things.. that are just easier to do in plain pixel drawing. Making retro looking games, isnt really rendering the same either.. hrm.
Maybe.. I will do it all in sw… then I dont need to worry about the precision. hrm… already have an example renderer for that too. hrm…
Im not keen on the floating origin thing because of the problems with all the other objects in the scene (ships, missiles, aircraft, cities, roads, bridges and so on). I have a renderapi interface that intercepts all go render updates its doable.. but the side effects and all the modelling … hrm.. maybe a few days of tinkering first.
Made a little test project. Will put up here in a bit. And heres the results.
On the same map with same scaling where these objects spnning roughly measure 0.01 of a unit which is 1m and are positioned some 34000 x 16000 away from the origin (thus to them its 3400,000m away or 3400km)
There is substantial vert movement. Oh well.
I have three options Im considering right now:
reduce the problem to grids or chunks (since you should never be able to see too much) and thus then do a chunk loading with changing origin.
Have a generic floating origin with the map as is.
Do a different style of render for the large world portions, and then have a more localized rendering scheme (maybe software renderer).
If anyone has any other ideas (especially reasonably simple ones to implement ) .. happy to hear from you. I probably should consider talking to gpt..
This has been a good process, since it took me back to making something I really wanted to build anyway. And thats a world builder for earth scale games (because I want to do a series of these anyway).
After some tinkering today.. I now have a realtime grid based terrain (and data) loader that will solve alot of these problems. Thus the player will probably only ever work within about a 40km - 50km range of the tile/world origin.
For the whole planet this will work out to about 4000 files if I did ocean as well, Im not, land objects only, so it should be less than 1500 files all up in a lat/lon gridded format. Quite nice.
Still alot to do, to get it in and working well, but the data as triangulated here, means I can push it to a tri buffer.. and render normally.
I might share the worldbuilder project will see how it goes first. Prob a couple more weeks of tweaks and changes to make this all fairly smooth (like stream loading.. assets and tiles.. etc).
A little late to this, and my understanding here is limited at best. When I’ve bumped into this issue my (quite basic) solution has been to keep the main object (in this case the plane) at origo and move the world instead. This may not be a feasible solution in your case, but thought is drop this “very simple” solution in here for flavour.
(a) Strap camera to the aircraft (and damp horizon shake)?
(b) Damp the movement on N*dt ?
(c) Apply delta’s to visuals in local/camera space.
Probably make little sense in your context.
Localizing the origin to the f18 is a problem. This means you need to update all objects (static and dynamic) every frame to ensure world consistency. With a busy battle area with missiles, numerous enemies and more, this is a pretty big burden on performance. Also, I have a number of camera modes (from different places - if youve played the original, youd know what Im referring to ) and they can be from a wide array of vantage points - mobile and static.
This method works ok for infinite runners and similar, but thats usually a quite constrained data set.
Its important too, that the models (physical) Im running for the planes and ships, that they dont become overly complicated as well (esp with collision with other entities etc).
The actual easiest way, is to use doubles in the engine This is what I have done in the past for some of the big sims Ive worked on (train sims, car sim, and defense sims). Especially when making systems > 50km.. you need to have clean and consistent modelling in these types of projects. Note: All my entities are using doubles.. so I dont have to worry too much about their actual position info getting corrupt. Its more about managing and handling larger scenes to get the rendering within good F32 scales.
Ive done grid chunking before and its looking like its going to be the easiest/least risk/least impact atm.
Thanks for the thoughts tho. Keep them coming!
I still have an interest in rendering all the terrain into something a little simpler. Now I have a good and small data set (whole world is tiny 24MB (packed of course).. I may need to use a detail lower which will blow out to 130MB+ but it means it can easily live in the GPU or something interesting like that. Ie.. do some sort of terrain raycasting in a shader. Dunno. Big worlds are always an interesting problem to solve
World builder is coming along now. The tile/chunk loader is pretty good. I can preload the whole data set, and generate blocks of tiles. Im doing 25 ( 5x5 ) atm.. but prob only really need 9 (3x3).
Will be a couple of weeks before I can regenerate my missions. Its a bit of a delay, but will mean I have a bespoke mission/world editor tool which will probably end up on steam workshop or something like that.
The roads have been added and they are quite detailed. I originally was going to use some gl methods to do this, then I thought of a bit of an odd solution.
The roads (segments) are all individual sprites .. there’s over 4000 in that picture alone.
Its alot faster and much easier to tweak too (I can texture roads differently and so on).
It uses the normal sprite material and shader atm, but I’ll be changing that a bit later to be able to batch the sprites better (since they are all static).
For a hint how this works .. its a 1 pixel sprite. That I stretch in X the length of the segment, and width can be adjusted to suit the line width I want (can be different for different road types). Its a super simple solution, and if people want a little sample project to play with, I’ll put it up.
Onto:
Engine runtime integration
Feature placement in the world builder
Mission setup in the world builder (copy over from my Blender scripts).
Hopefully in a few weeks, this will all be back in the main game, and the large world system will happily work! (over the whole planet ).
Interesting side note. Total world data for roads, land and urban is: 96MB. Kinda nice
Heres the sample line drawing project within the basic 3D sample from Defold.
I wanted to be able to render lines in 3D too.. not just in 2D. There’s only really one script of importance, and a couple of other necessary files - atlas, png, and factory.
Feel free to use it however you like
Thanks. There are a few limitations in this implementation though:
maximum of 32000 ish (I havent checked but prob 32767) sprites (in project max sprites config).
because they are factory go’s its not really super ideal from a batch/send setup. It would be better if it was a buffer with pos, len, angle, scale as data streams. You could render a tun more (Im working on this).
depth management and scaling is all in viewspace (this is how the sprite material shader works) so there are some things to watch out for.
no culling .. again.. something Im going to add to my batch one.
Its kinda cool how nice it works though, you can set every color and individual line thickness too. So there are some really nice benefits.
Slowly getting there. All the main layers are looking good in grid chunks.
Added mission imports from the Blender tool. Hope to have mission design and export done this weekend.
Then I need to do some integration. Im fairly happy with the rendering perf and the detail so it will be a matter of making sure all the lat/lon data maps in nicely. Fun stuff
Ughh. I dunno about others. But the editor is really getting bad. Simple things too:
Opening a collection with 3D world in it. This used to be pretty instant. Now.. on my i9 RTX 4060.. it is just slow (can take 30 seconds)
Closing a window tab in the editor with a go or a mesh. I have had the editor hang for minutes. And this has been happening for the last few releases.
General responsiveness. I get regular hangs and slow downs. The scenes are now smaller than before but Im getting much worse editor performance. Grrr.
Im posting here, because this is my dev blog and its becoming so annoying it really is slowing me down. Im now trying to do most of my work in VSCode and trying to avoid the editor as much as I can.
I really just want to get this finished. Very frustrating.
As a note on what I see in task manager, it looks like the java runtime is locked up alot. There are a number of times where I see multiple java runtimes and one is spinning using up a full core and making the editor unresponsive. This is on Win11. I would report a bug, but I do not intend to use Defold for future projects, I just have to get through this.
Im even wondering if I should convert to Unity.. just so I can get this game out. The amount of problems I keep running into is very frustrating.
On the good side of things, the worldbuilder is mostly done (enough to make missions etc). And the integration is also about 50% through and looking pretty good.
Few things to fix up, but overall should work well.
Both of these will improve performance by quite a bit when working with 3D content. I can’t say if these two are the source of your problems. We would like to figure that out. If you are willing to privately share your project with us (britzl (Björn Ritzl) · GitHub or bjorn@defold.se) we’d love to do a round of profiling to figure it out!
Ah, well, ok, you’ll just have to suffer through it then I guess…
Yeah. Soz. Im not sure either of those issues are related. This appears to be a java runtime thing. Maybe its Win11 antivirus going nuts or something (although it doesnt appear to be).
Project wise… its kinda getting sizable and I just dont really want to spend more time on it (soz).
Heres a vid of “closing a tab” problem that I have on a regular basis. This is a “good” and “reasonably quick” close
For some stats on project files and folders (minus build and .git etc)
Size: 316 890 955
Folders: 91
Files: 632
I dont think its really overly large. This was happening for at least a year. I know if I go back to 1.8.1, it all goes away, and theres no problems. However, much of the newer parts of my project use some fx which have changed a little so its not a great options (to use the old one), although I may.. will see.
Also, this project is a little different too, I guess. Since its probably doing a bunch of things generally outside the scope of some of the general Defold system. I have three extensions too (yes, Ive checked them) as well that are needed to run it, which shouldnt be the source of the problem since the old 1.8.1 works fine with the same extensions.
I hope thats some info on the matter. I wanted to add it in here, since its been an ongoing thing here that has really slowed progress - maybe others have similar.. maybe its project specific. Dunno. For the time being I live in VSCode
That looks like a terrible experience and not how we want the editor to behave. We would probably be able to pinpoint and fix that quite quickly if we could reproduce it on our end.
Ok. Ive put aside some more integration for the time being. Most of it is working well. Very happy.
Have moved onto replacing the whole UI. I did engage with some artists previously, but I wasnt overly happy with how much of it came out, and one discussion with an artist, suggested I do it in 3D and use some retro styled rendering and thus producing something more suited (This was great advice btw - because the pixel art 2D UI had other issues too - resolution scaling, fonts and more).
So.. first pass at this tonight, and I think its going to be ideal. Will need some more texturing and filling out the scene a little but its definitely the right direction.
Here’s a little taster
Lots to do (like nice transitions from the 3D to the 2D panel and so on). Will tweak the shader some more too, to have lights, some fake shadows and lights flickering, as well animations on the screens.
I might also put in a couple of pilots, with some audio chatter.