Shooting Circles - Defold ECS game example

The Defoldmine
#1

logo

Hello! I created a game example using the ECS (Entity Component System) architecture. It took several iterations, and now I want to share the result.

In case you missed my post in Community Challenge: Explosions, I’m sharing this as a separate forum thread.

The GitHub README contains several notes about how the game and architecture work. I followed these main rules for the project:

  • Use the tiny-ecs library without modifications.
  • Entities should be simple Lua tables without any special data.
  • No external require() calls in systems to maintain portability between projects.
  • Components can only be modified within one system. For example, entity.transform.position_x = 10 should only be changed in the transform system.

I think this might be interesting for programmers. If you have different approaches, ideas, or questions, let’s talk!

There are still some areas where it’s unclear how to do things the “correct” way (though in real games, we often cut corners because it’s faster and easier), such as event handling, debugging, and correct composing different systems, but I think this is a good start!

GitHub: https://github.com/Insality/shooting_circles
Play HTML: https://insality.github.io/shooting_circles

16 Likes
Community Challenge: Explosions! :boom:
#2

One interesting point might be how to handle the Defold GUI in ECS. In this project, I used the following approach:

  • The GUI collection contains only the GUI element. This collection can be loaded as a bootstrap collection, allowing the GUI elements to be built independently of the game. Once the game is ready, you can connect it to the GUI though their “bindings”.
  • The GUI script creates a set of “bindings” (I use defold-event to enable cross-context function calls). These “bindings” are stored in a lua module, indexed by the game object’s url key (acquired via msg.url()).
  • The GUI is an entity with a “game_object” component and a specific component for the GUI.
  • When the game_object is created, the GUI system can acquire the GUI bindings through the game object’s URL and subscribe to or call bindings to set up the GUI state.

This approach allows to switch or replace logic or visuals separately from each other, which is cool!

I also have an simple diagram how it works here:

GUI struct
GUI struct3671×1062 273 KB

8 Likes
#3

The tiny-ecs library is quite popular here, likely because it’s very easy to use and allows for flexible implementation. The @dlannan made an interesting topic about it: Flix Procedural Training and Movie Making.

There is a lot of discussion on the web about ECS, saying that additional tools are more crucial for this approach if compared to others.

An ECS admin panel can be a powerful tool. Since this type of architecture allows you to inspect and modify entities and systems easily at runtime, having a set of instruments to operate with it would be very cool!

8 Likes
#4

@Insality thank you very much for this! :heart:
How is your experience with ECS? Is it convenient? How is the development looking like while using it?

#5

Not much honestly.

I made 4 prototypes on tiny-ecs until build the “add-content” flow (you all know that I’m about :upside_down_face:). Tested different approaches with various things like saving, network, level design, UI, available tools, how iterate over balance and levels. And looking how in general the development flow is going with that. It’s pretty fun way to understand tech by trying it in various cases!

Hard to compare about convenient, cause all have different development flow. It was pretty hard to me start thinking in fully data driven component way, like on this shooting circles. But what it seems to me (from code side):

  • Most systems will be equal between your projects
  • Annotations for Lua works good in this way
  • For most feature/task for your project you will have only one way to start doing this. Feels like low friction before start implementing anything in the game. About one way I mean it can be even “step by step” flow to add instruction to yourself or your team members.
  • System’s code are pretty easy to understand and debug due it’s mostly standalone file logic.
  • More debug tools will be required as you grow. To see kind of chain of events, different collisions and leaks.
  • Saving process was different in my case. I had like “save_transform” or “save_field_state” components and each corresponding system save/load this component separately (they was linked by tiled_id field from level editor)
  • With Tiled the content design are linked with code development. I usually place data as I wish to design in this game and going to write systems to make this level work. From some time it feels like “game first” and not “code first” way to do game.

Also it’s hard to stay in this ECS rules. And probably not required and each should modify them. And sure use this as a tool in your toolset for games.

2 Likes
#6

It can be quite useful :slight_smile: The tiny-ecs repo I shared uses the same original tiny-ecs with a single modification (added a system counter for calls - so I could check systems call count during operation). The server can hook straight into pretty much any tiny-ecs setup. I will add docs on how to do this.

Love the little exploding game too. Its very nice.

Some things Ive found with tiny-ecs (and ecs in general too) are that they arent really ideal for very large entity counts. Up to about 100,000 is pretty good. After that, lua’s hash hurts it. Thats were I would move to a more graph oriented system (like the opensteer proximity database).
Also, if you have many systems it really needs some load balancing and sequencing, but that is a fairly small use case set. Overall its great for building game systems quickly, that need entities to be managed in some way.

if you are interested in the repo for the server + tiny-ecs its here:
https://forum.defold.com/t/tiny-ecs-http-server/77253
Sorry to cross post, not sure if you had seen the topic.
Thanks for sharing!

2 Likes
#7

Yeah, sorry! Forgot about this thread. Thanks for linking it here.

It’s true that doing things via update in each system are quite resource-consuming in Lua. One of option to resolve this is using more event-way approach to react in on_add instead of update (like in this shooting circle example). This can help optimize large cases, though it comes with some other costs.

From the other side I think acting in update is enough for a lot of games and can be faster to develop

1 Like
#8

Agreed. Although I would be wary of events too. The hash maps (tables) are easy to make very large and start impacting your work. Additionally tables will rebuild their indexes on runtime inserts and during other changes. Its brilliantly fast for reading, but can be messy for writing. I tend to recommend “pre-indexing” all your hashmaps before you get to run loop, then you can assign and read at very high speed :slight_smile:

There is also a little trick in lua too. If you use lua as arrays (assign key as an integer or declare a list of comma separated initializers) then it will store as arrays and can be much more performant on large scales - it does millions quite well.

There are some good docs about this (specifically luajit) that can help if needed for perf if you run into it. Lua can be made to work extremely fast (C like) if care is taken :wink:

Note: I tend to always leave these sort of optimizations to a post process. Its nice to have to have a way to really ramp up perf when you prepping for a release :slight_smile:

Again, really love your work too (not just this your games and such). Thanks again. Really should have made a 3D explosion for the challenge.

4 Likes
#9

I’m really under vast impression of book “Data Oriented Programming” by Yehonathan Sharvit (a lot from it is available online, e.g. What is Data Oriented Programming? | Yehonathan Sharvit, but I also recommend full book) which gathers all the data related approaches (that are existing here since LISP) into one set of 4 rules:

  1. Separate data from functionalities (behavior)
  2. Representing data with generic structures
  3. Treating data as immutable
  4. Separating data schema from data representation (so dynamic typing with optional runtime checks, e.g. with JSON schema, but I use Lua table here simply)

This differs from DOD which focuses on cache misses and general performance, when thinking about data first, but data in memory. DOP on the other hand focuses on data as abstraction and on designing proper architectures basing on them. It’s language agnostic and paradigm agnostic, so can be applied (or broken) in FP or OOP.

Rules #1 is undoubtly good imho, I do it anyway as much as possibile, because I spent my whole life writing OOP and find stateful classes bugprone as hell.

Because of this rule #2 feels also good, because you can then reuse a lot of code that works with all the data (but there is a cost to it - data validation, that is addressed by rule #4)

Point number 3 is most controversial, because not all languages support it natively (but more and more do), but it can be achieved even in Lua (I made a Lua module for this, testing it and will soon open source it).

But beside such a set of rules looks very beneficial, especially for bigger architectures and data heavy games. You can store only one set of immutable data (think of it like “version”) or multiple of them (and traverse back in time, if you wish). You don’t change the data, but you create new version of data and “commit” it - the name is purposeful - imho GIT perfectly follows DOP rules and is a great example on how to think about data in DOP way. You also don’t need to mąkę copies of ALL your data only to change one field, e.g. updated player’s position or some single item in inventory - you can use something like structural sharing to only change affected data and reference to the data from previous version for the rest (those field can also reference to previous versions, etc.)

I’m testing full DOP approach for one project (in Defold, in Lua), so I can tell more after it, but so far, I’m very pleased.

Ah, and finally - ECS is one of the Architecture that perfectly fits DOP, that’s why I come up with it here. I would say it’s one of the DOP implementations that was vastly adopted in gamedev, especially for similar objects clones (aka enemies)-heavy games.

And Clojure is definitely the best for DOP, but since I still can’t come to an agreement with FP I can’t tell if Defold Editor is using approach that benefis of those rules ( and can’t tell if the graph based implementation is).

7 Likes
#10

Sounds like you might like FBP :slight_smile: Its a brilliant development system, but sadly, most developers struggle with the concepts and thus it has had limited traction over the last 50+yrs :slight_smile:

1 Like
#11

Thanks for the reference, didn’t know about FBP.

I’ve been doing some tests also with ECS and Defold, but I’m still trying to figure out how to organize the code. I’m doing a mix of OOP and ECS and I’m quite happy with it!

2 Likes
#12

Wrote a 3D engine in FBP back in early 2000’s. Was pretty awesome, could switch between DX7 DX8 and OGL live. When you have these sorts of architecture, you can do some pretty amazing things. :slight_smile:
If you get bored and want to look at something weird (and prob broken):
https://github.com/dlannan/deity
Uses old original Lua too :slight_smile: One day I will make a Defold version of it… when I retire from my day job :slight_smile:

3 Likes
#13

Wow that’s quite a change to do it live!
I love this type of design that can allow you to abstract the game from the game engine itself, but at the same time I think: making a game is hard enough without trying to change the coding mindset lol

1 Like
#14

Yeah. Its quite a different architecture. The key benefits are large tho:

  • runtime flexibility
  • no dependencies
  • cross platform is included (since you are using ports/channels to move streams)
  • very much like pc nodes in the web in design
  • extreme flexibility
  • huge reuse - this is probably one of the biggest benefits. Modules actually can be reused in any program, since the program becomes more of a “wiring” + data process.

The major downside is its complex to debug (hence why I builtin a debugger into the kernel).
Will revist with Defold one day. Its very powerful.

2 Likes