# E2E testing in Defold **URL:** \ **Category:** Questions\ **Created:** [April 21, 2021, 11:22pm UTC](https://forum.defold.com/t/e2e-testing-in-defold/68214 "2021-04-21T23:22:19Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![banool](https://forum-defold.b-cdn.net/user_avatar/forum.defold.com/banool/32/24632_2.png) [@banool](https://forum.defold.com/u/banool)\ **Post date:** [April 21, 2021, 11:22pm UTC](https://forum.defold.com/t/e2e-testing-in-defold/68214/1 "2021-04-21T23:22:19Z")
Hi all, I am getting to the point in my game dev journey that it is becoming unwieldy to test new features. Every time I add something new that I want to test in a multiplayer environment, I have to do the following: 1. Run the server locally. 2. Bundle the app. 3. Bundle the app a second time (this time I do HTML 5 so I can get two at once). 4. Click Create Game on one client. 5. Type in the room code that I see in the server logs into the second client and click Join Game. Only after this do I have two players loaded into the level and finally test what I changed. I see a post from 4 years ago about integration testing but it didn’t look too easy, and some of the links are dead: [Testing with dmengine\_headless - #2 by britzl](https://forum.defold.com/t/testing-with-dmengine-headless/1772/2). These days are there tools out there to enable easier E2E testing, such as clicking through the UI or perhaps even sending messages on the side to progress my game to a certain state? Thanks! ---
**Author:** ![Mathias\_Westerdahl](https://forum-defold.b-cdn.net/user_avatar/forum.defold.com/mathias_westerdahl/32/228_2.png) [@Mathias\_Westerdahl](https://forum.defold.com/u/Mathias_Westerdahl)\ **Post date:** [April 22, 2021, 7:14am UTC](https://forum.defold.com/t/e2e-testing-in-defold/68214/2 "2021-04-22T07:14:52Z")
Although I don’t see a quwstion, I can perhaps share some ideas for improving some of the steps at least. When rebuilding/bundling a project often, I always use our command line tool [Bob](https://defold.com/manuals/bob/?q=bob). It would allow you to bundle for different platforms to different directories. I’d collect all these steps in a script: “rebuild.sh” The server. Depending on the control you have over it, there are a few options. - Output the room id directly to a separate text file (e.g. srv\_debug\_room.txt) to make it easier to find/read - Pipe the output to a log file, and the output for room id, then output the room id to the text file When adding a debug room to your build, you can output the room id into a separate debug.ini file: ```lua [debug] room = my_room_id_from_the_logs ``` Then you can pass that to bob with the “–settings=debug.ini”, and that setting will be available ingame as “debug.room”. (Should be nil or empty string otherwise) I would then launch the local engine executable. And finally, I’d start a python server in the HTML5 bundle directory: ```lua cd ./bundle_html5/ && python3 -m http.server # and possibly open a new browser page # different command for different platforms open ./bundle_html5/path_to_index.html ``` ---
**Author:** ![britzl](https://forum-defold.b-cdn.net/user_avatar/forum.defold.com/britzl/32/23_2.png) [@britzl](https://forum.defold.com/u/britzl)\ **Post date:** [April 22, 2021, 8:53am UTC](https://forum.defold.com/t/e2e-testing-in-defold/68214/3 "2021-04-22T08:53:12Z")
> [@banool](#): > > These days are there tools out there to enable easier E2E testing, such as clicking through the UI or perhaps even sending messages on the side to progress my game to a certain state. Yes, check Poco SDK for instance: [https://github.com/defold/extension-poco](https://github.com/defold/extension-poco) I also did some tests with Cucumber automation a couple of years ago: [https://github.com/britzl/cucumber-defold](https://github.com/britzl/cucumber-defold) ---
**Author:** ![banool](https://forum-defold.b-cdn.net/user_avatar/forum.defold.com/banool/32/24632_2.png) [@banool](https://forum.defold.com/u/banool)\ **Post date:** [April 22, 2021, 8:06pm UTC](https://forum.defold.com/t/e2e-testing-in-defold/68214/4 "2021-04-22T20:06:31Z")
Hey @Mathias_Westerdahl, this advice was gold, I’ve managed to get 95% of the way towards my desired setup. I have a Python script that spins up an instance of my backend and two instances of my frontend. I also have it set up almost as you described: 1. Build client 1 with a settings file that looks like this: ```lua [localtest] create_room=blah ``` 1. Start up client 1. It reads this and then creates a room. 2. The server writes the room ID to a file. 3. Build client 2 with a settings file like this: ```lua [localtest] join_room_id=xDOvLT_tV ``` 1. Client 2 joins the room. This part doesn’t work right now, instead the create\_room block triggers, even though I’m confident the file is correct. I have this working most of the way, but I am running into a couple of issues: 1. I can no longer see the debug console output from the client, since I am running the built app directly. I was hoping perhaps it would get output to stdout + stderr, but it doesn’t seem that way. Is there some way to make this happen, or should I add some custom logging solution? (and do you have a recommendation there) 2. When I check certain config values (`sys.get_config("localtest.create_room")`) (which I learned to do from here: [Access gravity in Project Settings (SOLVED)](https://forum.defold.com/t/access-gravity-in-project-settings-solved/11087)) it says they are true even though I did not set that value in the settings for that build. I think I could probably figure out 2 if I had 1 working though. Overall though this is a way better solution than what I had, thanks a lot for the advice. Now I just run one command and I get the full E2E setup. Thanks so much! @britzl Those look great. I’ll report back when I get around to using poco. Thanks! ---
**Author:** ![britzl](https://forum-defold.b-cdn.net/user_avatar/forum.defold.com/britzl/32/23_2.png) [@britzl](https://forum.defold.com/u/britzl)\ **Post date:** [April 22, 2021, 8:51pm UTC](https://forum.defold.com/t/e2e-testing-in-defold/68214/5 "2021-04-22T20:51:37Z")
> [@banool](#): > > I can no longer see the debug console output from the client, since I am running the built app directly. I was hoping perhaps it would get output to stdout + stderr, but it doesn’t seem that way. The default build variant is a release build when using bob.jar. You can specify `--variant=debug` when calling bob to get a debug build. Bob options: > **[Usage - Defold project builder manual](https://defold.com/manuals/bob/#usage)** > > Bob is a command line tool for building Defold projects outside of the normal editor workflow. | Bob is a command line tool for building Defold projects. This manual explains how to use the tool. > [@banool](#): > > When I check certain config values ( `sys.get_config("localtest.create_room")` ) (which I learned to do from here: [Access gravity in Project Settings (SOLVED)](https://forum.defold.com/t/access-gravity-in-project-settings-solved/11087)) it says they are true even though I did not set that value in the settings for that build. Hmm, sys.et\_config() should return nil if the value doesn’t exist. ---
**Author:** ![banool](https://forum-defold.b-cdn.net/user_avatar/forum.defold.com/banool/32/24632_2.png) [@banool](https://forum.defold.com/u/banool)\ **Post date:** [April 22, 2021, 9:40pm UTC](https://forum.defold.com/t/e2e-testing-in-defold/68214/6 "2021-04-22T21:40:40Z")
@britzl, setting `--variant debug` did the trick to be able to see the output! I realize that the settings are acting very strangely. See these two command invocations: ```lua # First ['/usr/local/bin/bob', 'build', 'bundle', '--settings', '/var/folders/lb/8_8c7mts709833552qtv70n40000gn/T/hotdogshl8eddoy/localtest.ini', '--platform', 'x86_64-darwin', '--variant', 'debug', '--archive'] # Second ['/usr/local/bin/bob', 'build', 'bundle', '--settings', '/var/folders/lb/8_8c7mts709833552qtv70n40000gn/T/hotdogskonw0tvk/localtest.ini', '--platform', 'x86_64-darwin', '--variant', 'debug', '--archive'] ``` I confirmed that the two different settings files are as I describe in the previous comment. I see that what happens is the second client just uses the value for these settings from the first build of the client. If I specify `clean`, this does not fix it, nor does `distclean` (or both). The only fix is to manually delete the `build` directory. When I look in `frontend/build/default/game.projectc` I see that this is indeed the case: ```lua $ cat build/default/game.projectc | grep -A 10 localtest [localtest] create_room = blah ``` The immediate solution here is to just keep doing what I’m doing, where I manually delete the entire `build` directory at the start and in between building each instance of the client, but I wonder if there is a better way? ---
**Author:** ![britzl](https://forum-defold.b-cdn.net/user_avatar/forum.defold.com/britzl/32/23_2.png) [@britzl](https://forum.defold.com/u/britzl)\ **Post date:** [April 22, 2021, 10:04pm UTC](https://forum.defold.com/t/e2e-testing-in-defold/68214/7 "2021-04-22T22:04:42Z")
> [@banool](#): > > but I wonder if there is a better way? Yes, there should be. Can you please share your script here or the relevant parts where you call bob.jar? ---
**Author:** ![banool](https://forum-defold.b-cdn.net/user_avatar/forum.defold.com/banool/32/24632_2.png) [@banool](https://forum.defold.com/u/banool)\ **Post date:** [April 22, 2021, 10:16pm UTC](https://forum.defold.com/t/e2e-testing-in-defold/68214/8 "2021-04-22T22:16:10Z")
The script is sort of complicated, and getting more so. The command invocations above are what I’m running though, just those two commands. If you need something more specific though please let me know. ---
**Author:** ![britzl](https://forum-defold.b-cdn.net/user_avatar/forum.defold.com/britzl/32/23_2.png) [@britzl](https://forum.defold.com/u/britzl)\ **Post date:** [April 23, 2021, 4:08am UTC](https://forum.defold.com/t/e2e-testing-in-defold/68214/9 "2021-04-23T04:08:39Z")
What happens if your specify different --build-output folders for the two builds? ---
**Author:** ![banool](https://forum-defold.b-cdn.net/user_avatar/forum.defold.com/banool/32/24632_2.png) [@banool](https://forum.defold.com/u/banool)\ **Post date:** [April 23, 2021, 4:26am UTC](https://forum.defold.com/t/e2e-testing-in-defold/68214/10 "2021-04-23T04:26:12Z")
I have figured out the issue, it’s just dumb user error. I sort of assumed that if I did something like this: ```lua bob build bundle clean ``` That it would order them in the way I wanted (which doesn’t make sense in retrospect). What actually happens is it runs each command in order, so I what I actually need to do is this: ```lua bob clean build bundle ``` With that, the directory gets cleaned appropriately before the build, ensuring that artifacts aren’t left behind. I am sort of surprised that `bob build` after a previous build doesn’t produce the same output as if I had run it from scratch, but I’m sure there’s a good reason for it. Or maybe not and this is a bug, perhaps specific to settings files? Nonetheless, thanks @britzl for helping me figure this out!