Game crashes on user's devices (SOLVED)

Made a hot fix update to Google Market this night. The build assembled in 1.2.139. Seems it’s more stable and users found it as workable. Also shows reducing of number of Crash report in Google console.

Well. What’s wrong with 1.2.137?

2018-10-30_10-48-23
0.98 assembled in 1.2.137
0.981 in 1.2.139

ps. noticed that in 1.2.139 we have a new render of “text leading”. The same font size, but a bit more line spacing.

1 Like

We did have a problem when closing the game and music was playing which caused a crash. This has since been fixed. Could it be this issue you’ve come across?

I think we’re releasing a hotfix today for this crash: Font crashes engine with OpenGL error

There shouldn’t be any change to how fonts are rendered, but the hotfix and crash might be related to what you are experiencing with fonts. @jhonny.goransson?

1 Like

Yeah the leading and OpenGL issue should be fixed with todays release and everything should render as normal!

3 Likes

May I please steal this thread to remind everyone that we at Defold have little means (currently) to actively monitor ANRs and crashes on live games overall. So please, people, don’t be shy, report crashes, ANRs and just weird things with your games. It is of value to us!

10 Likes

This is still an issue in 1.2.140 =(

1 Like

yep, “tkgill” is most popular crash on Android

Reports 38 Last 7 days
Android 8.0	38	100.0%
Samsung Galaxy S7 edge (hero2lte), Android 8.0
Report 1

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> com.dragosha.hamster <<<

backtrace:
  #00  pc 000000000004b748  /system/lib/libc.so (tgkill+12)
  #01  pc 000000000001a4c3  /system/lib/libc.so (abort+54)
  #02  pc 000000000033a063  /system/lib/libart.so (_ZN3art7Runtime5AbortEPKc+370)
  #03  pc 000000000033a65b  /system/lib/libart.so (_ZN3art7Runtime7AborterEPKc+10)
  #04  pc 00000000003f2231  /system/lib/libart.so (_ZN7android4base10LogMessageD1Ev+456)
  #05  pc 00000000001c81eb  /system/lib/libart.so (_ZN3art22IndirectReferenceTable17AbortIfNoCheckJNIERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE+166)
  #06  pc 0000000000234e2f  /system/lib/libart.so (_ZNK3art22IndirectReferenceTable10GetCheckedEPv+266)
  #07  pc 000000000023162f  /system/lib/libart.so (_ZN3art9JavaVMExt12DecodeGlobalEPv+10)
  #08  pc 000000000035beab  /system/lib/libart.so (_ZNK3art6Thread13DecodeJObjectEP8_jobject+162)
  #09  pc 00000000003352d7  /system/lib/libart.so (_ZN3art35InvokeVirtualOrInterfaceWithVarArgsERKNS_33ScopedObjectAccessAlreadyRunnableEP8_jobjectP10_jmethodIDSt9__va_list+42)
  #10  pc 0000000000264d7d  /system/lib/libart.so (_ZN3art3JNI18CallBooleanMethodVEP7_JNIEnvP8_jobjectP10_jmethodIDSt9__va_list+436)
  #11  pc 000000000008536c  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #12  pc 000000000016a190  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #13  pc 000000000015e92c  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #14  pc 0000000000019844  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #15  pc 000000000001b794  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #16  pc 000000000001b808  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #17  pc 0000000000017d80  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #18  pc 0000000000017cd4  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #19  pc 000000000013e584  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #20  pc 0000000000017cfc  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #21  pc 000000000013ee5c  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #22  pc 0000000000048a5f  /system/lib/libc.so (_ZL15__pthread_startPv+22)
  #23  pc 000000000001b333  /system/lib/libc.so (__start_thread+32)

next one called “abort”

|Android 8.1|21|84.0%|
|Android 9|4|16.0%|

Today, 18:37 on app version 57
Nokia Nokia 7 plus (B2N), Android 8.1
Report 2

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> com.dragosha.hamster <<<

backtrace:
  #00  pc 000000000001a71e  /system/lib/libc.so (abort+63)
  #01  pc 000000000035b469  /system/lib/libart.so (art::Runtime::Abort(char const*)+392)
  #02  pc 000000000041ae2d  /system/lib/libart.so (android::base::LogMessage::~LogMessage()+452)
  #03  pc 00000000001d55d1  /system/lib/libart.so (art::IndirectReferenceTable::AbortIfNoCheckJNI(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)+164)
  #04  pc 0000000000251d77  /system/lib/libart.so (art::IndirectReferenceTable::GetChecked(void*) const+262)
  #05  pc 000000000024e5fb  /system/lib/libart.so (art::JavaVMExt::DecodeGlobal(void*)+10)
  #06  pc 000000000037d429  /system/lib/libart.so (art::Thread::DecodeJObject(_jobject*) const+164)
  #07  pc 0000000000356ba3  /system/lib/libart.so (art::InvokeVirtualOrInterfaceWithVarArgs(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jmethodID*, std::__va_list)+42)
  #08  pc 00000000002830e5  /system/lib/libart.so (art::JNI::CallBooleanMethodV(_JNIEnv*, _jobject*, _jmethodID*, std::__va_list)+420)
  #09  pc 000000000008536c  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #10  pc 000000000016a190  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #11  pc 000000000015e92c  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #12  pc 0000000000019844  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #13  pc 000000000001b794  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #14  pc 000000000001b808  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #15  pc 0000000000017d80  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #16  pc 0000000000017cd4  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #17  pc 000000000013e584  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #18  pc 0000000000017cfc  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #19  pc 000000000013ee5c  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #20  pc 00000000000479ef  /system/lib/libc.so (__pthread_start(void*)+22)
  #21  pc 000000000001b2ed  /system/lib/libc.so (__start_thread+32)

another “tgkill”

Samsung Galaxy S6 (zerofltelgt), Android 7.0
Report 1

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> com.dragosha.hamster <<<

backtrace:
  #00  pc 000000000004a230  /system/lib/libc.so (tgkill+12)
  #01  pc 00000000000479c3  /system/lib/libc.so (pthread_kill+34)
  #02  pc 000000000001d9c5  /system/lib/libc.so (raise+10)
  #03  pc 0000000000019511  /system/lib/libc.so (__libc_android_abort+34)
  #04  pc 0000000000017150  /system/lib/libc.so (abort+4)
  #05  pc 000000000031bc5d  /system/lib/libart.so (_ZN3art7Runtime5AbortEv+252)
  #06  pc 00000000000b4d4b  /system/lib/libart.so (_ZN3art10LogMessageD2Ev+866)
  #07  pc 00000000001bc951  /system/lib/libart.so (_ZN3art22IndirectReferenceTable17AbortIfNoCheckJNIEv+84)
  #08  pc 000000000023dc5f  /system/lib/libart.so (_ZNK3art22IndirectReferenceTable10GetCheckedEPv+498)
  #09  pc 000000000023ae0f  /system/lib/libart.so (_ZN3art9JavaVMExt12DecodeGlobalEPv+10)
  #10  pc 00000000003359b5  /system/lib/libart.so (_ZNK3art6Thread13DecodeJObjectEP8_jobject+124)
  #11  pc 0000000000317ac7  /system/lib/libart.so (_ZN3art35InvokeVirtualOrInterfaceWithVarArgsERKNS_33ScopedObjectAccessAlreadyRunnableEP8_jobjectP10_jmethodIDSt9__va_list+42)
  #12  pc 0000000000266bb5  /system/lib/libart.so (_ZN3art3JNI18CallBooleanMethodVEP7_JNIEnvP8_jobjectP10_jmethodIDSt9__va_list+444)
  #13  pc 000000000008536c  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #14  pc 000000000016a190  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #15  pc 000000000015e92c  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #16  pc 0000000000019844  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #17  pc 000000000001b794  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #18  pc 000000000001b808  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #19  pc 0000000000017d80  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #20  pc 0000000000017cd4  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #21  pc 000000000013e584  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #22  pc 0000000000017cfc  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #23  pc 000000000013ee5c  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #24  pc 0000000000047493  /system/lib/libc.so (_ZL15__pthread_startPv+22)
  #25  pc 0000000000019f6d  /system/lib/libc.so (__start_thread+6)

java.lang.IllegalArgumentException
com.dynamo.android.DefoldActivity.onCreate

Android 6.0	2	100.0%

Today, 13:44 on app version 57
ZTE BLADE A510 (P635A50), Android 6.0
Report 1 of 2

java.lang.RuntimeException: 
  at android.app.ActivityThread.performLaunchActivity (ActivityThread.java:2572)
  at android.app.ActivityThread.handleLaunchActivity (ActivityThread.java:2654)
  at android.app.ActivityThread.access$900 (ActivityThread.java:175)
  at android.app.ActivityThread$H.handleMessage (ActivityThread.java:1488)
  at android.os.Handler.dispatchMessage (Handler.java:111)
  at android.os.Looper.loop (Looper.java:207)
  at android.app.ActivityThread.main (ActivityThread.java:5728)
  at java.lang.reflect.Method.invoke (Native Method)
  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run (ZygoteInit.java:789)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:679)
Caused by: java.lang.IllegalArgumentException: 
  at android.app.NativeActivity.onCreate (NativeActivity.java:170)
  at com.dynamo.android.DefoldActivity.onCreate (DefoldActivity.java:205)
  at android.app.Activity.performCreate (Activity.java:6270)
  at android.app.Instrumentation.callActivityOnCreate (Instrumentation.java:1113)
  at android.app.ActivityThread.performLaunchActivity (ActivityThread.java:2519)

total:


Builded in 1.2.139

1 Like

1.2.140 did not address these issues unfortunately, it was more of a hotfix for the font changes that was introduced in the last release.

2 Likes

Let’s try and narrow this down and gather the information in one place.

  • What is the latest defold version that you perceive as being stable for you?
  • Are the crashes Android only or are you seeing crashes on iOS or other platforms as well?
2 Likes

last stable is 135

This is Android only crashes because I didn’t update iOS version of the game.

1 Like

I could try to symbolicate the Defold symbols, if you give me your libLookYourLoot.so file.

5 Likes
4 Likes

For comparison reason list of all crashes from previous release on Android (Aug 28, engine ver:1.2.135) for the last 60 days:

java.lang.NullPointerException
com.defold.iap.IapGooglePlayActivity.onCreate

Android 5.1	4	100.0%
TCT (Alcatel) 8050E (Pixi4-6_3G), Android 5.1
Report 1 of 4

java.lang.RuntimeException: 
  at android.app.ActivityThread.performLaunchActivity (ActivityThread.java:2521)
  at android.app.ActivityThread.handleLaunchActivity (ActivityThread.java:2601)
  at android.app.ActivityThread.access$800 (ActivityThread.java:178)
  at android.app.ActivityThread$H.handleMessage (ActivityThread.java:1470)
  at android.os.Handler.dispatchMessage (Handler.java:111)
  at android.os.Looper.loop (Looper.java:194)
  at android.app.ActivityThread.main (ActivityThread.java:5637)
  at java.lang.reflect.Method.invoke (Native Method)
  at java.lang.reflect.Method.invoke (Method.java:372)
  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run (ZygoteInit.java:960)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:755)
Caused by: java.lang.NullPointerException: 
  at com.defold.iap.IapGooglePlayActivity.onCreate (IapGooglePlayActivity.java:223)
  at android.app.Activity.performCreate (Activity.java:6100)
  at android.app.Instrumentation.callActivityOnCreate (Instrumentation.java:1112)
  at android.app.ActivityThread.performLaunchActivity (ActivityThread.java:2468)
1 Like

1.2.135 ± stable
1.2.137 - :sob:
1.2.139 - hot fix, ± stable

2018-11-01_11-18-47

1 Like

Added DEF-3585 for this. It should land in 1.2.143 (in about two weeks)

2 Likes

As for the other callstacks (tgkill/abort), they all look mostly the same:

000000000008536c	Java_com_defold_iac_IACJNI_onInvocation
000000000016a190	Java_com_dynamo_android_DefoldActivity_nativeOnActivityResult

Now, as to why it fails there, I’m not sure yet. That code is quite small, and the edge cases seems taken care of.

Do you happen to have any logcat from such a crash? It might give a clue.

So, probably I have the same problems? On the Family Age project

My last builds was on 1.2.136, builded with Bob.
But got a more tgkill errors than usual then build after 26.10. Prev build was 18.10, but there is not too much errors. (It is strange, probably our fault. The players/install count not very different on this dates)

135 build have tgkill errors too

The tgkill error:

   #00  pc 0000000000041f50  /system/lib/libc.so (tgkill+12)
   #01  pc 0000000000040fa1  /system/lib/libc.so (pthread_kill+32)
   #02  pc 000000000001c88b  /system/lib/libc.so (raise+10)
   #03  pc 0000000000019a3d  /system/lib/libc.so (__libc_android_abort+34)
   #04  pc 00000000000175ec  /system/lib/libc.so (abort+4)
   #05  pc 000000000001b49f  /system/lib/libc.so (__libc_fatal+16)
   #06  pc 0000000000019ac5  /system/lib/libc.so (__assert2+20)
   #07  pc 0000000000049818  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #08  pc 000000000004aefc  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #09  pc 0000000000038824  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #10  pc 0000000000060444  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #11  pc 0000000000038824  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #12  pc 000000000001d588  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #13  pc 000000000008bd9c  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #14  pc 000000000001fb4c  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #15  pc 000000000001fbf0  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #16  pc 000000000001c168  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #17  pc 000000000001c0bc  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #18  pc 000000000015da94  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #19  pc 000000000001c0e4  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #20  pc 000000000015e36c  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #21  pc 00000000000408a3  /system/lib/libc.so (_ZL15__pthread_startPv+30)
   #22  pc 000000000001a0c1  /system/lib/libc.so (__start_thread+6)

The abort error

  #00  pc 000000000001a518  /system/lib/libc.so (abort+63)
  #01  pc 000000000001a9e9  /system/lib/libc.so (__assert2+20)
  #02  pc 0000000000049818  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #03  pc 000000000004aefc  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #04  pc 0000000000038824  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #05  pc 0000000000060444  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #06  pc 0000000000038824  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #07  pc 000000000001d588  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #08  pc 000000000008bd9c  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #09  pc 000000000001fb4c  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #10  pc 000000000001fbf0  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #11  pc 000000000001c168  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #12  pc 000000000001c0bc  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #13  pc 000000000015da94  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #14  pc 000000000001c0e4  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #15  pc 000000000015e36c  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #16  pc 00000000000487ef  /system/lib/libc.so (__pthread_start(void*)+22)
  #17  pc 000000000001b34d  /system/lib/libc.so (__start_thread+32)

libFamily_Age.so: https://drive.google.com/open?id=1yhaARkep0mFjPQNFzNU5ks6kMtlWra0e

Also I have about 1 game crash per day (logs, get from crash.* module).
If I can help with more information, write me on slack

5 Likes

nope :frowning:
It’s all happened on users devices, so, I need a way to register this issues.

Also I found this topic:
Best way to handle crash reporting on device?

I use this module (Google Analytics) from @britzl, but there is no any mentions about crashes in GA console:

:thinking:

UPD: i didn’t enable this feature, arghhh

ga.get_default_tracker().enable_crash_reporting(true)
4 Likes

1.2.140 better than 138, but still has a lot of crashes (
But I am not sure, because after 1.2.140 we paused our UA company

I would recommend going with 1.2.141 directly since it only contains a hot fix of a font crash issue introduced in 1.2.140.

2 Likes

I’ll try today later

1 Like