Game crashes on user's devices (SOLVED)

Dragosha · October 29, 2018, 5:50pm

Today I’ve updated my game to Google Market after long pause from previous update (Aug, 28) and got a lot of feedback from players. They said that game crashes on loading. I think it is going on loading process of previous save file (just guess, cause the some users said that reinstall the game or clear the game cache fix this issue). One user says me that new version has crash on his Phone, but works fine on the tablet. >__<. Of course I tested new build on my devices, computers and web version and had not this issue. Also I didn’t any changes in saving system from August update. (Use sys.load(sys.get_save_file(M.APP_NAME, M.SAVE_FILE_NAME)) for loading)

New build was made with editor 1.2.137.

Also Google console shows a lots of new crashes:

Most:

Second:

As you can see it’s new devices with new Android.

Any ideas?

My plan: try to generate build in 1.2.135 (August version of Defold) and make new update to Store.

AGulev · October 29, 2018, 6:28pm

The same issues.

Crash free version uses Defold 1.2.134
1.02 - Defold 1.2.138

Dragosha · October 29, 2018, 7:27pm

Just sent new apk direct to users with issue and seems that build from 1.2.139 works correctly (hope). The same code.

Dragosha · October 30, 2018, 5:46am

Made a hot fix update to Google Market this night. The build assembled in 1.2.139. Seems it’s more stable and users found it as workable. Also shows reducing of number of Crash report in Google console.

Well. What’s wrong with 1.2.137?

0.98 assembled in 1.2.137
0.981 in 1.2.139

ps. noticed that in 1.2.139 we have a new render of “text leading”. The same font size, but a bit more line spacing.

britzl · October 30, 2018, 6:10am

We did have a problem when closing the game and music was playing which caused a crash. This has since been fixed. Could it be this issue you’ve come across?

I think we’re releasing a hotfix today for this crash: Font crashes engine with OpenGL error - #10 by Mathias_Westerdahl

There shouldn’t be any change to how fonts are rendered, but the hotfix and crash might be related to what you are experiencing with fonts. @jhonny.goransson?

jhonny.goransson · October 30, 2018, 8:08am

Yeah the leading and OpenGL issue should be fixed with todays release and everything should render as normal!

Oleg_The_Evangelist · October 30, 2018, 8:32am

May I please steal this thread to remind everyone that we at Defold have little means (currently) to actively monitor ANRs and crashes on live games overall. So please, people, don’t be shy, report crashes, ANRs and just weird things with your games. It is of value to us!

AGulev · October 31, 2018, 3:13pm

This is still an issue in 1.2.140 =(

Dragosha · October 31, 2018, 3:24pm

yep, “tkgill” is most popular crash on Android

Reports 38 Last 7 days
Android 8.0	38	100.0%
Samsung Galaxy S7 edge (hero2lte), Android 8.0
Report 1

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> com.dragosha.hamster <<<

backtrace:
  #00  pc 000000000004b748  /system/lib/libc.so (tgkill+12)
  #01  pc 000000000001a4c3  /system/lib/libc.so (abort+54)
  #02  pc 000000000033a063  /system/lib/libart.so (_ZN3art7Runtime5AbortEPKc+370)
  #03  pc 000000000033a65b  /system/lib/libart.so (_ZN3art7Runtime7AborterEPKc+10)
  #04  pc 00000000003f2231  /system/lib/libart.so (_ZN7android4base10LogMessageD1Ev+456)
  #05  pc 00000000001c81eb  /system/lib/libart.so (_ZN3art22IndirectReferenceTable17AbortIfNoCheckJNIERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE+166)
  #06  pc 0000000000234e2f  /system/lib/libart.so (_ZNK3art22IndirectReferenceTable10GetCheckedEPv+266)
  #07  pc 000000000023162f  /system/lib/libart.so (_ZN3art9JavaVMExt12DecodeGlobalEPv+10)
  #08  pc 000000000035beab  /system/lib/libart.so (_ZNK3art6Thread13DecodeJObjectEP8_jobject+162)
  #09  pc 00000000003352d7  /system/lib/libart.so (_ZN3art35InvokeVirtualOrInterfaceWithVarArgsERKNS_33ScopedObjectAccessAlreadyRunnableEP8_jobjectP10_jmethodIDSt9__va_list+42)
  #10  pc 0000000000264d7d  /system/lib/libart.so (_ZN3art3JNI18CallBooleanMethodVEP7_JNIEnvP8_jobjectP10_jmethodIDSt9__va_list+436)
  #11  pc 000000000008536c  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #12  pc 000000000016a190  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #13  pc 000000000015e92c  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #14  pc 0000000000019844  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #15  pc 000000000001b794  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #16  pc 000000000001b808  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #17  pc 0000000000017d80  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #18  pc 0000000000017cd4  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #19  pc 000000000013e584  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #20  pc 0000000000017cfc  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #21  pc 000000000013ee5c  /data/app/com.dragosha.hamster-0Tyl0K2A6txIJuQnTfxD_g==/lib/arm/libLookYourLoot.so
  #22  pc 0000000000048a5f  /system/lib/libc.so (_ZL15__pthread_startPv+22)
  #23  pc 000000000001b333  /system/lib/libc.so (__start_thread+32)

next one called “abort”

|Android 8.1|21|84.0%|
|Android 9|4|16.0%|

Today, 18:37 on app version 57
Nokia Nokia 7 plus (B2N), Android 8.1
Report 2

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> com.dragosha.hamster <<<

backtrace:
  #00  pc 000000000001a71e  /system/lib/libc.so (abort+63)
  #01  pc 000000000035b469  /system/lib/libart.so (art::Runtime::Abort(char const*)+392)
  #02  pc 000000000041ae2d  /system/lib/libart.so (android::base::LogMessage::~LogMessage()+452)
  #03  pc 00000000001d55d1  /system/lib/libart.so (art::IndirectReferenceTable::AbortIfNoCheckJNI(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)+164)
  #04  pc 0000000000251d77  /system/lib/libart.so (art::IndirectReferenceTable::GetChecked(void*) const+262)
  #05  pc 000000000024e5fb  /system/lib/libart.so (art::JavaVMExt::DecodeGlobal(void*)+10)
  #06  pc 000000000037d429  /system/lib/libart.so (art::Thread::DecodeJObject(_jobject*) const+164)
  #07  pc 0000000000356ba3  /system/lib/libart.so (art::InvokeVirtualOrInterfaceWithVarArgs(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jmethodID*, std::__va_list)+42)
  #08  pc 00000000002830e5  /system/lib/libart.so (art::JNI::CallBooleanMethodV(_JNIEnv*, _jobject*, _jmethodID*, std::__va_list)+420)
  #09  pc 000000000008536c  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #10  pc 000000000016a190  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #11  pc 000000000015e92c  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #12  pc 0000000000019844  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #13  pc 000000000001b794  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #14  pc 000000000001b808  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #15  pc 0000000000017d80  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #16  pc 0000000000017cd4  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #17  pc 000000000013e584  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #18  pc 0000000000017cfc  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #19  pc 000000000013ee5c  /data/app/com.dragosha.hamster-ICOtM7gbtkMJ2u50heQZXg==/lib/arm/libLookYourLoot.so
  #20  pc 00000000000479ef  /system/lib/libc.so (__pthread_start(void*)+22)
  #21  pc 000000000001b2ed  /system/lib/libc.so (__start_thread+32)

another “tgkill”

Samsung Galaxy S6 (zerofltelgt), Android 7.0
Report 1

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> com.dragosha.hamster <<<

backtrace:
  #00  pc 000000000004a230  /system/lib/libc.so (tgkill+12)
  #01  pc 00000000000479c3  /system/lib/libc.so (pthread_kill+34)
  #02  pc 000000000001d9c5  /system/lib/libc.so (raise+10)
  #03  pc 0000000000019511  /system/lib/libc.so (__libc_android_abort+34)
  #04  pc 0000000000017150  /system/lib/libc.so (abort+4)
  #05  pc 000000000031bc5d  /system/lib/libart.so (_ZN3art7Runtime5AbortEv+252)
  #06  pc 00000000000b4d4b  /system/lib/libart.so (_ZN3art10LogMessageD2Ev+866)
  #07  pc 00000000001bc951  /system/lib/libart.so (_ZN3art22IndirectReferenceTable17AbortIfNoCheckJNIEv+84)
  #08  pc 000000000023dc5f  /system/lib/libart.so (_ZNK3art22IndirectReferenceTable10GetCheckedEPv+498)
  #09  pc 000000000023ae0f  /system/lib/libart.so (_ZN3art9JavaVMExt12DecodeGlobalEPv+10)
  #10  pc 00000000003359b5  /system/lib/libart.so (_ZNK3art6Thread13DecodeJObjectEP8_jobject+124)
  #11  pc 0000000000317ac7  /system/lib/libart.so (_ZN3art35InvokeVirtualOrInterfaceWithVarArgsERKNS_33ScopedObjectAccessAlreadyRunnableEP8_jobjectP10_jmethodIDSt9__va_list+42)
  #12  pc 0000000000266bb5  /system/lib/libart.so (_ZN3art3JNI18CallBooleanMethodVEP7_JNIEnvP8_jobjectP10_jmethodIDSt9__va_list+444)
  #13  pc 000000000008536c  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #14  pc 000000000016a190  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #15  pc 000000000015e92c  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #16  pc 0000000000019844  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #17  pc 000000000001b794  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #18  pc 000000000001b808  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #19  pc 0000000000017d80  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #20  pc 0000000000017cd4  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #21  pc 000000000013e584  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #22  pc 0000000000017cfc  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #23  pc 000000000013ee5c  /data/app/com.dragosha.hamster-1/lib/arm/libLookYourLoot.so
  #24  pc 0000000000047493  /system/lib/libc.so (_ZL15__pthread_startPv+22)
  #25  pc 0000000000019f6d  /system/lib/libc.so (__start_thread+6)

java.lang.IllegalArgumentException
com.dynamo.android.DefoldActivity.onCreate

Android 6.0	2	100.0%

Today, 13:44 on app version 57
ZTE BLADE A510 (P635A50), Android 6.0
Report 1 of 2

java.lang.RuntimeException: 
  at android.app.ActivityThread.performLaunchActivity (ActivityThread.java:2572)
  at android.app.ActivityThread.handleLaunchActivity (ActivityThread.java:2654)
  at android.app.ActivityThread.access$900 (ActivityThread.java:175)
  at android.app.ActivityThread$H.handleMessage (ActivityThread.java:1488)
  at android.os.Handler.dispatchMessage (Handler.java:111)
  at android.os.Looper.loop (Looper.java:207)
  at android.app.ActivityThread.main (ActivityThread.java:5728)
  at java.lang.reflect.Method.invoke (Native Method)
  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run (ZygoteInit.java:789)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:679)
Caused by: java.lang.IllegalArgumentException: 
  at android.app.NativeActivity.onCreate (NativeActivity.java:170)
  at com.dynamo.android.DefoldActivity.onCreate (DefoldActivity.java:205)
  at android.app.Activity.performCreate (Activity.java:6270)
  at android.app.Instrumentation.callActivityOnCreate (Instrumentation.java:1113)
  at android.app.ActivityThread.performLaunchActivity (ActivityThread.java:2519)

total:

Builded in 1.2.139

jhonny.goransson · October 31, 2018, 3:29pm

1.2.140 did not address these issues unfortunately, it was more of a hotfix for the font changes that was introduced in the last release.

Johan_Beck-Noren · October 31, 2018, 3:32pm

Let’s try and narrow this down and gather the information in one place.

What is the latest defold version that you perceive as being stable for you?
Are the crashes Android only or are you seeing crashes on iOS or other platforms as well?

Dragosha · October 31, 2018, 3:41pm

last stable is 135

This is Android only crashes because I didn’t update iOS version of the game.

Mathias_Westerdahl · October 31, 2018, 4:11pm

I could try to symbolicate the Defold symbols, if you give me your libLookYourLoot.so file.

Dragosha · October 31, 2018, 5:44pm

https://www.dropbox.com/s/4usiyy8l9ida8af/libLookYourLoot.so?dl=0

Dragosha · November 1, 2018, 6:06am

For comparison reason list of all crashes from previous release on Android (Aug 28, engine ver:1.2.135) for the last 60 days:

java.lang.NullPointerException
com.defold.iap.IapGooglePlayActivity.onCreate

Android 5.1	4	100.0%
TCT (Alcatel) 8050E (Pixi4-6_3G), Android 5.1
Report 1 of 4

java.lang.RuntimeException: 
  at android.app.ActivityThread.performLaunchActivity (ActivityThread.java:2521)
  at android.app.ActivityThread.handleLaunchActivity (ActivityThread.java:2601)
  at android.app.ActivityThread.access$800 (ActivityThread.java:178)
  at android.app.ActivityThread$H.handleMessage (ActivityThread.java:1470)
  at android.os.Handler.dispatchMessage (Handler.java:111)
  at android.os.Looper.loop (Looper.java:194)
  at android.app.ActivityThread.main (ActivityThread.java:5637)
  at java.lang.reflect.Method.invoke (Native Method)
  at java.lang.reflect.Method.invoke (Method.java:372)
  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run (ZygoteInit.java:960)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:755)
Caused by: java.lang.NullPointerException: 
  at com.defold.iap.IapGooglePlayActivity.onCreate (IapGooglePlayActivity.java:223)
  at android.app.Activity.performCreate (Activity.java:6100)
  at android.app.Instrumentation.callActivityOnCreate (Instrumentation.java:1112)
  at android.app.ActivityThread.performLaunchActivity (ActivityThread.java:2468)

Dragosha · November 1, 2018, 6:26am

1.2.135 ± stable
1.2.137 -
1.2.139 - hot fix, ± stable

Mathias_Westerdahl · November 1, 2018, 8:04am

Added DEF-3585 for this. It should land in 1.2.143 (in about two weeks)

Mathias_Westerdahl · November 1, 2018, 8:31am

As for the other callstacks (tgkill/abort), they all look mostly the same:

000000000008536c	Java_com_defold_iac_IACJNI_onInvocation
000000000016a190	Java_com_dynamo_android_DefoldActivity_nativeOnActivityResult

Now, as to why it fails there, I’m not sure yet. That code is quite small, and the edge cases seems taken care of.

Do you happen to have any logcat from such a crash? It might give a clue.

Insality · November 1, 2018, 9:00am

So, probably I have the same problems? On the Family Age project

My last builds was on 1.2.136, builded with Bob.
But got a more tgkill errors than usual then build after 26.10. Prev build was 18.10, but there is not too much errors. (It is strange, probably our fault. The players/install count not very different on this dates)

135 build have tgkill errors too

The tgkill error:

   #00  pc 0000000000041f50  /system/lib/libc.so (tgkill+12)
   #01  pc 0000000000040fa1  /system/lib/libc.so (pthread_kill+32)
   #02  pc 000000000001c88b  /system/lib/libc.so (raise+10)
   #03  pc 0000000000019a3d  /system/lib/libc.so (__libc_android_abort+34)
   #04  pc 00000000000175ec  /system/lib/libc.so (abort+4)
   #05  pc 000000000001b49f  /system/lib/libc.so (__libc_fatal+16)
   #06  pc 0000000000019ac5  /system/lib/libc.so (__assert2+20)
   #07  pc 0000000000049818  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #08  pc 000000000004aefc  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #09  pc 0000000000038824  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #10  pc 0000000000060444  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #11  pc 0000000000038824  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #12  pc 000000000001d588  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #13  pc 000000000008bd9c  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #14  pc 000000000001fb4c  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #15  pc 000000000001fbf0  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #16  pc 000000000001c168  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #17  pc 000000000001c0bc  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #18  pc 000000000015da94  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #19  pc 000000000001c0e4  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #20  pc 000000000015e36c  /data/app/com.test.familyage-2/lib/arm/libFamily_Age.so
   #21  pc 00000000000408a3  /system/lib/libc.so (_ZL15__pthread_startPv+30)
   #22  pc 000000000001a0c1  /system/lib/libc.so (__start_thread+6)

The abort error

  #00  pc 000000000001a518  /system/lib/libc.so (abort+63)
  #01  pc 000000000001a9e9  /system/lib/libc.so (__assert2+20)
  #02  pc 0000000000049818  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #03  pc 000000000004aefc  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #04  pc 0000000000038824  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #05  pc 0000000000060444  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #06  pc 0000000000038824  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #07  pc 000000000001d588  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #08  pc 000000000008bd9c  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #09  pc 000000000001fb4c  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #10  pc 000000000001fbf0  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #11  pc 000000000001c168  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #12  pc 000000000001c0bc  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #13  pc 000000000015da94  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #14  pc 000000000001c0e4  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #15  pc 000000000015e36c  /data/app/com.test.familyage-dFCVGRwuh2nZLUX9iKrGDw==/lib/arm/libFamily_Age.so
  #16  pc 00000000000487ef  /system/lib/libc.so (__pthread_start(void*)+22)
  #17  pc 000000000001b34d  /system/lib/libc.so (__start_thread+32)

libFamily_Age.so: https://drive.google.com/open?id=1yhaARkep0mFjPQNFzNU5ks6kMtlWra0e

Also I have about 1 game crash per day (logs, get from crash.* module).
If I can help with more information, write me on slack

Dragosha · November 1, 2018, 9:23am

nope
It’s all happened on users devices, so, I need a way to register this issues.

Also I found this topic:
Best way to handle crash reporting on device? - #2 by britzl

I use this module (Google Analytics) from @britzl, but there is no any mentions about crashes in GA console:

UPD: i didn’t enable this feature, arghhh

ga.get_default_tracker().enable_crash_reporting(true)