DefTensor – TensorFlow Lite Native Extension

Dev diaries
#1

I started to work on a Native Extension for TensorFlow Lite, so you can use neural networks in Defold games. The obvious framework pick was TensorFlow Lite, because it’s targeted to mobile devices.

The demo project has pre-trained model for MNIST and it allows you to draw digits and recognize them.
I took the trained model from this project: https://github.com/nex3z/tfmobile-mnist-android

Here’s the current development status:

The extension works only on macOS. I couldn’t make it work on iOS, because the provided TFLite framework is compiled for iOS 9.0, but Defold requires iOS 8.0. I was really surprised when iOS framework worked on macOS!

I tried to compile TFLite myself for iOS 8.0, but the library I got doesn’t work. It seems to load a model fine, but you can’t run it – the app crashes. I tried the library in both iOS native app and in Defold NE. I thought that TFLite really can’t work on iOS 8.0 and re-build the library for iOS 9.0, but it also doesn’t work in the native app, unfortunately. It seems that something is wrong with my build environment.

I’ll try to continue experiments and finally make NE working on iOS and Android.

18 Likes
How to change minimum iOS version? (DEF-2598)
#2

Interesting! Please keep us updated of any new developments.

2 Likes
#3

so cool! I hope our community figures out some cool use cases immediately =]

2 Likes
#4

I need to make it work on iOS and Android. Just macOS is not enough :slight_smile:

2 Likes
#5

well, it makes sense to think about the use cases already :wink:

Like ways to help generate or playtest levels for exmple…

#6

Or train some AI for board games.

#7

As I understand on iOS it blocked because of DEF-2598 issue?

#8

Yes, exactly. If I can raise the minimum iOS version to 9.0, then iOS support will appear immediately. I’ll still try to build working TFLite from source myself. I’m able to build the library using the provided makefile, but the problem is that it doesn’t work, doesn’t matter if I build for 9.0 or 8.0. Maybe something is wrong with my build environment, I continue to search for answers. But if I fail in the end, then yes — DEF-2598 is the show stopper for DefTensor on iOS :frowning:

My next step is to make it work on Android. I think that even with Android only it can be useful for Defold community.

1 Like
#9

I’m still struggling with building iOS library myself (no matter, if it’s iOS 9 or iOS 8). I tried updating TensorFlow, because I found a mention of a similar problem in TFLite Github issues, but it didn’t help, unfortunately. Even worse problems with Android library. I managed to build what seems to be a shared C++ library for Android, but when I try to build the bundle in Defold, I get unresolved symbol error. It looks like this library doesn’t export necessary symbols. I’ll continue my experiments, of course, but I don’t have any ETA on this, because I have limited time.

7 Likes