Application of tflite models and how to convert them.

[WuSiangRu]

Hi, I encounter the following error when using a tflite model that has been converted and quantized by toco.

Process: com.ml.quaterion.facenetdetection, PID: 4034
java.lang.IllegalArgumentException: Cannot copy to a TensorFlowLite tensor (input) with 76800 bytes from a Java Buffer with 307200 bytes.
at org.tensorflow.lite.TensorImpl.throwIfSrcShapeIsIncompatible(TensorImpl.java:416)
at org.tensorflow.lite.TensorImpl.setTo(TensorImpl.java:140)
at org.tensorflow.lite.NativeInterpreterWrapper.run(NativeInterpreterWrapper.java:243)
at org.tensorflow.lite.InterpreterImpl.runForMultipleInputsOutputs(InterpreterImpl.java:107)
at org.tensorflow.lite.Interpreter.runForMultipleInputsOutputs(Interpreter.java:80)
at org.tensorflow.lite.InterpreterImpl.run(InterpreterImpl.java:100)
at org.tensorflow.lite.Interpreter.run(Interpreter.java:80)
at com.ml.quaterion.facenetdetection.model.FaceNetModel.runFaceNet(FaceNetModel.kt:104)
at com.ml.quaterion.facenetdetection.model.FaceNetModel.getFaceEmbedding(FaceNetModel.kt:83)
at com.ml.quaterion.facenetdetection.FileReader$getEmbedding$2.invokeSuspend(FileReader.kt:108)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664) 

The model is converted from facenet pretrained model to tflite by QUANTIZED_UINT8 format, the model size is reduced from 93MB to about 23MB, the problem should be caused by the mismatch between UINT8 and FLOAT, I would like to ask how to convert the tflite model in the assets directory of the project and reduce the model size while maintaining the FLOAT format.

[shubham0204]

@WuSiangRu You can convert to quantized integer formats, and TensorFlow preserves the float data-type of input and output tensors. Refer these docs. It says,

Now all weights and variable data are quantized, and the model is significantly smaller compared to the original TensorFlow Lite model. However, to maintain compatibility with applications that traditionally use float model input and output tensors, the TensorFlow Lite Converter leaves the model input and output tensors in float

Considering your error, you would have to provide a ByteBuffer where each pixel is encoded as an uint8. In your code, you're providing 307200 sized buffer (probably in float32) whereas the model needs 76800 = 307200 / 4 sized buffer (probably uint8)

[WuSiangRu]

Hi ,thanks for your reply,sorry for not asking you which version of tensorflow you were using at the beginning. I'm using version 1.15, and the method mentioned in the link you provided seems to require 2.0 or higher.
So I referred to this method and converted the model to keras and then to tfllite in FLOAT32 format and it works.

Also I would like to ask you how to add the ByteBuffer mentioned in the above suggestion to the program code?