Convert tflite model of float32 model to int8

Question

Is it possible to convert a tflite float32 to tflite model of uint8? I read this page https://www.tensorflow.org/lite/performance/post_training_quantization, but they are only loading Keras of Tensorflow models.

Answer 1

You cannot make changes to the architecture by loading a serialized tflite binary.

To quantize the model, you need the model file in one of the formats accepted by TFLiteConverter. You can check the following page for alternative formats. https://www.tensorflow.org/api_docs/python/tf/lite/TFLiteConverter

Answer 2

It is possible. When we convert a TF model into TFlite float32, the weights are not quantized. Hence we can extract those weights and initialise them into a TF model. If the TFlite model is in int8 or uint8 quantization, it is different story.

Start by converting the tflite float32 model to the corresponding TensorFlow model. Use this repo - https://github.com/PINTO0309/tflite2tensorflow

Then you have to convert the float32 tensorflow model to int8. Ideally, you would need the dataset to do the post-training quantization. But you can also use a dummy dataset with similar data distribution. If you insist on converting into uint8, you have to use an older version of TensorFlow. TensorFlow doesn't support uint8 conversions now.