TF Lite (mobile)
TensorFlow Lite converts a TF model into a flat .tflite file optimised for on-device inference: mobile, embedded, microcontrollers. Conversion can quantise weights and activations (float16, int8) for 2–4x size shrink and matching speed gains on supported hardware. The interpreter is tiny (~1 MB) and runs without the full TF runtime.
Convert, quantise, and run a tiny image classifier
EXAMPLE
import tensorflow as tf
import numpy as np
# 1) Train (or load) a Keras model
model = tf.keras.Sequential([
tf.keras.layers.Input(shape=(28, 28, 1)),
tf.keras.layers.Conv2D(8, 3, activation='relu'),
tf.keras.layers.MaxPooling2D(),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(10),
])
model.compile(optimizer='adam',
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
(x_train, y_train), _ = tf.keras.datasets.mnist.load_data()
x_train = x_train[..., None].astype('float32') / 255.0
model.fit(x_train, y_train, epochs=1, batch_size=128, verbose=0)
# 2) Convert to TF Lite with float16 weights — half the size, near-zero loss
conv = tf.lite.TFLiteConverter.from_keras_model(model)
conv.optimizations = [tf.lite.Optimize.DEFAULT]
conv.target_spec.supported_types = [tf.float16]
tflite_model = conv.convert()
with open('mnist_fp16.tflite', 'wb') as f: f.write(tflite_model)
# 3) Full int8 quantisation needs a representative dataset
def representative():
for i in range(100):
yield [x_train[i:i+1]]
conv = tf.lite.TFLiteConverter.from_keras_model(model)
conv.optimizations = [tf.lite.Optimize.DEFAULT]
conv.representative_dataset = representative
conv.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
conv.inference_input_type = tf.int8
conv.inference_output_type = tf.int8
int8_model = conv.convert()
with open('mnist_int8.tflite', 'wb') as f: f.write(int8_model)
# 4) Inference with the Python interpreter (same API as on-device)
itp = tf.lite.Interpreter(model_path='mnist_fp16.tflite')
itp.allocate_tensors()
i_in = itp.get_input_details()[0]
i_out = itp.get_output_details()[0]
sample = x_train[:1]
itp.set_tensor(i_in['index'], sample.astype(np.float32))
itp.invoke()
logits = itp.get_tensor(i_out['index'])
print('predicted class:', int(np.argmax(logits)))
# Size comparison
import os
for f in ('mnist_fp16.tflite', 'mnist_int8.tflite'):
print(f, os.path.getsize(f) / 1024, 'KB')
Why it matters
Always benchmark with the representative dataset you actually serve, not the training set. Quantisation error is concentrated on inputs whose value distribution differs from the calibration set — a model that hits 98% in your notebook can crater to 70% on phone-camera photos if you calibrate on clean MNIST.
Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.
Example
Example
import tensorflow as tf
converter = tf.lite.TFLiteConverter.from_keras_model(model)
open('model.tflite', 'wb').write(converter.convert())
Try it Yourself »
Discussion
Loading…