iwantcoding.com
🔥 Daily 👥 Rooms 🏆 Top Log in Sign up

Metrics

By · Reviewed & last updated · Part of TensorFlow Tutorial

Keras metrics measure how the model is doing during training and evaluation. Pick metrics for the task: accuracy / AUC / F1 for classification, MAE / MSE / R² for regression, BLEU / perplexity for NLP.

Built-ins + a custom metric

EXAMPLE
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers, metrics

# 1) Classification — multi-class
model = keras.Sequential([
    layers.Dense(64, activation='relu', input_shape=(10,)),
    layers.Dense(3,  activation='softmax'),
])
model.compile(
    optimizer='adam',
    loss='sparse_categorical_crossentropy',
    metrics=[
        metrics.SparseCategoricalAccuracy(name='acc'),
        metrics.SparseTopKCategoricalAccuracy(k=2, name='top2'),
    ],
)

# 2) Binary classification — go beyond accuracy
bin_model = keras.Sequential([layers.Dense(32, activation='relu'), layers.Dense(1, activation='sigmoid')])
bin_model.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=[
        metrics.BinaryAccuracy(name='acc'),
        metrics.Precision(name='prec'),
        metrics.Recall(name='rec'),
        metrics.AUC(name='auc', curve='ROC'),
        metrics.AUC(name='pr-auc', curve='PR'),
    ],
)

# 3) Regression
reg_model = keras.Sequential([layers.Dense(16, activation='relu'), layers.Dense(1)])
reg_model.compile(
    optimizer='adam',
    loss='mse',
    metrics=[
        metrics.MeanAbsoluteError(),
        metrics.MeanAbsolutePercentageError(),
        metrics.RootMeanSquaredError(),
    ],
)

# 4) Multi-label — per-class metrics
multi = keras.Sequential([layers.Dense(64, activation='relu'), layers.Dense(5, activation='sigmoid')])
multi.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=[metrics.BinaryAccuracy(threshold=0.5)],
)

# 5) Custom metric — keep state with subclassing
class F1(metrics.Metric):
    def __init__(self, threshold=0.5, name='f1', **kwargs):
        super().__init__(name=name, **kwargs)
        self.precision = metrics.Precision(thresholds=threshold)
        self.recall    = metrics.Recall(thresholds=threshold)

    def update_state(self, y_true, y_pred, sample_weight=None):
        self.precision.update_state(y_true, y_pred, sample_weight)
        self.recall.update_state(y_true, y_pred, sample_weight)

    def result(self):
        p, r = self.precision.result(), self.recall.result()
        return 2 * p * r / (p + r + 1e-7)

    def reset_state(self):
        self.precision.reset_state(); self.recall.reset_state()

bin_model.compile(
    optimizer='adam', loss='binary_crossentropy',
    metrics=[metrics.BinaryAccuracy(), F1(threshold=0.5)],
)

# 6) Watch metrics with TensorBoard
tb = keras.callbacks.TensorBoard(log_dir='./logs', histogram_freq=1)
# Then in training:
# model.fit(x, y, validation_split=0.2, epochs=20, callbacks=[tb])

# 7) Evaluate independently
results = model.evaluate(X_test, y_test, return_dict=True)
print(results)
# {'loss': 0.32, 'acc': 0.91, 'top2': 0.97}

Why it matters

Accuracy alone hides imbalanced datasets. Track precision, recall, and PR-AUC for binary classifiers — especially when one class is 1% of the data. The right metric prevents you shipping a useless model.

Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.

Example

Example
from tensorflow.keras import metrics
metrics.SparseCategoricalAccuracy()
metrics.AUC()
metrics.Precision()
metrics.Recall()
Try it Yourself »

Discussion

Loading…