iwantcoding.com
🔥 Daily 👥 Rooms 🏆 Top Log in Sign up

TensorBoard

By · Reviewed & last updated · Part of TensorFlow Tutorial

TensorBoard is the visualisation tool that turns training runs into graphs, histograms, distributions, embeddings, profiler traces, and HParam tables. It is the cheapest way to see what is happening inside training — pin it open in a tab during runs and you will catch divergence, dead activations, and overfitting in minutes.

Log scalars, images, histograms, and the HParam dashboard

EXAMPLE
import tensorflow as tf
import datetime
import numpy as np
from tensorboard.plugins.hparams import api as hp

# 1) Set up unique log directories per run (timestamped)
log_root = 'logs/fit'
run_dir  = f'{log_root}/{datetime.datetime.now().strftime("%Y%m%d-%H%M%S")}'

# 2) Standard scalar + histogram logging via TensorBoard callback
cb_tb = tf.keras.callbacks.TensorBoard(
    log_dir=run_dir,
    histogram_freq=1,     # weight histograms every epoch
    write_graph=True,
    update_freq='epoch',
    profile_batch=(10, 20),    # capture a profiler trace for batches 10..20
)

# Toy model + data
(x_tr, y_tr), (x_te, y_te) = tf.keras.datasets.mnist.load_data()
x_tr = x_tr[..., None] / 255.0
x_te = x_te[..., None] / 255.0

model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(28, 28, 1)),
    tf.keras.layers.Conv2D(8, 3, activation='relu'),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(10),
])
model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])

# 3) Train with the callback — scalars and graph appear in TensorBoard
model.fit(x_tr, y_tr, validation_data=(x_te, y_te), epochs=3, batch_size=128, callbacks=[cb_tb])

# 4) Log custom things from your own training loop or hooks
writer = tf.summary.create_file_writer(run_dir + '/custom')
with writer.as_default():
    tf.summary.scalar('lr', model.optimizer.learning_rate.numpy(), step=0)
    tf.summary.image('sample_inputs', x_tr[:8], step=0, max_outputs=8)
    tf.summary.histogram('layer_kernel', model.layers[0].kernel, step=0)
    tf.summary.text('notes', 'baseline run; LR fixed at 1e-3', step=0)

# 5) HParam dashboard — sweep small hyperparameter grids and compare
HP_LR    = hp.HParam('lr',         hp.Discrete([1e-3, 5e-4, 1e-4]))
HP_HSIZE = hp.HParam('hidden',     hp.Discrete([32, 64]))
HP_OPT   = hp.HParam('optimizer',  hp.Discrete(['adam', 'sgd']))

with tf.summary.create_file_writer('logs/hparam').as_default():
    hp.hparams_config(
        hparams=[HP_LR, HP_HSIZE, HP_OPT],
        metrics=[hp.Metric('val_accuracy', display_name='val_acc')],
    )

def train_run(lr, hidden, opt):
    m = tf.keras.Sequential([
        tf.keras.layers.Input(shape=(28, 28, 1)),
        tf.keras.layers.Flatten(),
        tf.keras.layers.Dense(hidden, activation='relu'),
        tf.keras.layers.Dense(10),
    ])
    m.compile(optimizer=(tf.keras.optimizers.Adam(lr) if opt == 'adam'
                          else tf.keras.optimizers.SGD(lr)),
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])
    h = m.fit(x_tr, y_tr, validation_data=(x_te, y_te), epochs=1, batch_size=256, verbose=0)
    return h.history['val_accuracy'][-1]

session_count = 0
for lr in HP_LR.domain.values:
    for hsize in HP_HSIZE.domain.values:
        for opt in HP_OPT.domain.values:
            run_id = f'run-{session_count:02d}'
            with tf.summary.create_file_writer('logs/hparam/' + run_id).as_default():
                hp.hparams({HP_LR: lr, HP_HSIZE: hsize, HP_OPT: opt})
                acc = train_run(lr, hsize, opt)
                tf.summary.scalar('val_accuracy', acc, step=1)
            session_count += 1

# 6) Launch the UI
# tensorboard --logdir logs --bind_all
# Open http://localhost:6006

Why it matters

Use the HParam dashboard for any non-trivial sweep — it gives you parallel-coordinates and scatter plots that make trade-offs (e.g. LR vs hidden size on val_acc vs train_time) immediately visible, instead of comparing scalar tabs by hand.

Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.

Example

Example
%load_ext tensorboard
%tensorboard --logdir logs/
Try it Yourself »

Discussion

Loading…