Every model team we work with eventually hits the same wall: the architecture is fine, the pipeline is fine, and the model is still mediocre because there are 2,000 labels when the task needs 20,000. The instinct is to send the whole unlabelled pool to an annotation vendor and wait a month. That is usually the most expensive possible way to buy accuracy.
Active learning is the cheaper path. Instead of labelling randomly, you train on what you have, ask the model which unlabelled examples it is least sure about, label those, and repeat. On the classification and detection projects we run, a well-tuned active learning loop reaches a target accuracy with somewhere between a third and two thirds of the labels random sampling would need. This tutorial builds that loop on TensorFlow and Keras 3, with the parts people usually get wrong called out explicitly.
The loop
seed set -> train -> score unlabelled pool -> select batch -> label -> merge -> retrain
Four things determine whether it works: the acquisition function (how you score candidates), the batch size per round, how you handle class imbalance and duplicates, and whether your evaluation set is fixed and random. That last one is not optional. If you evaluate on data selected by the same uncertainty rule you used to label, your numbers are fiction.
Setup and the seed set
pip install -U tensorflow keras
Start with a random seed set — a few hundred to a few thousand examples, stratified if you already know the label distribution. Active learning needs a model that is better than noise before its uncertainty means anything; cold-starting the loop on 20 labels mostly selects outliers and corrupt files.
import keras, numpy as np, tensorflow as tf
rng = np.random.default_rng(0)
pool_idx = np.arange(len(X_unlabelled))
seed_idx = rng.choice(pool_idx, size=1000, replace=False)
labelled_idx = set(seed_idx.tolist())
Carve out a held-out test set by pure random sampling, once, before the loop starts, and never let the loop touch it. Keep a random validation split too, for early stopping.
A model that reports honest uncertainty
Softmax outputs from a network trained to convergence are famously overconfident. Two cheap fixes make the scores usable.
Label smoothing plus temperature calibration for the pointwise case:
def build_model(n_classes):
inputs = keras.Input(shape=(224, 224, 3))
x = keras.applications.efficientnet_v2.preprocess_input(inputs)
backbone = keras.applications.EfficientNetV2S(include_top=False, pooling="avg")
x = backbone(x)
x = keras.layers.Dropout(0.3)(x) # kept active at inference for MC dropout
outputs = keras.layers.Dense(n_classes, activation="softmax")(x)
model = keras.Model(inputs, outputs)
model.compile(
optimizer=keras.optimizers.AdamW(1e-3),
loss=keras.losses.CategoricalCrossentropy(label_smoothing=0.05),
metrics=["accuracy"],
)
return model
MC dropout for a poor-man's ensemble: run the same input several times with dropout enabled and look at the spread of predictions.
@tf.function
def mc_predict(model, batch, passes=10):
preds = tf.stack([model(batch, training=True) for _ in range(passes)]) # (T, B, C)
return tf.reduce_mean(preds, axis=0), preds
Ten passes is usually enough. If you can afford it, a true ensemble of 3–5 independently seeded models gives better-calibrated disagreement than MC dropout, and you can reuse the ensemble at serving time.
Acquisition functions
Three that earn their keep, all computed from the mean predictive distribution p of shape (B, C):
EPS = 1e-9
def least_confidence(p):
return 1.0 - tf.reduce_max(p, axis=-1)
def margin(p):
top2 = tf.math.top_k(p, k=2).values
return -(top2[:, 0] - top2[:, 1]) # smaller margin = more informative
def entropy(p):
return -tf.reduce_sum(p * tf.math.log(p + EPS), axis=-1)
And the one that specifically targets model uncertainty rather than inherent label noise — BALD, the mutual information between prediction and model parameters:
def bald(preds): # preds: (T, B, C) from mc_predict
p_mean = tf.reduce_mean(preds, axis=0)
total = -tf.reduce_sum(p_mean * tf.math.log(p_mean + EPS), axis=-1)
expected = tf.reduce_mean(
-tf.reduce_sum(preds * tf.math.log(preds + EPS), axis=-1), axis=0
)
return total - expected
Which to use: margin is the reliable default for multi-class classification and costs one forward pass. Entropy behaves similarly but over-selects classes with many confusable neighbours. BALD is the right choice when a chunk of your data is genuinely ambiguous — blurry images, honest disagreement between annotators — because it will not keep buying labels for examples no annotator can resolve.
Batch selection: don't just take the top-k
Take the 500 highest-entropy examples and you will typically get 500 near-duplicates of the same hard situation — the same production line at night, the same document template. You pay for 500 labels and learn one thing.
Fix it by combining uncertainty with diversity. A practical approach: take the top ~5× your batch budget by uncertainty, embed them with the model's penultimate layer, and cluster.
embedder = keras.Model(model.inputs, model.layers[-3].output) # pooled features
def select_batch(cand_idx, scores, embeddings, k):
top = cand_idx[np.argsort(-scores)[: 5 * k]]
emb = embeddings[top]
emb = emb / (np.linalg.norm(emb, axis=1, keepdims=True) + 1e-9)
# greedy k-center: repeatedly take the candidate farthest from what's chosen
chosen = [0]
dist = 1.0 - emb @ emb[0]
for _ in range(k - 1):
nxt = int(np.argmax(dist))
chosen.append(nxt)
dist = np.minimum(dist, 1.0 - emb @ emb[nxt])
return top[chosen]
That is core-set selection in miniature, and it is the single change that most often turns a disappointing active learning experiment into a working one.
Two more guards worth adding before anything reaches an annotator:
- Near-duplicate filter. Drop candidates whose cosine similarity to an already-labelled example exceeds ~0.98.
- Sanity filter. Corrupt, blank, or out-of-domain files score as maximally uncertain. Screen them with a cheap heuristic or you will pay annotators to label garbage.
Scoring the pool efficiently
The pool is usually far bigger than the labelled set, and you score all of it every round. Keep it in tf.data, batch it hard, and cap the work by subsampling the pool when it is huge — scoring a random 100k slice per round loses almost nothing.
pool_ds = (
tf.data.Dataset.from_tensor_slices(pool_paths)
.map(load_image, num_parallel_calls=tf.data.AUTOTUNE)
.batch(256)
.prefetch(tf.data.AUTOTUNE)
)
scores, embs = [], []
for batch in pool_ds:
p_mean, preds = mc_predict(model, batch, passes=10)
scores.append(bald(preds).numpy())
embs.append(embedder(batch, training=False).numpy())
scores = np.concatenate(scores); embs = np.concatenate(embs)
If this stage dominates your round time, the fixes are the usual ones from our tf.data pipelines tutorial: cache decoded inputs, use mixed precision for inference, and drop the MC passes to 5.
Retraining each round
Retrain from the pretrained initialisation each round, not from the previous round's weights. Warm-starting is faster but compounds the bias of early rounds and makes round-to-round accuracy comparisons meaningless. If budget forces warm starts, at least reset the classifier head and re-run early stopping.
Keep the same hyperparameters, seeds, and epoch budget across rounds so that the learning curve reflects the data, not tuning drift. Log every round:
history = {"round": r, "n_labels": len(labelled_idx),
"test_acc": float(test_acc), "test_macro_f1": float(f1),
"selected": selected.tolist(), "acquisition": "bald+kcenter"}
Measuring whether it actually worked
The only honest evaluation is a label-efficiency curve: test metric on the y-axis, number of labels on the x-axis, with your active learning run plotted against a random-sampling baseline run under identical conditions. Report the horizontal gap — "we hit 92% macro-F1 at 6k labels instead of 14k" — because that is the number in dollars.
Run both arms with at least three seeds. Active learning curves are noisy, and a single seed will happily show a 4-point "gain" that is variance.
Be prepared for the baseline to win sometimes. Active learning underperforms random sampling when the pool is small relative to the target label count, when labels are very noisy, or when the model is too weak to rank uncertainty meaningfully. Finding that out after two rounds on 3k labels is a cheap experiment; finding it out after buying 50k labels is not.
Where the loop meets production
Active learning is not a one-off project. In the systems we build, it runs continuously beside the serving stack:
- Production inference logs predictions and uncertainty scores (the same acquisition function, computed at serving time).
- A nightly job pulls the highest-scoring, most-diverse slice into the annotation queue.
- New labels land in the training set; TFX retrains, evaluates against the blessed model, and only promotes on improvement.
- TensorFlow Data Validation watches for drift, which is your signal to raise the labelling budget for a while.
That closes the loop: the things the model is worst at in production are exactly the things it gets taught next. It also gives you an answer to the question every CFO asks — "why do we keep paying for annotation?" — with a curve rather than a shrug.
Gotchas we see repeatedly
- Evaluating on actively-selected data. Your test set must stay random and frozen. This is the number one error.
- Uncalibrated confidence. Check reliability on the validation set; if confidence is wildly off, fit a temperature scalar before using confidence-based acquisition.
- Ignoring class imbalance. Uncertainty sampling drifts toward rare classes, which is often good — but verify with macro-F1, not accuracy.
- No annotator agreement measurement. If two annotators disagree 20% of the time, no acquisition function will save you; fix the guidelines first.
- Batches that are too large. One 10k-label round is barely active learning. Smaller, more frequent rounds extract more value per label, bounded by how fast your annotation vendor can turn around.
Want help scoping it?
The engineering here is a week or two of work. The judgement — how big the seed set should be, which acquisition function fits your noise profile, when random sampling is genuinely the right answer — is what takes the experience. Our TensorFlow consultants have built these loops for vision, document, and tabular classification teams. If you are staring at an annotation quote with too many zeros in it, get in touch and we will sanity-check the plan before you sign it.