Nearly every operations team we work with has the same first ML question: what does next week look like? Demand, load, headcount, cash, sensor readings, ticket volume - the shape of the problem is always a time series, and the shape of the failure is always the same too. Someone trains an LSTM on a random 80/20 split, reports a spectacular validation score, and then the model falls apart in production because the split leaked the future into the past.
This tutorial is the forecasting playbook we use on client engagements: build the windowed dataset correctly with tf.data, establish baselines you have to beat, train a few Keras 3 architectures worth trying, evaluate with backtesting instead of a random split, and produce forecasts with uncertainty attached rather than a single deceptively precise number.
None of our other tutorials cover this ground - tf.data Pipelines That Keep Your GPU Busy is about throughput, and Tabular ML on TensorFlow treats rows as independent. Time series are neither.
1. Frame the problem before you write a model
Write these four things down before any code. Every forecasting project we have had to rescue got at least one of them wrong.
- Input width - how much history the model sees (e.g. 168 hourly steps = one week).
- Horizon - how far ahead you predict (e.g. the next 24 hours).
- Cadence - how often the model runs in production, and how stale the newest feature is at that moment.
- Loss - what an error actually costs. Over-forecasting inventory and under-forecasting it are rarely equally painful.
The cadence question is the sneaky one. If yesterday's actuals only land in the warehouse at 11:00 today, then a model trained with data up to t-1 is not deployable. Build the lag into the dataset from day one.
2. Split by time, then normalize
import numpy as np, pandas as pd, tensorflow as tf, keras
df = pd.read_parquet("hourly.parquet").sort_values("timestamp")
n = len(df)
train_df = df[: int(n * 0.7)]
val_df = df[int(n * 0.7) : int(n * 0.85)]
test_df = df[int(n * 0.85) :]
Chronological, never shuffled. And compute normalization statistics on the training slice only:
feature_cols = ["load", "temp_c", "is_holiday"]
mean = train_df[feature_cols].mean()
std = train_df[feature_cols].std().replace(0, 1.0)
train_a = ((train_df[feature_cols] - mean) / std).to_numpy("float32")
val_a = ((val_df[feature_cols] - mean) / std).to_numpy("float32")
test_a = ((test_df[feature_cols] - mean) / std).to_numpy("float32")
Using global statistics is the most common leak we find in client notebooks, and it is invisible: the model just looks a little better than it will ever be again.
3. Encode time as periodic features
Raw hour-of-day and month integers imply that hour 23 and hour 0 are maximally distant. Sine/cosine pairs fix that:
def add_time_features(frame):
ts = pd.to_datetime(frame["timestamp"])
secs = ts.map(pd.Timestamp.timestamp).to_numpy()
day, year = 24 * 3600, 365.2425 * 24 * 3600
frame["day_sin"] = np.sin(secs * (2 * np.pi / day))
frame["day_cos"] = np.cos(secs * (2 * np.pi / day))
frame["year_sin"] = np.sin(secs * (2 * np.pi / year))
frame["year_cos"] = np.cos(secs * (2 * np.pi / year))
return frame
For most business series, calendar features plus a handful of lags recover a surprising fraction of the achievable accuracy. Add holidays and known promotions as explicit columns - the model cannot infer Black Friday from a sine wave.
4. Build windows with tf.data
keras.utils.timeseries_dataset_from_array handles the sliding window, but it gives you inputs and targets from the same offsets, so you build the two arrays yourself and zip them:
INPUT_WIDTH, HORIZON = 168, 24
TARGET = feature_cols.index("load")
def make_ds(arr, batch_size=64, shuffle=False):
inputs = arr[:-HORIZON]
targets = arr[INPUT_WIDTH:, TARGET] # aligned to window end
ds = keras.utils.timeseries_dataset_from_array(
data=inputs,
targets=None,
sequence_length=INPUT_WIDTH,
sequence_stride=1,
batch_size=None,
)
tgt = keras.utils.timeseries_dataset_from_array(
data=targets,
targets=None,
sequence_length=HORIZON,
sequence_stride=1,
batch_size=None,
)
ds = tf.data.Dataset.zip((ds, tgt))
if shuffle:
ds = ds.shuffle(4096)
return ds.batch(batch_size).prefetch(tf.data.AUTOTUNE)
train_ds = make_ds(train_a, shuffle=True)
val_ds = make_ds(val_a)
test_ds = make_ds(test_a)
Shuffling windows is fine and helps optimization; shuffling rows destroys the series. Always print one batch and eyeball the alignment before training - for x, y in train_ds.take(1): print(x.shape, y.shape) should give (64, 168, n_features) (64, 24).
If train, validation, and test are contiguous slices, windows that straddle a boundary will mix two regimes. Drop INPUT_WIDTH + HORIZON rows at the start of each non-training slice, or accept a small amount of contamination knowingly.
5. Baselines you must beat
class Persistence(keras.Model):
"""Repeat the last observed value for the whole horizon."""
def call(self, x):
return tf.repeat(x[:, -1:, TARGET], HORIZON, axis=1)
class SeasonalNaive(keras.Model):
"""Repeat the same hours from 24 steps ago."""
def call(self, x):
return x[:, -24:, TARGET][:, :HORIZON]
Score both with the same metric your neural net will use. On strongly seasonal data, seasonal-naive is brutally hard to beat, and knowing that number is what keeps a project honest. We have closed engagements by telling a client their seasonal-naive baseline was within 2% of the deep model and the deep model was not worth operating - that conversation buys more trust than a marginal accuracy win.
6. Three models worth trying, in order
Linear. One dense layer over the flattened window. Fast, interpretable, and often within a few percent of anything fancier.
linear = keras.Sequential([
keras.layers.Flatten(),
keras.layers.Dense(HORIZON),
])
Dilated 1-D CNN. Cheap to train, parallel, and covers long receptive fields with few parameters. This is our default recommendation for hourly business data.
cnn = keras.Sequential([
keras.layers.Conv1D(64, 3, padding="causal", dilation_rate=1, activation="relu"),
keras.layers.Conv1D(64, 3, padding="causal", dilation_rate=2, activation="relu"),
keras.layers.Conv1D(64, 3, padding="causal", dilation_rate=4, activation="relu"),
keras.layers.Conv1D(64, 3, padding="causal", dilation_rate=8, activation="relu"),
keras.layers.Lambda(lambda t: t[:, -1, :]),
keras.layers.Dense(HORIZON),
])
padding="causal" is mandatory: ordinary "same" padding lets a convolution see future timesteps, which is leakage dressed up as architecture.
Attention / LSTM. A small transformer encoder block or a stacked LSTM helps when the series has long-range dependencies or many correlated covariates. It also costs the most to train and to operate. Reach for it only after the CNN plateaus.
Compile and train all of them identically so the comparison means something:
model.compile(
optimizer=keras.optimizers.AdamW(1e-3),
loss=keras.losses.Huber(),
metrics=[keras.metrics.MeanAbsoluteError()],
)
model.fit(
train_ds, validation_data=val_ds, epochs=50,
callbacks=[keras.callbacks.EarlyStopping(patience=5, restore_best_weights=True)],
)
Huber loss instead of MSE is a small change with a large effect on series that contain outage spikes: MSE lets one bad hour dominate the gradient.
7. Forecast a range, not a point
A single number invites false confidence. Quantile regression gives you an interval for almost no extra work - predict several quantiles at once with a pinball loss:
QUANTILES = [0.1, 0.5, 0.9]
def pinball_loss(y_true, y_pred):
y_true = tf.expand_dims(y_true, -1) # (b, H, 1)
q = tf.constant(QUANTILES, dtype=y_pred.dtype)
e = y_true - y_pred # (b, H, Q)
return tf.reduce_mean(tf.maximum(q * e, (q - 1.0) * e))
head = keras.layers.Dense(HORIZON * len(QUANTILES))
# reshape output to (batch, HORIZON, len(QUANTILES)) and compile with pinball_loss
The 0.5 quantile is your forecast; 0.1 and 0.9 are the band planners actually use. Check calibration on the test period: roughly 80% of actuals should fall inside the 10-90 band. If 98% do, your intervals are useless padding; if 50% do, you are about to under-stock.
8. Backtest, don't validate once
A single held-out block tells you how the model did in one arbitrary period. Rolling-origin backtesting tells you how it behaves across regimes:
def backtest(df, n_folds=6, horizon=24):
fold = len(df) // (n_folds + 1)
for k in range(1, n_folds + 1):
train = df.iloc[: fold * k]
test = df.iloc[fold * k : fold * k + fold]
yield train, test
Retrain (or at least re-evaluate) on each fold and report the distribution of errors, not the mean alone. Report MAE and MASE rather than MAPE: MAPE explodes near zero values and quietly rewards under-forecasting. MASE - error divided by the seasonal-naive error - is scale-free and directly answers the only question the client has, which is "is this better than the obvious thing?"
Also break error down by horizon step. A model that is excellent at t+1 and worthless at t+24 is a different product than one that degrades gracefully, and the average hides the difference.
9. Ship it
Export the model with normalization baked in so no serving code has to remember the training statistics:
inputs = keras.Input(shape=(INPUT_WIDTH, len(feature_cols)))
x = keras.layers.Normalization(mean=mean.to_numpy(), variance=(std.to_numpy() ** 2))(inputs)
outputs = trained_core(x)
serving_model = keras.Model(inputs, outputs)
serving_model.export("serving/forecast/1")
From there it is the same deployment story as any other TensorFlow model - see Serving TensorFlow Models in Production for signatures, batching, and version labels, and Catching Data Drift Early with TensorFlow Data Validation for the monitoring side. Two forecasting-specific additions:
- Log the forecast alongside the actual when it arrives. Forecast accuracy is only measurable in arrears; if you don't store predictions with their issue time, you can never compute it.
- Alarm on input freshness. Most forecasting incidents are stale or missing upstream data, not a bad model. A pipeline that silently forward-fills a dead sensor produces confident nonsense.
What we would tell you on a scoping call
Time-series work rewards discipline over novelty. Get the windowing and the split right, beat seasonal-naive by a margin that survives backtesting, ship intervals instead of points, and monitor the inputs. In a majority of engagements the winning model has been a dilated CNN or even a regularized linear head on well-built features - and the value came from the evaluation harness that let the client trust it.
If you have a forecasting problem and want a second opinion on whether deep learning is the right tool for it, our TensorFlow consultants run short feasibility engagements that end with a backtested baseline and an honest recommendation. Get in touch and describe the series.