Feature Engineering for Time Series¶
A date column is not a feature — it is a label. The model cannot learn from "2023-11-15" as text. Feature engineering for time series is the process of converting that date into numbers that encode the patterns the EDA revealed: which day of week it is, which month, how sales behaved recently. The EDA showed that weekly and annual seasonality are strong and that lag-7 has a correlation of 0.91 with today's sales. This file translates those findings into concrete features.
Learning Objectives¶
- Extract calendar features from a datetime column
- Create lag features that let the model see its own recent history
- Build rolling window statistics without introducing lookahead leakage
- Perform a chronologically correct train-test split
- One-hot encode the category column
- Identify and drop NaN rows created by lag and rolling operations
Setup — Regenerate the Dataset¶
import pandas as pd
import numpy as np
rng = np.random.default_rng(42)
dates = pd.date_range(start="2021-01-01", end="2023-12-31", freq="D")
n = len(dates)
trend = np.linspace(1000, 1600, n)
day_of_week = pd.Series(dates).dt.dayofweek
weekly = np.where(day_of_week < 5, 200, -150)
month = pd.Series(dates).dt.month
annual = np.where(month.isin([11, 12]), 400,
np.where(month.isin([10]), 200,
np.where(month.isin([1, 2]), -200,
np.where(month.isin([7, 8]), 100, 0))))
noise = rng.normal(0, 80, n)
sales = np.clip(trend + weekly + annual + noise, 100, None)
df = pd.DataFrame({"date": dates, "sales": sales.round(0).astype(int)})
categories = ["Electronics", "Clothing", "Food", "Home"]
category_multipliers = {"Electronics": 1.0, "Clothing": 0.7, "Food": 0.5, "Home": 0.4}
rows = []
for cat, mult in category_multipliers.items():
cat_df = df.copy()
cat_df["category"] = cat
cat_df["sales"] = (
cat_df["sales"] * mult * rng.uniform(0.9, 1.1, n)
).round(0).astype(int)
rows.append(cat_df)
df_full = pd.concat(rows, ignore_index=True).sort_values("date").reset_index(drop=True)
print(df_full.shape) # (4384, 3)
Step 1 — Calendar Features¶
The EDA showed weekly and annual seasonality. Calendar features encode the temporal position of each observation as numbers the model can use directly.
df_full["date"] = pd.to_datetime(df_full["date"])
df_full["year"] = df_full["date"].dt.year
df_full["month"] = df_full["date"].dt.month
df_full["day_of_month"] = df_full["date"].dt.day
df_full["day_of_week"] = df_full["date"].dt.dayofweek # 0=Monday, 6=Sunday
df_full["week_of_year"] = df_full["date"].dt.isocalendar().week.astype(int)
df_full["quarter"] = df_full["date"].dt.quarter
df_full["is_weekend"] = (df_full["day_of_week"] >= 5).astype(int)
df_full["is_month_start"] = df_full["date"].dt.is_month_start.astype(int)
df_full["is_month_end"] = df_full["date"].dt.is_month_end.astype(int)
calendar_cols = [
"year", "month", "day_of_month", "day_of_week",
"week_of_year", "quarter", "is_weekend", "is_month_start", "is_month_end"
]
print(df_full[["date", "category", "sales"] + calendar_cols].head(4))
date category sales year month day_of_month day_of_week week_of_year quarter is_weekend is_month_start is_month_end
0 2021-01-01 Electronics 640 2021 1 1 4 53 1 1 1 0
1 2021-01-01 Clothing 448 2021 1 1 4 53 1 1 1 0
2 2021-01-01 Food 320 2021 1 1 4 53 1 1 1 0
3 2021-01-01 Home 256 2021 1 1 4 53 1 1 1 0
Info
Why not pass the raw date to the model? Tree-based models cannot use a datetime object — they need numeric inputs. Even if you convert dates to integers (Unix timestamp), the model would treat the distance between January and February as the same type of relationship as the distance between one year and the next. Calendar features decompose time into meaningful cycles that the model can split on meaningfully.
Step 2 — Lag Features¶
Lag features give the model access to its own recent history. A lag-1 feature at row t contains the sales value from time t-1. A lag-7 feature contains sales from exactly one week ago.
Why lags must be computed per category: The row below row t for Electronics is not Electronics on the next day — after sorting by date, the next row is Clothing on the same day. Computing shifts on the unsorted frame would mix categories. Always sort by ['category', 'date'] and use groupby('category') before shifting.
# Sort so that each category's rows are consecutive by date
df_full = df_full.sort_values(["category", "date"]).reset_index(drop=True)
# Lag features — shift within each category group
df_full["lag_1"] = df_full.groupby("category")["sales"].shift(1)
df_full["lag_7"] = df_full.groupby("category")["sales"].shift(7)
df_full["lag_14"] = df_full.groupby("category")["sales"].shift(14)
df_full["lag_28"] = df_full.groupby("category")["sales"].shift(28)
# Inspect — the first rows per category will have NaN lags
print(df_full[df_full["category"] == "Electronics"][
["date", "sales", "lag_1", "lag_7", "lag_14", "lag_28"]
].head(6))
date sales lag_1 lag_7 lag_14 lag_28
0 2021-01-01 640 NaN NaN NaN NaN
1 2021-01-02 638 640 NaN NaN NaN
2 2021-01-03 907 638 NaN NaN NaN
3 2021-01-04 906 907 NaN NaN NaN
4 2021-01-05 905 906 NaN NaN NaN
5 2021-01-06 684 905 NaN NaN NaN
Warning
It is safe to compute lags before the train-test split. Lag features look backward — lag_7 at a test-set row uses only the 7-day-old sales value, which is in the past. No future information crosses from test into train. However, the NaN rows created by lagging must be dropped before training. A lag-28 feature means the first 28 rows per category are unusable. Dropping them before splitting is fine because those rows contain no information from the test period.
Step 3 — Rolling Window Features¶
Rolling window features summarise recent behavior over a window of past observations. A 7-day rolling mean captures the recent local level; a 28-day rolling mean captures the monthly trend.
The critical detail: a rolling feature at time t must not include the value at time t itself — that would be using the thing you are trying to predict as an input. Shift the series by 1 before rolling to ensure the window covers [t-window, t-1].
# Shift-then-roll to avoid lookahead leakage
shifted_sales = df_full.groupby("category")["sales"].shift(1)
df_full["rolling_mean_7"] = shifted_sales.groupby(df_full["category"]).transform(
lambda x: x.rolling(7, min_periods=1).mean()
)
df_full["rolling_mean_28"] = shifted_sales.groupby(df_full["category"]).transform(
lambda x: x.rolling(28, min_periods=1).mean()
)
df_full["rolling_std_7"] = shifted_sales.groupby(df_full["category"]).transform(
lambda x: x.rolling(7, min_periods=2).std()
)
print(df_full[df_full["category"] == "Electronics"][
["date", "sales", "lag_1", "rolling_mean_7", "rolling_mean_28", "rolling_std_7"]
].iloc[25:31])
date sales lag_1 rolling_mean_7 rolling_mean_28 rolling_std_7
25 2021-01-26 912 912 878.7 869.4 48.3
26 2021-01-27 910 912 885.0 870.2 45.7
27 2021-01-28 649 910 885.1 870.5 50.2
28 2021-01-29 651 649 877.0 869.0 56.4
29 2021-01-30 900 651 852.9 867.3 63.5
30 2021-01-31 897 900 845.6 866.1 61.8
Warning
Concrete lookahead leakage example: Suppose you compute rolling_mean_7 on November 10 as the mean of November 4–10. On November 10, sales spiked due to a promotion. That spike appears in the rolling feature for every day from November 11–16. Now your model, trained on 2021–2022, "knows" that a promotion spike happened in that week — information it would not have in a real deployment where forecasts are made before the week begins. The fix is always to shift by 1 so the window ends at t-1.
Step 4 — Correct Train-Test Split¶
Time series data must be split in chronological order. Everything before the cutoff date is training data; everything after is test data. Never shuffle.
cutoff = pd.Timestamp("2023-07-01")
train = df_full[df_full["date"] < cutoff].copy()
test = df_full[df_full["date"] >= cutoff].copy()
print(f"Train: {train['date'].min().date()} to {train['date'].max().date()} — {len(train):,} rows")
print(f"Test: {test['date'].min().date()} to {test['date'].max().date()} — {len(test):,} rows")
Train: 2021-01-01 to 2023-06-30 — 3,504 rows (before NaN drop)
Test: 2023-07-01 to 2023-12-31 — 880 rows (before NaN drop)
Warning
Why random shuffling destroys the evaluation. If you call train_test_split(df, shuffle=True), rows from December 2023 end up in the training set and rows from January 2021 end up in the test set. The model is trained on the future and evaluated on the past. Lag features computed on this shuffled dataset contain future values as "past" inputs. Every metric you compute is meaningless. This is the single most common mistake in time series modelling — be ready to explain it clearly.
Step 5 — One-Hot Encode Category¶
Tree-based models can use label-encoded categories, but one-hot encoding is more interpretable and avoids any implicit ordinal relationship between category names.
train = pd.get_dummies(train, columns=["category"], drop_first=False)
test = pd.get_dummies(test, columns=["category"], drop_first=False)
# Verify both frames have the same columns after encoding
category_cols = [c for c in train.columns if c.startswith("category_")]
print("Category columns:", category_cols)
print(train[category_cols].head(4))
Category columns: ['category_Clothing', 'category_Electronics', 'category_Food', 'category_Home']
category_Clothing category_Electronics category_Food category_Home
0 0 1 0 0
1 1 0 0 0
2 0 0 1 0
3 0 0 0 1
Step 6 — Drop NaN Rows¶
Lag-28 creates 28 NaN rows at the start of each category's series. These cannot be used for training and must be dropped.
print(f"Before dropping NaNs — train: {len(train):,} rows, test: {len(test):,} rows")
train = train.dropna()
test = test.dropna()
print(f"After dropping NaNs — train: {len(train):,} rows, test: {len(test):,} rows")
print(f"NaN rows dropped from train: {3504 - len(train)}")
Before dropping NaNs — train: 3,504 rows, test: 880 rows
After dropping NaNs — train: 3,392 rows, test: 880 rows
NaN rows dropped from train: 112
112 rows dropped = 28 per category × 4 categories. The test set retains all 880 rows because the first 28 days of the test period have valid lag values from the training period.
Step 7 — Define the Feature Matrix¶
feature_cols = [
# Calendar
"year", "month", "day_of_month", "day_of_week",
"week_of_year", "quarter", "is_weekend", "is_month_start", "is_month_end",
# Lag
"lag_1", "lag_7", "lag_14", "lag_28",
# Rolling
"rolling_mean_7", "rolling_mean_28", "rolling_std_7",
# Category (one-hot)
"category_Clothing", "category_Electronics", "category_Food", "category_Home",
]
X_train = train[feature_cols]
y_train = train["sales"]
X_test = test[feature_cols]
y_test = test["sales"]
print(f"X_train: {X_train.shape}") # (3392, 20)
print(f"X_test: {X_test.shape}") # (880, 20)
Feature Summary¶
| Feature | Type | What it encodes |
|---|---|---|
year |
Calendar | Long-run trend direction |
month |
Calendar | Month within year — annual seasonality |
day_of_month |
Calendar | Position within month |
day_of_week |
Calendar | Day within week — weekly seasonality |
week_of_year |
Calendar | Week number — finer annual granularity |
quarter |
Calendar | Q4 spike, Q1 dip |
is_weekend |
Calendar | Binary weekday/weekend flag |
is_month_start |
Calendar | First-of-month purchasing patterns |
is_month_end |
Calendar | End-of-month purchasing patterns |
lag_1 |
Lag | Yesterday's sales — captures momentum |
lag_7 |
Lag | Same day last week — captures weekly cycle |
lag_14 |
Lag | Two weeks ago |
lag_28 |
Lag | Four weeks ago — captures monthly cycle |
rolling_mean_7 |
Rolling | 7-day average of recent sales level |
rolling_mean_28 |
Rolling | 28-day average — smoothed recent trend |
rolling_std_7 |
Rolling | 7-day sales volatility |
category_* |
Encoding | Which product category (one-hot) |
Success
Twenty features from a 3-column dataset. Every feature encodes something the EDA confirmed matters: weekly patterns, annual patterns, trend, recent history, and category identity. A model handed these 20 columns starts with a meaningful representation of the problem — not raw dates it cannot use.