Dataset Guide

You will generate the dataset from scratch using NumPy and pandas. This gives you complete control over the data — you know exactly what signals are in it, which makes it an excellent learning environment. You can engineer features for trends and seasonality with confidence, because you built them in.

Generating the Dataset

Run this code once at the top of your notebook. Every subsequent file in this project assumes df_full is already in memory.

import pandas as pd
import numpy as np

rng = np.random.default_rng(42)
dates = pd.date_range(start="2021-01-01", end="2023-12-31", freq="D")
n = len(dates)

# Trend: gradual growth from ~1000 to ~1600 units per day
trend = np.linspace(1000, 1600, n)

# Weekly seasonality: weekdays sell more than weekends
day_of_week = pd.Series(dates).dt.dayofweek  # 0=Monday, 6=Sunday
weekly = np.where(day_of_week < 5, 200, -150)

# Annual seasonality: Q4 peak, Q1 trough, slight summer bump
month = pd.Series(dates).dt.month
annual = np.where(month.isin([11, 12]), 400,
         np.where(month.isin([10]), 200,
         np.where(month.isin([1, 2]), -200,
         np.where(month.isin([7, 8]), 100, 0))))

# Gaussian noise — real sales always have unexplained variation
noise = rng.normal(0, 80, n)

sales = trend + weekly + annual + noise
sales = np.clip(sales, 100, None)  # floor at 100 — no negative sales

df = pd.DataFrame({
    "date": dates,
    "sales": sales.round(0).astype(int),
})

# Add product category dimension with realistic relative sizes
categories = ["Electronics", "Clothing", "Food", "Home"]
category_multipliers = {"Electronics": 1.0, "Clothing": 0.7, "Food": 0.5, "Home": 0.4}

rows = []
for cat, mult in category_multipliers.items():
    cat_df = df.copy()
    cat_df["category"] = cat
    cat_df["sales"] = (
        cat_df["sales"] * mult * rng.uniform(0.9, 1.1, n)
    ).round(0).astype(int)
    rows.append(cat_df)

df_full = pd.concat(rows, ignore_index=True).sort_values("date").reset_index(drop=True)

print(df_full.shape)          # (4384, 3)
print(df_full.dtypes)
print(df_full.head(8))

Expected output:

(4384, 3)
date        datetime64[ns]
sales                int64
category            object
dtype: object
        date  sales     category
0 2021-01-01    640  Electronics
1 2021-01-01    448      Clothing
2 2021-01-01    320         Food
3 2021-01-01    256         Home
4 2021-01-02    638  Electronics
5 2021-01-02    447      Clothing
6 2021-01-02    319         Food
7 2021-01-02    255         Home

Column Reference

Column	Type	Description
date	datetime64	Calendar date, daily frequency, 2021-01-01 to 2023-12-31
sales	int64	Units sold on that date for that category
category	object	Product category: Electronics, Clothing, Food, Home

Data Statistics by Category

# Summary statistics per category
summary = df_full.groupby("category")["sales"].agg(
    count="count",
    mean="mean",
    std="std",
    min="min",
    max="max"
).round(1)

print(summary)

             count    mean    std  min   max
category
Clothing      1096   734.3   97.4  314  1101
Electronics   1096  1049.0  139.2  449  1573
Food          1096   524.5   69.6  224   793
Home          1096   419.6   55.7  179   634

Info

Electronics is the largest category (multiplier 1.0), Home is the smallest (0.4). All four categories share the same underlying date index, so there are exactly 1,096 rows per category — one per calendar day over three years.

What We Built Into the Data

Understanding the data-generating process helps you verify that your features and model actually capture what you put in.

Trend — sales grow steadily from ~1000 to ~1600 units over three years. This represents a business that is expanding its customer base year over year.

Weekly seasonality — weekdays (Mon–Fri) get a +200 boost; weekends (Sat–Sun) get a -150 penalty. This matches retail patterns where workweek purchasing (online orders, B2B) outpaces weekend leisure purchasing for most product types.

Annual seasonality — November and December spike by +400 (holiday shopping), October rises by +200 (pre-holiday stocking), January and February dip by -200 (post-holiday hangover), July and August lift by +100 (summer spending).

Noise — Gaussian noise with standard deviation 80. Even a perfect model that learns all three components will still have residual error around this level.

Category scaling — each category multiplies the base signal by a fixed ratio plus a small random perturbation per day (uniform(0.9, 1.1)). Categories trend together because they share the same underlying economy, but they diverge slightly in day-to-day variation.

Why Synthetic Data Is Realistic

This dataset matches patterns found in real retail data:

• Q4 sales spikes are among the most consistent signals in consumer retail. November and December reliably account for 30–40% of annual revenue for many categories.
• Weekend dips vary by category but are structurally real. Electronics, for example, shows stronger weekday purchasing online while grocery shows the opposite.
• Year-over-year growth is the norm for a healthy business. A flat trend would indicate stagnation; a declining trend would indicate a problem.

What Real-World Data Has That This Doesn't

Real sales data contains signals this synthetic version omits. Once you learn the base workflow here, these are the next layers to add:

• Promotions and discounts — price cuts create sharp, temporary spikes that look like noise but are actually predictable if you have the promotion calendar
• Holidays — national holidays (Thanksgiving, Christmas, New Year) create extreme spikes or closures that calendar features alone cannot fully capture without explicit flags
• Stockouts — if inventory runs out, sales drop to zero even when demand is high; this corrupts your training signal
• Store or region IDs — real data often has thousands of store-level series, each with its own intercept and sensitivity to local events
• SKU-level granularity — forecasting at the individual product level is much harder due to intermittent demand (many products sell zero units on a given day)
• External signals — weather, competitor pricing, search trends, and macroeconomic indicators all influence sales and can improve forecasts significantly

Tip

When an interviewer asks "how would you improve this model?", the answer almost always includes: add holiday flags, add promotion data, and add external economic indicators. These signals have high lift and are relatively cheap to obtain once a forecasting infrastructure exists.

---

← README | Next: EDA →