Exercises: Feature Engineering¶

These exercises use a realistic customer dataset. Work through them in order — each level builds on the previous. Do not look at the solutions before attempting each exercise.

Dataset Setup¶

Run this block first. All exercises reference these DataFrames.

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split

np.random.seed(42)
n = 800

# Core customer dataset
customers = pd.DataFrame({
    "customer_id": range(1001, 1001 + n),
    "signup_date": pd.to_datetime("2022-01-01") + pd.to_timedelta(
        np.random.randint(0, 730, n), unit="D"
    ),
    "last_purchase_date": pd.to_datetime("2023-06-01") + pd.to_timedelta(
        np.random.randint(-180, 180, n), unit="D"
    ),
    "city": np.random.choice(
        ["Mumbai", "Delhi", "Bangalore", "Chennai", "Pune",
         "Hyderabad", "Kolkata", "Ahmedabad",
         "Jaipur", "Surat", "Lucknow", "Kanpur",
         "Nagpur", "Indore", "Thane", "Bhopal"],
        size=n,
        p=[0.18, 0.16, 0.14, 0.10, 0.08,
           0.07, 0.06, 0.05,
           0.03, 0.03, 0.02, 0.02,
           0.01, 0.01, 0.02, 0.02]
    ),
    "plan_type": np.random.choice(["Basic", "Standard", "Premium"], n, p=[0.5, 0.35, 0.15]),
    "monthly_spend": np.random.exponential(scale=2000, size=n),
    "total_orders": np.random.poisson(lam=8, size=n),
    "support_tickets_opened": np.random.poisson(lam=1.5, size=n),
    "review_text": np.random.choice([
        "Great service, very satisfied with everything",
        "Product quality could be better, delivery was slow",
        "Excellent! Fast shipping and good packaging",
        "Not happy with the quality, expected more for the price",
        "Average experience, nothing special but no major issues",
        "Outstanding customer support helped me resolve my issue quickly",
        "Terrible experience, product arrived damaged",
        "Very good value for money, will definitely order again"
    ], n),
    "churn": np.random.binomial(1, 0.28, n)
})

# Introduce some missing values
customers.loc[np.random.choice(n, 40, replace=False), "monthly_spend"] = np.nan
customers.loc[np.random.choice(n, 25, replace=False), "support_tickets_opened"] = np.nan

print(f"Dataset shape: {customers.shape}")
print(customers.dtypes)
print(f"\nMissing values:\n{customers.isnull().sum()[customers.isnull().sum() > 0]}")

Warm-Up Exercises¶

These build core muscle memory. Each should take 5–10 minutes.

Warm-Up 1 — Numeric Transforms¶

Task: Apply the following transforms to the monthly_spend column. Print the skewness before and after each transform.

Fill the missing values with the column median (not the overall mean — why?)
Apply np.log1p and store as log_monthly_spend
Compute a winsorized version by clipping at the 2nd and 98th percentiles, store as monthly_spend_winsorized
Bin total_orders into quartiles using pd.qcut with labels ["Low", "Medium", "High", "Very High"]

# Your code here

Show answer

import numpy as np
import pandas as pd
from scipy.stats.mstats import winsorize

# 1. Fill missing values with median (not mean — monthly_spend is right-skewed; mean is pulled
#    by extreme values and is a worse estimate of a "typical" customer's spend)
customers["monthly_spend"] = customers["monthly_spend"].fillna(customers["monthly_spend"].median())

# 2. Log transform
print(f"monthly_spend skewness before: {customers['monthly_spend'].skew():.3f}")
customers["log_monthly_spend"] = np.log1p(customers["monthly_spend"])
print(f"monthly_spend skewness after:  {customers['log_monthly_spend'].skew():.3f}")

# 3. Winsorize (clip at 2nd and 98th percentile)
lower_bound = customers["monthly_spend"].quantile(0.02)
upper_bound = customers["monthly_spend"].quantile(0.98)
customers["monthly_spend_winsorized"] = customers["monthly_spend"].clip(
    lower=lower_bound, upper=upper_bound
)
print(f"\nBefore winsorize — max: {customers['monthly_spend'].max():.0f}")
print(f"After winsorize  — max: {customers['monthly_spend_winsorized'].max():.0f}")

# 4. Quantile binning for total_orders
customers["order_tier"] = pd.qcut(
    customers["total_orders"],
    q=4,
    labels=["Low", "Medium", "High", "Very High"],
    duplicates="drop"   # handle ties in quantile edges
)
print(f"\nOrder tier distribution:\n{customers['order_tier'].value_counts().sort_index()}")

Warm-Up 2 — Datetime Features¶

Task: Using the signup_date and last_purchase_date columns, extract the following features. Use pd.Timestamp("2024-01-01") as the reference date.

customer_tenure_days — days since signup as of the reference date
days_since_last_purchase — days since last purchase as of the reference date
signup_month — month number (1–12) of the signup date
signup_day_of_week — day of week (0=Monday) of signup
is_weekend_signup — 1 if the customer signed up on a Saturday or Sunday

# Your code here

Show answer

import pandas as pd

reference_date = pd.Timestamp("2024-01-01")

customers["signup_date"] = pd.to_datetime(customers["signup_date"])
customers["last_purchase_date"] = pd.to_datetime(customers["last_purchase_date"])

customers["customer_tenure_days"] = (
    reference_date - customers["signup_date"]
).dt.days

customers["days_since_last_purchase"] = (
    reference_date - customers["last_purchase_date"]
).dt.days

customers["signup_month"] = customers["signup_date"].dt.month
customers["signup_day_of_week"] = customers["signup_date"].dt.dayofweek
customers["is_weekend_signup"] = customers["signup_day_of_week"].isin([5, 6]).astype(int)

print(customers[[
    "customer_tenure_days", "days_since_last_purchase",
    "signup_month", "signup_day_of_week", "is_weekend_signup"
]].head(8))

Warm-Up 3 — Label Encoding vs One-Hot Encoding¶

Task: Encode the plan_type column twice: 1. Using a manual integer mapping (treat it as ordinal: Basic=1, Standard=2, Premium=3) 2. Using OneHotEncoder from sklearn with drop='first' and handle_unknown='ignore'

Print both results and explain in a comment why both could be valid or invalid depending on the model.

# Your code here

Show answer

import pandas as pd
from sklearn.preprocessing import OneHotEncoder

# 1. Ordinal mapping — valid if the model should treat Premium as "more" than Standard
plan_order = {"Basic": 1, "Standard": 2, "Premium": 3}
customers["plan_type_ordinal"] = customers["plan_type"].map(plan_order)

# 2. One-hot encoding — valid when we do NOT want to assume a linear ordering
ohe = OneHotEncoder(drop="first", handle_unknown="ignore", sparse_output=False)
plan_encoded = ohe.fit_transform(customers[["plan_type"]])
plan_feature_names = ohe.get_feature_names_out(["plan_type"])
plan_ohe_df = pd.DataFrame(plan_encoded, columns=plan_feature_names, index=customers.index)

print("Ordinal encoding:")
print(customers[["plan_type", "plan_type_ordinal"]].value_counts().reset_index())

print("\nOne-hot encoding (first 5 rows):")
print(plan_ohe_df.head())

# Comment: For a logistic regression, ordinal encoding forces the model to assume
# the effect of moving from Basic→Standard equals Standard→Premium. If that linear
# assumption holds (e.g., spend correlates linearly with plan tier), ordinal is fine.
# One-hot encoding makes no such assumption and lets the model learn independent
# coefficients for each category. For tree models, either works — trees split
# independently and don't care about scale.

Main Exercises¶

These require combining multiple techniques. Each should take 20–30 minutes.

Main 1 — High-Cardinality Encoding with Target Encoding¶

The city column has 16 unique values. That is borderline for one-hot encoding (16 → 15 columns after dropping one). For this exercise, implement target encoding with cross-validation instead.

Task:

Count the occurrences of each city and identify any with fewer than 50 rows. Group them into "Other_City".
Implement the cross_val_target_encode function from 03-categorical-features on the cleaned city column, with n_splits=5 and smoothing=5.
Compare: train a LogisticRegression on (a) one-hot-encoded city and (b) target-encoded city. Use 5-fold CV and report ROC-AUC for each.

# Your code here

Show answer

import pandas as pd
import numpy as np
from sklearn.preprocessing import OneHotEncoder
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import KFold, cross_val_score
from sklearn.pipeline import Pipeline

# Step 1: Group rare cities
city_counts = customers["city"].value_counts()
frequent_cities = city_counts[city_counts >= 50].index
customers["city_clean"] = customers["city"].where(
    customers["city"].isin(frequent_cities), other="Other_City"
)
print(f"Unique cities before grouping: {customers['city'].nunique()}")
print(f"Unique cities after grouping:  {customers['city_clean'].nunique()}")

# Step 2: Cross-validated target encoding
def cross_val_target_encode(df, col, target, n_splits=5, smoothing=5.0):
    global_mean = df[target].mean()
    encoded = pd.Series(np.nan, index=df.index, name=f"{col}_te")
    kf = KFold(n_splits=n_splits, shuffle=True, random_state=42)
    for train_idx, val_idx in kf.split(df):
        train_fold = df.iloc[train_idx]
        means = train_fold.groupby(col)[target].agg(["mean", "count"])
        smoothed = (means["count"] * means["mean"] + smoothing * global_mean) / \
                   (means["count"] + smoothing)
        encoded.iloc[val_idx] = df.iloc[val_idx][col].map(smoothed).fillna(global_mean)
    return encoded

customers["city_te"] = cross_val_target_encode(
    customers, col="city_clean", target="churn", n_splits=5
)

# Step 3: Compare OHE vs target encoding on city alone
# Prepare a simple feature set for the comparison
feature_base = customers[["customer_tenure_days", "log_monthly_spend",
                           "total_orders", "plan_type_ordinal"]].copy()

# (a) OHE city
ohe = OneHotEncoder(handle_unknown="ignore", drop="first", sparse_output=False)
city_ohe = ohe.fit_transform(customers[["city_clean"]])
city_ohe_df = pd.DataFrame(city_ohe, index=customers.index)
X_ohe = pd.concat([feature_base, city_ohe_df], axis=1).fillna(0)

# (b) Target encoded city
X_te = pd.concat([feature_base, customers[["city_te"]]], axis=1).fillna(0)

y = customers["churn"]

lr = LogisticRegression(max_iter=500, random_state=42)

scores_ohe = cross_val_score(lr, X_ohe, y, cv=5, scoring="roc_auc")
scores_te  = cross_val_score(lr, X_te,  y, cv=5, scoring="roc_auc")

print(f"\nOHE city  — CV ROC-AUC: {scores_ohe.mean():.4f} ± {scores_ohe.std():.4f}")
print(f"Target enc — CV ROC-AUC: {scores_te.mean():.4f} ± {scores_te.std():.4f}")

Main 2 — Full ColumnTransformer Pipeline¶

Build a production-quality pipeline that handles all columns correctly.

Task:

Start with a fresh split of the customers DataFrame (use test_size=0.2, random_state=42)
Build a ColumnTransformer that:
Imputes and scales these numeric columns: customer_tenure_days, days_since_last_purchase, monthly_spend, total_orders, support_tickets_opened
One-hot-encodes these categorical columns: plan_type, city_clean (after rare-city grouping)
Wrap the ColumnTransformer in a Pipeline with a GradientBoostingClassifier
Fit on training data only
Report CV ROC-AUC on training data and final accuracy on test data
Save the pipeline to disk with joblib.dump

# Your code here

Show answer

import pandas as pd
import numpy as np
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.metrics import roc_auc_score, classification_report
import joblib

# Prepare feature matrix (using columns we have already engineered)
feature_cols = [
    "customer_tenure_days", "days_since_last_purchase",
    "monthly_spend", "total_orders", "support_tickets_opened",
    "plan_type", "city_clean"
]

X = customers[feature_cols].copy()
y = customers["churn"]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Column groups
numeric_cols = [
    "customer_tenure_days", "days_since_last_purchase",
    "monthly_spend", "total_orders", "support_tickets_opened"
]
categorical_cols = ["plan_type", "city_clean"]

# Sub-pipelines
numeric_pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="median")),
    ("scaler", StandardScaler())
])

categorical_pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="most_frequent")),
    ("encoder", OneHotEncoder(handle_unknown="ignore", drop="first", sparse_output=False))
])

preprocessor = ColumnTransformer(transformers=[
    ("numeric", numeric_pipeline, numeric_cols),
    ("categorical", categorical_pipeline, categorical_cols)
])

# Full pipeline
churn_pipeline = Pipeline([
    ("preprocessing", preprocessor),
    ("model", GradientBoostingClassifier(n_estimators=100, max_depth=3, random_state=42))
])

# Fit on training data only
churn_pipeline.fit(X_train, y_train)

# Cross-validate on training data
cv_auc = cross_val_score(churn_pipeline, X_train, y_train, cv=5, scoring="roc_auc")
print(f"CV ROC-AUC (train): {cv_auc.mean():.4f} ± {cv_auc.std():.4f}")

# Evaluate on test data
y_pred = churn_pipeline.predict(X_test)
y_proba = churn_pipeline.predict_proba(X_test)[:, 1]
print(f"Test ROC-AUC: {roc_auc_score(y_test, y_proba):.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred))

# Save the pipeline
joblib.dump(churn_pipeline, "churn_pipeline_v1.pkl")
print("\nPipeline saved to churn_pipeline_v1.pkl")

Main 3 — Text Feature Engineering¶

Task:

Extract the following features from the review_text column:
review_word_count
review_char_count (character count excluding spaces)
review_avg_word_length
review_exclamation_count
Build a TfidfVectorizer with max_features=50, stop_words='english', ngram_range=(1,2). Fit it on training data only.
Add the TF-IDF matrix to your training and test feature sets using scipy.sparse.hstack
Train a LogisticRegression on the combined features and report test accuracy

# Your code here

Show answer

import pandas as pd
import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
import scipy.sparse as sp

# Step 1: Statistical text features
customers["review_word_count"]      = customers["review_text"].str.split().str.len()
customers["review_char_count"]      = customers["review_text"].str.replace(" ", "").str.len()
customers["review_avg_word_length"] = customers["review_text"].apply(
    lambda t: np.mean([len(w) for w in t.split()]) if t.split() else 0
)
customers["review_exclamation_count"] = customers["review_text"].str.count("!")

# Step 2: Split
text_feature_cols = [
    "review_word_count", "review_char_count",
    "review_avg_word_length", "review_exclamation_count"
]
X_text_stats = customers[text_feature_cols]
y = customers["churn"]

X_train_stats, X_test_stats, y_train, y_test, text_train, text_test = train_test_split(
    X_text_stats, y, customers["review_text"], test_size=0.2, random_state=42
)

# Step 3: TF-IDF — fit only on training text
tfidf = TfidfVectorizer(max_features=50, stop_words="english", ngram_range=(1, 2))
X_train_tfidf = tfidf.fit_transform(text_train)   # fit_transform on train
X_test_tfidf  = tfidf.transform(text_test)         # transform only on test

# Step 4: Combine sparse TF-IDF with dense statistical features
X_train_combined = sp.hstack([
    sp.csr_matrix(X_train_stats.values), X_train_tfidf
])
X_test_combined = sp.hstack([
    sp.csr_matrix(X_test_stats.values), X_test_tfidf
])

# Train and evaluate
lr = LogisticRegression(max_iter=500, random_state=42, solver="lbfgs")
lr.fit(X_train_combined, y_train)
y_pred = lr.predict(X_test_combined)

print(classification_report(y_test, y_pred))
print(f"Top TF-IDF terms by absolute coefficient magnitude:")
tfidf_feature_names = tfidf.get_feature_names_out()
# Coefficients for class 1 (churn)
tfidf_coefs = lr.coef_[0][len(text_feature_cols):]
top_indices = np.argsort(np.abs(tfidf_coefs))[-10:][::-1]
for idx in top_indices:
    print(f"  {tfidf_feature_names[idx]:30s}: {tfidf_coefs[idx]:.4f}")

Stretch Exercises¶

These require independent thinking and go beyond the patterns shown in the notes.

Stretch 1 — Identify the Leakage¶

The following pipeline has multiple data leakage problems. Find all of them and explain why each is leakage. Then rewrite the pipeline correctly.

import pandas as pd
import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

df = pd.DataFrame({
    "age": [25, 32, 45, np.nan, 28, 52, np.nan, 35],
    "income": [40000, 85000, 120000, 65000, np.nan, 95000, 72000, np.nan],
    "approved": [0, 1, 1, 0, 1, 1, 0, 1]
})

# --- LEAKY PIPELINE BELOW ---

# Fill missing values using the full dataset
df["age"] = df["age"].fillna(df["age"].mean())
df["income"] = df["income"].fillna(df["income"].median())

# Add a feature that reveals the target
df["loan_status_flag"] = df["approved"] * 2 + 1  # clearly encodes the target

# Scale on the full dataset
scaler = StandardScaler()
df[["age_s", "income_s"]] = scaler.fit_transform(df[["age", "income"]])

# Split after all transformations
X = df[["age_s", "income_s", "loan_status_flag"]]
y = df["approved"]
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)

model = LogisticRegression()
model.fit(X_train, y_train)
print(f"Accuracy: {model.score(X_test, y_test):.2f}")

# Your corrected pipeline here

Show answer

Three leakage problems in the original code:

Train/test contamination (imputation): df["age"].fillna(df["age"].mean()) uses the mean of the entire dataset including test rows. The imputer should be fitted on training data only.
Train/test contamination (scaling): scaler.fit_transform(df[...]) is fitted on the full dataset before the split. Test set statistics contaminate the scaler.
Target leakage: loan_status_flag = approved * 2 + 1 directly encodes the target variable. Any model trained with this feature is trivially learning to predict the target from the target itself.

import pandas as pd
import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

df = pd.DataFrame({
    "age": [25, 32, 45, np.nan, 28, 52, np.nan, 35],
    "income": [40000, 85000, 120000, 65000, np.nan, 95000, 72000, np.nan],
    "approved": [0, 1, 1, 0, 1, 1, 0, 1]
})

# CORRECT — drop the target-leaking feature entirely
X = df[["age", "income"]]   # loan_status_flag removed
y = df["approved"]

# CORRECT — split FIRST, before any preprocessing
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)

# CORRECT — all preprocessing inside a Pipeline
clean_pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="mean")),   # fit on X_train only
    ("scaler", StandardScaler()),                  # fit on X_train only
    ("model", LogisticRegression(random_state=42))
])

clean_pipeline.fit(X_train, y_train)
print(f"Accuracy: {clean_pipeline.score(X_test, y_test):.2f}")
# Note: with only 8 rows and no real signal, accuracy is not meaningful here —
# the point is the pipeline structure, not the score

The signup_month feature (1–12) is cyclical — December (12) is close to January (1).

Task: 1. Apply sin/cos cyclical encoding to signup_month 2. Visualise the (sin, cos) pairs on a unit circle using matplotlib to verify that December and January are adjacent 3. Add the cyclical features to the churn_pipeline from Main Exercise 2 and check whether CV ROC-AUC improves

# Your code here

Show answer

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import cross_val_score

# Step 1: Cyclical encoding
customers["signup_month_sin"] = np.sin(2 * np.pi * customers["signup_month"] / 12)
customers["signup_month_cos"] = np.cos(2 * np.pi * customers["signup_month"] / 12)

# Step 2: Visualise on unit circle
month_points = pd.DataFrame({
    "month": range(1, 13),
    "sin": np.sin(2 * np.pi * np.arange(1, 13) / 12),
    "cos": np.cos(2 * np.pi * np.arange(1, 13) / 12),
    "label": ["Jan", "Feb", "Mar", "Apr", "May", "Jun",
              "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]
})

fig, ax = plt.subplots(figsize=(7, 7))
ax.scatter(month_points["cos"], month_points["sin"], s=80)
for _, row in month_points.iterrows():
    ax.annotate(row["label"], (row["cos"], row["sin"]),
                textcoords="offset points", xytext=(8, 4))
circle = plt.Circle((0, 0), 1, fill=False, linestyle="--", color="gray")
ax.add_patch(circle)
ax.set_xlim(-1.3, 1.3)
ax.set_ylim(-1.3, 1.3)
ax.set_aspect("equal")
ax.set_title("Cyclical Encoding — Signup Month on Unit Circle")
ax.set_xlabel("cos(month)")
ax.set_ylabel("sin(month)")
plt.tight_layout()
plt.savefig("signup_month_cyclical.png", dpi=120)
plt.show()
# December (Dec) and January (Jan) are adjacent on the circle — correct!

# Step 3: Add to pipeline and compare CV scores
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.ensemble import GradientBoostingClassifier

feature_cols_extended = [
    "customer_tenure_days", "days_since_last_purchase",
    "monthly_spend", "total_orders", "support_tickets_opened",
    "signup_month_sin", "signup_month_cos",   # cyclical features added
    "plan_type", "city_clean"
]
numeric_cols_extended = [
    "customer_tenure_days", "days_since_last_purchase",
    "monthly_spend", "total_orders", "support_tickets_opened",
    "signup_month_sin", "signup_month_cos"
]

X_extended = customers[feature_cols_extended].copy()
y = customers["churn"]

preprocessor_ext = ColumnTransformer(transformers=[
    ("numeric", Pipeline([
        ("imputer", SimpleImputer(strategy="median")),
        ("scaler", StandardScaler())
    ]), numeric_cols_extended),
    ("categorical", Pipeline([
        ("imputer", SimpleImputer(strategy="most_frequent")),
        ("encoder", OneHotEncoder(handle_unknown="ignore", drop="first", sparse_output=False))
    ]), ["plan_type", "city_clean"])
])

extended_pipeline = Pipeline([
    ("preprocessing", preprocessor_ext),
    ("model", GradientBoostingClassifier(n_estimators=100, max_depth=3, random_state=42))
])

cv_ext = cross_val_score(extended_pipeline, X_extended, y, cv=5, scoring="roc_auc")
print(f"CV ROC-AUC with cyclical month features: {cv_ext.mean():.4f} ± {cv_ext.std():.4f}")

Stretch 3 — Custom Transformer¶

Sklearn's Pipeline accepts any transformer that follows the fit / transform interface. Build a custom transformer that computes days_since_last_purchase from a raw datetime column, then insert it into a pipeline so it runs automatically during fit and transform.

Task: Create a class RecencyTransformer that: - Takes a DataFrame with a last_purchase_date column - Accepts a reference_date parameter - Returns the DataFrame with an additional days_since_last_purchase column (raw date column dropped) - Follows the sklearn transformer interface: fit, transform, fit_transform, get_params

# Your code here

Show answer

import pandas as pd
import numpy as np
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

class RecencyTransformer(BaseEstimator, TransformerMixin):
    """
    Converts a datetime column into a recency (days since event) numeric feature.
    Inheriting from BaseEstimator gives get_params/set_params for free.
    Inheriting from TransformerMixin gives fit_transform for free.
    """

    def __init__(self, date_col: str = "last_purchase_date",
                 reference_date: str = "2024-01-01"):
        self.date_col = date_col
        self.reference_date = reference_date

    def fit(self, X, y=None):
        # Nothing to learn — recency calculation has no learnable parameters
        return self

    def transform(self, X, y=None):
        X = X.copy()
        ref = pd.Timestamp(self.reference_date)
        X[self.date_col] = pd.to_datetime(X[self.date_col])
        X["days_since_last_purchase"] = (ref - X[self.date_col]).dt.days
        X = X.drop(columns=[self.date_col])
        return X


# Test the transformer
sample = pd.DataFrame({
    "last_purchase_date": pd.date_range("2023-01-01", periods=5, freq="30D"),
    "monthly_spend": [500.0, 1200.0, 3000.0, 750.0, 2100.0]
})

transformer = RecencyTransformer(
    date_col="last_purchase_date",
    reference_date="2024-01-01"
)
result = transformer.fit_transform(sample)
print(result)
# Output: last_purchase_date is gone, days_since_last_purchase is added

# Use inside a Pipeline
recency_pipeline = Pipeline([
    ("recency", RecencyTransformer(date_col="last_purchase_date")),
    ("scaler", StandardScaler()),
])

X_with_date = pd.DataFrame({
    "last_purchase_date": pd.date_range("2023-06-01", periods=10, freq="15D"),
    "monthly_spend": np.random.exponential(1500, 10)
})

X_transformed = recency_pipeline.fit_transform(X_with_date)
print(f"\nPipeline output shape: {X_transformed.shape}")
print(f"Columns after transform: ['days_since_last_purchase', 'monthly_spend'] (both scaled)")

Session Complete

You have built numeric transformations, datetime features, categorical encodings, text features, and a production-quality ColumnTransformer pipeline. The leakage identification exercise in Stretch 1 is the most important skill to carry forward — it is the mistake that silently destroys production models.

Back: Pipelines and Leakage | Next: Model Evaluation

Exercises: Feature Engineering¶

Dataset Setup¶

Warm-Up Exercises¶

Warm-Up 1 — Numeric Transforms¶

Warm-Up 2 — Datetime Features¶

Warm-Up 3 — Label Encoding vs One-Hot Encoding¶

Main Exercises¶

Main 1 — High-Cardinality Encoding with Target Encoding¶

Main 2 — Full ColumnTransformer Pipeline¶

Main 3 — Text Feature Engineering¶

Stretch Exercises¶

Stretch 1 — Identify the Leakage¶

Stretch 2 — Cyclical Encoding for Signup Month¶

Stretch 3 — Custom Transformer¶