Feature Engineering¶

Raw features rarely tell the full story. The signal that predicts churn is not just how many times a customer called support — it is how intensely they called given how long they have been a customer. Feature engineering is where you translate domain knowledge into signals the model can exploit.

Learning Objectives¶

By the end of this file you will be able to:

Apply domain-informed imputation strategies for age and support_calls
Create four engineered features that capture customer behaviour patterns not visible in raw columns
Encode categorical and ordinal variables correctly
Scale numeric features using StandardScaler
Wrap the entire process in a reusable sklearn Pipeline that prevents data leakage

Setup: Generate the Dataset¶

Run this block first. All code in this file assumes df is in scope.

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder, OrdinalEncoder
from sklearn.impute import SimpleImputer

rng = np.random.default_rng(42)
n = 1000

tenure = rng.integers(1, 72, n)
age = rng.integers(22, 65, n).astype(float)
segment = rng.choice(["Basic", "Premium", "Enterprise"], n, p=[0.55, 0.35, 0.10])
region = rng.choice(["North", "South", "East", "West"], n)
support_calls = rng.integers(0, 15, n).astype(float)
monthly_fee = np.where(
    segment == "Enterprise", rng.uniform(2000, 5000, n),
    np.where(segment == "Premium", rng.uniform(800, 1500, n),
             rng.uniform(200, 600, n))
)
num_products = rng.integers(1, 6, n)
has_contract = rng.choice([0, 1], n, p=[0.4, 0.6])

churn_prob = (
    0.55
    - tenure * 0.008
    + support_calls * 0.04
    - (segment == "Premium") * 0.12
    - (segment == "Enterprise") * 0.20
    - has_contract * 0.18
    + rng.normal(0, 0.05, n)
)
churn_prob = np.clip(churn_prob, 0.02, 0.95)
churn = (rng.uniform(size=n) < churn_prob).astype(int)

df = pd.DataFrame({
    "customer_id": range(10001, 10001 + n),
    "age": age,
    "tenure_months": tenure,
    "segment": segment,
    "region": region,
    "support_calls": support_calls,
    "monthly_fee": monthly_fee.round(2),
    "num_products": num_products,
    "has_contract": has_contract,
    "churn": churn,
})

df.loc[rng.choice(df.index, 30, replace=False), "age"] = np.nan
df.loc[rng.choice(df.index, 15, replace=False), "support_calls"] = np.nan

Step 1: Train-Test Split First¶

Split before any transformation. This is the most important sequencing rule in feature engineering.

X = df.drop(columns=["customer_id", "churn"])
y = df["churn"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.20, random_state=42, stratify=y
)

print(f"Train: {X_train.shape}  |  Test: {X_test.shape}")
# Train: (800, 9)  |  Test: (200, 9)

print(f"Train churn rate: {y_train.mean():.1%}  |  Test churn rate: {y_test.mean():.1%}")
# Both should be ~30% — stratify= ensures this

Warning

Never fit any imputer, scaler, or encoder on the full dataset before splitting. If you do, your test set has already influenced the imputation medians and scaler means. That is data leakage — your test metrics will be optimistic and the model will underperform on real customers.

Step 2: Handle Missing Values¶

`age` — Median Imputation¶

Age is roughly symmetric across the 22–64 range. The 30 missing values are randomly distributed (confirmed in EDA). Median imputation is robust to any skew and adds minimal bias.

train_age_median = X_train["age"].median()
print(f"Training set age median: {train_age_median:.1f}")
# ~42.0

print(f"Age missing in train: {X_train['age'].isna().sum()}")
# ~24 (approximately 80% of the 30 total injected)

`support_calls` — Zero Imputation¶

Missing support call records almost certainly mean the customer never called — not that the data was lost. A customer with no recorded calls is a different behavioural profile from one with an unreported count.

# Confirm: is missingness random or correlated with churn?
print("Churn rate (support_calls missing):",
      df[df["support_calls"].isna()]["churn"].mean().round(3))
print("Churn rate (support_calls observed):",
      df[df["support_calls"].notna()]["churn"].mean().round(3))
# Both should be close to ~0.30 — missingness is random, not informative

Info

Imputing with 0 is a domain-informed choice, not a statistical guess. If customers with missing support_calls had a dramatically higher churn rate, it would suggest the missingness itself is a signal, and you should consider a calls_missing binary flag instead of zero imputation.

Step 3: Engineer New Features¶

Apply these transformations to training data first to understand the distributions, then bake them into the pipeline.

`calls_per_tenure` — Contact Intensity¶

Raw support call counts do not account for how long the customer has been around. Ten calls in 60 months is fine. Ten calls in two months signals serious friction.

calls_filled = X_train["support_calls"].fillna(0)
calls_per_tenure = calls_filled / (X_train["tenure_months"] + 1)

print(calls_per_tenure.describe().round(3))
#  min     0.000
#  25%     0.048
#  50%     0.110
#  75%     0.210
#  max     1.400

The + 1 in the denominator prevents division by zero for any hypothetical customer with tenure_months = 0.

`high_call_flag` — Binary Churn Risk Indicator¶

EDA showed that customers with 7 or more support calls in 90 days churn at ~52%. This threshold is the point where churn probability crosses 50%. A binary flag preserves this non-linear threshold effect explicitly.

calls_filled = X_train["support_calls"].fillna(0)
high_call_flag = (calls_filled >= 7).astype(int)

churn_rate_high = y_train[high_call_flag == 1].mean()
churn_rate_low  = y_train[high_call_flag == 0].mean()
print(f"Churn rate — high calls (>=7): {churn_rate_high:.1%}")   # ~52%
print(f"Churn rate — low calls  (< 7): {churn_rate_low:.1%}")    # ~17%

Tip

Threshold features like high_call_flag are valuable even with tree-based models. Trees can find splits on their own, but an explicit flag reduces the depth needed to capture this effect and makes feature importance output more interpretable.

`tenure_bucket` — Ordinal Lifecycle Stage¶

The relationship between tenure and churn is non-linear. The drop in churn risk is steep in the first 18 months, then levels off. Bucketing captures this shape better than treating tenure as a continuous linear predictor.

tenure_bucket = pd.cut(
    X_train["tenure_months"],
    bins=[0, 6, 18, 36, 72],
    labels=["new", "developing", "established", "loyal"]
)

print(pd.crosstab(tenure_bucket, y_train, normalize="index").round(3))
# tenure_months    0       1
# new            ~0.40   ~0.60   <- highest churn risk
# developing     ~0.62   ~0.38
# established    ~0.77   ~0.23
# loyal          ~0.87   ~0.13   <- lowest churn risk

`fee_per_product` — Value Density¶

A customer paying INR 1,200/month for 4 products is getting good value. A customer paying INR 1,000/month for 1 product may feel overcharged. Dividing fee by products captures this perceived-value ratio.

fee_per_product = X_train["monthly_fee"] / X_train["num_products"]
print(fee_per_product.describe().round(1))
# min ~40,  mean ~500,  max ~5000

Step 4: Encoding¶

`segment` — One-Hot (nominal, no natural order)¶

Three categories: Basic, Premium, Enterprise. No ordinal relationship. One-hot encoding produces three binary columns.

# Preview what one-hot looks like
pd.get_dummies(X_train["segment"].head(3), prefix="segment")
#    segment_Basic  segment_Enterprise  segment_Premium
# 0              1                   0                0
# 1              0                   0                1
# 2              1                   0                0

`region` — One-Hot (nominal, weak signal)¶

Four regions, no natural order. EDA confirmed region adds almost no signal. The model will assign low feature importance — confirming the EDA finding. Encode it correctly and let the model confirm it.

`tenure_bucket` — Ordinal (0, 1, 2, 3)¶

The lifecycle stages have a natural order: new < developing < established < loyal. Ordinal encoding preserves this and produces a single numeric column rather than four binary columns.

from sklearn.preprocessing import OrdinalEncoder

oe = OrdinalEncoder(categories=[["new", "developing", "established", "loyal"]])
sample = np.array(["new", "loyal", "developing", "established"]).reshape(-1, 1)
print(oe.fit_transform(sample).flatten())
# [0. 3. 1. 2.]

Step 5: Scaling¶

Logistic regression is sensitive to feature magnitude. StandardScaler centres and scales each numeric feature to mean=0, std=1. Tree-based models do not require scaling, but applying it consistently means the pipeline works with any classifier without modification.

Feature	Typical Range	Scaling applied
`age`	22–64	Yes — after median imputation
`tenure_months`	1–71	Yes
`support_calls`	0–14	Yes — after zero imputation in engineering step
`monthly_fee`	200–5,000	Yes — high variance across segments
`calls_per_tenure`	0.0–1.4	Yes — small values, needs normalisation
`fee_per_product`	40–5,000	Yes — high variance

Step 6: Build the sklearn Pipeline¶

A Pipeline applies all transformations in a defined order and ensures scaler and encoder parameters are learned only from training data. This is the pipeline you will attach a classifier to in the next file.

def add_engineered_features(X):
    """Add derived columns. Handles support_calls NaNs with zero fill."""
    X = X.copy()
    calls = X["support_calls"].fillna(0)
    X["calls_per_tenure"] = calls / (X["tenure_months"] + 1)
    X["high_call_flag"]   = (calls >= 7).astype(int)
    X["fee_per_product"]  = X["monthly_fee"] / X["num_products"]
    X["tenure_bucket"]    = pd.cut(
        X["tenure_months"],
        bins=[0, 6, 18, 36, 72],
        labels=["new", "developing", "established", "loyal"]
    ).astype(str)      # ColumnTransformer expects string, not Categorical dtype
    return X


# Apply engineering to both splits
X_train_eng = add_engineered_features(X_train)
X_test_eng  = add_engineered_features(X_test)

# --- define column groups after engineering ---
numeric_features  = ["age", "tenure_months", "support_calls",
                     "monthly_fee", "calls_per_tenure", "fee_per_product"]
binary_features   = ["has_contract", "high_call_flag"]
onehot_features   = ["segment", "region"]
ordinal_features  = ["tenure_bucket"]

# --- sub-pipelines ---
numeric_pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="median")),
    ("scaler",  StandardScaler()),
])

onehot_pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="most_frequent")),
    ("encoder", OneHotEncoder(handle_unknown="ignore", sparse_output=False)),
])

ordinal_pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="most_frequent")),
    ("encoder", OrdinalEncoder(
        categories=[["new", "developing", "established", "loyal"]],
        handle_unknown="use_encoded_value",
        unknown_value=-1,
    )),
])

# --- combine all sub-pipelines ---
preprocessor = ColumnTransformer([
    ("num",     numeric_pipeline,  numeric_features),
    ("bin",     "passthrough",     binary_features),
    ("onehot",  onehot_pipeline,   onehot_features),
    ("ordinal", ordinal_pipeline,  ordinal_features),
])

print("Preprocessor defined. Attach a classifier in model-building.md.")
# Verify shape after transformation
X_transformed = preprocessor.fit_transform(X_train_eng)
print(f"Transformed shape: {X_transformed.shape}")
# (800, 16)

Warning

Always call preprocessor.fit_transform(X_train_eng) during training and preprocessor.transform(X_test_eng) during evaluation — never fit_transform on the test set. The Pipeline class handles this automatically when you use pipeline.fit() and pipeline.predict() together, which is exactly why pipelines exist.

Step 7: Final Feature Set¶

After the ColumnTransformer, the model receives 16 input features:

#	Feature	Origin	Transformation
1	`age`	Raw	Median impute → StandardScaler
2	`tenure_months`	Raw	StandardScaler
3	`support_calls`	Raw	Median impute → StandardScaler
4	`monthly_fee`	Raw	StandardScaler
5	`calls_per_tenure`	Engineered	StandardScaler
6	`fee_per_product`	Engineered	StandardScaler
7	`has_contract`	Raw	Passthrough (already binary)
8	`high_call_flag`	Engineered	Passthrough (already binary)
9	`segment_Basic`	Raw	One-hot
10	`segment_Enterprise`	Raw	One-hot
11	`segment_Premium`	Raw	One-hot
12	`region_East`	Raw	One-hot
13	`region_North`	Raw	One-hot
14	`region_South`	Raw	One-hot
15	`region_West`	Raw	One-hot
16	`tenure_bucket`	Engineered	Ordinal (0–3)

16 input features from 9 raw columns plus 4 engineered features.

Success

A well-structured pipeline is your most important deliverable in a real project — not the model accuracy. The pipeline is what makes your work reproducible, reviewable, and deployable. A model sitting outside a pipeline is a model you cannot safely put into production.

← EDA | Next: Model Building →

Feature Engineering¶

Learning Objectives¶

Setup: Generate the Dataset¶

Step 1: Train-Test Split First¶

Step 2: Handle Missing Values¶

age — Median Imputation¶

support_calls — Zero Imputation¶

Step 3: Engineer New Features¶

calls_per_tenure — Contact Intensity¶

high_call_flag — Binary Churn Risk Indicator¶

tenure_bucket — Ordinal Lifecycle Stage¶

fee_per_product — Value Density¶