Exercises: Machine Learning Basics

These exercises test understanding, not just recall. Most have a coding component and a reasoning component. The stretch exercises are genuinely hard — they require combining ideas from multiple sections. Work through them in order.

---

Warm-Up Exercises

W1: Map the problem

For each scenario, state: the learning type (supervised/unsupervised), the task type (regression/classification/clustering/dimensionality reduction), and identify the target variable (or state "no target").

Scenario	Type	Task	Target
Predict next month's electricity bill from historical usage	?	?	?
Detect whether a bank transaction is fraudulent	?	?	?
Group 10,000 news articles into topic buckets	?	?	?
Predict whether a patient will be readmitted within 30 days	?	?	?
Compress satellite images from 256 bands to 10	?	?	?
Estimate the selling price of a used car	?	?	?

Show answers

Scenario	Type	Task	Target
Electricity bill prediction	Supervised	Regression	Bill amount (£)
Fraud detection	Supervised	Classification (binary)	Fraud: yes/no
News article grouping	Unsupervised	Clustering	No target
Patient readmission	Supervised	Classification (binary)	Readmitted: yes/no
Satellite image compression	Unsupervised	Dimensionality Reduction	No target
Used car price	Supervised	Regression	Sale price (£)

---

W2: Spot the leakage

Each dataset below has at least one leaky feature. Identify it and explain why it is leaky — what information does it encode that would not be available at real prediction time?

Dataset A: Predicting customer churn (will they cancel in the next 30 days?)

Features: tenure_months, monthly_spend, num_support_calls, cancellation_date, plan_type, last_login_days_ago

Dataset B: Predicting loan default

Features: income, credit_score, loan_amount, employment_years, collections_referred, monthly_payment_missed_count

Dataset C: Predicting student exam pass/fail

Features: hours_studied, attendance_pct, prior_gpa, exam_score, tutor_sessions

Show answers

Dataset A: cancellation_date is leaky. It only exists for customers who have already cancelled. At prediction time, you are asking "will this customer cancel?" — you cannot know their cancellation date because they have not cancelled yet. Including it lets the model use a feature that is created by the very event you are trying to predict.

Dataset B: collections_referred is likely leaky (it depends on timing — if it is populated after default occurs, it is using the outcome to predict itself). monthly_payment_missed_count needs scrutiny too: if this count includes the current period and you are predicting whether they will default this period, it may be partially leaky depending on how the data was constructed.

Dataset C: exam_score is the label, not a feature. Including it is not leakage — it is an error: you would be predicting pass/fail from the score that determines pass/fail. In practice this means the model trivially learns "if score > 50, pass" — which is the definition of pass. You have accidentally included the target as a feature.

---

W3: Correct the split

The code below has a leakage error. Identify it, explain why it causes leakage, and rewrite it correctly.

from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_breast_cancer

data = load_breast_cancer()
X, y = data.data, data.target

# Preprocessing
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)  # ERROR IS HERE

# Split
X_train, X_test, y_train, y_test = train_test_split(
    X_scaled, y, test_size=0.2, random_state=42
)

# Train
model = LogisticRegression()
model.fit(X_train, y_train)
print(model.score(X_test, y_test))

Show answer

The error: scaler.fit_transform(X) is called before the split. This means the scaler computes mean and standard deviation using the test-set rows. The test data has influenced the training process — that is leakage.

Corrected version:

from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_breast_cancer

data = load_breast_cancer()
X, y = data.data, data.target

# Split FIRST — test set is untouched from this point
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# Scale AFTER splitting — fit on train only
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)  # fit+transform on train
X_test_scaled  = scaler.transform(X_test)        # transform only on test

model = LogisticRegression(max_iter=10000)
model.fit(X_train_scaled, y_train)
print(model.score(X_test_scaled, y_test))

# Better still: use a Pipeline so this cannot go wrong
from sklearn.pipeline import Pipeline

pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("model",  LogisticRegression(max_iter=10000)),
])
pipe.fit(X_train, y_train)
print(pipe.score(X_test, y_test))

---

Main Exercises

M1: Build a pipeline from scratch

Build a complete end-to-end ML pipeline using the wine quality dataset. The dataset has both numeric features and a target that can be treated as binary (quality >= 7 → good wine).

Requirements:

1. Load the data and create a binary target
2. Perform a stratified 80/20 train/test split
3. Build a preprocessing pipeline: impute missing values with median, then scale
4. Attach a LogisticRegression model to the pipeline
5. Evaluate with 5-fold stratified cross-validation (metric: AUC)
6. Report: CV mean AUC, CV std AUC, and the confusion matrix on the test set

import pandas as pd
import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split, cross_val_score, StratifiedKFold
from sklearn.metrics import confusion_matrix, roc_auc_score, classification_report

# Load data
url = "https://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv"

# Offline fallback: generate synthetic version
np.random.seed(42)
n = 1599
wine = pd.DataFrame({
    "fixed_acidity":        np.random.normal(8.3, 1.7, n).round(1),
    "volatile_acidity":     np.random.normal(0.53, 0.18, n).round(2),
    "citric_acid":          np.random.normal(0.27, 0.19, n).round(2),
    "residual_sugar":       np.random.exponential(2.5, n).round(1),
    "chlorides":            np.random.normal(0.087, 0.047, n).round(3),
    "free_sulfur_dioxide":  np.random.normal(15.9, 10.5, n).round(0),
    "total_sulfur_dioxide": np.random.normal(46.5, 32.9, n).round(0),
    "density":              np.random.normal(0.997, 0.002, n).round(4),
    "pH":                   np.random.normal(3.31, 0.15, n).round(2),
    "sulphates":            np.random.normal(0.66, 0.17, n).round(2),
    "alcohol":              np.random.normal(10.4, 1.07, n).round(1),
    "quality":              np.random.choice(range(3, 9), n,
                                p=[0.006, 0.053, 0.426, 0.399, 0.104, 0.011]),
})

# YOUR CODE HERE
# Step 1: Create binary target (quality >= 7 → good wine = 1)

# Step 2: Define X and y, perform stratified 80/20 split

# Step 3: Build preprocessing Pipeline

# Step 4: Attach LogisticRegression

# Step 5: Cross-validate

# Step 6: Evaluate on test set

Show answer

import pandas as pd
import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split, cross_val_score, StratifiedKFold
from sklearn.metrics import confusion_matrix, roc_auc_score, classification_report

np.random.seed(42)
n = 1599
wine = pd.DataFrame({
    "fixed_acidity":        np.random.normal(8.3, 1.7, n).round(1),
    "volatile_acidity":     np.random.normal(0.53, 0.18, n).round(2),
    "citric_acid":          np.random.normal(0.27, 0.19, n).round(2),
    "residual_sugar":       np.random.exponential(2.5, n).round(1),
    "chlorides":            np.random.normal(0.087, 0.047, n).round(3),
    "free_sulfur_dioxide":  np.random.normal(15.9, 10.5, n).round(0),
    "total_sulfur_dioxide": np.random.normal(46.5, 32.9, n).round(0),
    "density":              np.random.normal(0.997, 0.002, n).round(4),
    "pH":                   np.random.normal(3.31, 0.15, n).round(2),
    "sulphates":            np.random.normal(0.66, 0.17, n).round(2),
    "alcohol":              np.random.normal(10.4, 1.07, n).round(1),
    "quality":              np.random.choice(range(3, 9), n,
                                p=[0.006, 0.053, 0.426, 0.399, 0.104, 0.011]),
})

# Step 1: Binary target
wine["good_wine"] = (wine["quality"] >= 7).astype(int)
print("Class balance:")
print(wine["good_wine"].value_counts())
# Output:
# 0    1483
# 1     116

# Step 2: Split
X = wine.drop(columns=["quality", "good_wine"])
y = wine["good_wine"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

# Step 3 + 4: Pipeline
pipe = Pipeline([
    ("imputer",    SimpleImputer(strategy="median")),
    ("scaler",     StandardScaler()),
    ("classifier", LogisticRegression(max_iter=10000, random_state=42,
                                      class_weight="balanced")),
])

# Step 5: Cross-validate
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
cv_scores = cross_val_score(pipe, X_train, y_train, cv=cv, scoring="roc_auc")

print(f"\nCV AUC: {cv_scores.mean():.4f} ± {cv_scores.std():.4f}")
print(f"Fold scores: {cv_scores.round(4)}")

# Step 6: Final evaluation
pipe.fit(X_train, y_train)
y_pred  = pipe.predict(X_test)
y_proba = pipe.predict_proba(X_test)[:, 1]

print(f"\nTest AUC: {roc_auc_score(y_test, y_proba):.4f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred, target_names=["Not good", "Good"]))
print("Confusion Matrix:")
print(confusion_matrix(y_test, y_pred))

Note the use of class_weight="balanced" — with only ~7% positive class, the model will predict "not good" for everything if you do not account for imbalance. Balanced weighting makes the model give more penalty to false negatives.

---

M2: Diagnose a broken experiment

The code below is running an ML experiment, but the results are suspicious. Find all the problems, explain what each one does wrong, and produce a corrected version that gives honest results.

import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.preprocessing import MinMaxScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import KFold, cross_val_score

data = load_breast_cancer()
X = data.data
y = data.target

# Preprocessing
scaler = MinMaxScaler()
X_scaled = scaler.fit_transform(X)

# Cross-validation
kf = KFold(n_splits=5)
model = KNeighborsClassifier(n_neighbors=3)
scores = cross_val_score(model, X_scaled, y, cv=kf)

print(f"Mean accuracy: {scores.mean():.4f}")  # Output: 0.9649
print(f"This model is ready for production!")

Show answer

Problem 1: Scaling before cross-validation (leakage) scaler.fit_transform(X) uses all 569 samples to compute min/max. When cross-validation then holds out one fold as validation, that fold's data was already used to fit the scaler. The validation data has leaked into preprocessing.

Problem 2: Using KFold instead of StratifiedKFold for classification KFold splits randomly without preserving class proportions. For a dataset with class imbalance, some folds may have disproportionate class representation, giving noisy and non-comparable fold scores.

Problem 3: Over-interpreting a single metric Accuracy is reported without checking class balance. Even if accuracy is honest, it can be misleading for imbalanced datasets. At minimum, report precision, recall, and AUC alongside accuracy.

Problem 4: "Ready for production" after cross-validation Cross-validation on one dataset does not mean production-ready. You still need to: test on a held-out test set, check performance on different data slices, assess calibration, and monitor post-deployment drift.

Corrected code:

import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.preprocessing import MinMaxScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import StratifiedKFold, cross_val_score, train_test_split
from sklearn.pipeline import Pipeline
from sklearn.metrics import classification_report, roc_auc_score

data = load_breast_cancer()
X, y = data.data, data.target

# Split FIRST — hold out a true test set
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

# Fix 1: Put scaler inside a Pipeline so cross-validation handles it correctly
# Fix 2: Use StratifiedKFold to preserve class balance per fold
pipe = Pipeline([
    ("scaler", MinMaxScaler()),
    ("model",  KNeighborsClassifier(n_neighbors=3)),
])

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)

# Fix 3: Report multiple metrics
acc_scores = cross_val_score(pipe, X_train, y_train, cv=cv, scoring="accuracy")
auc_scores = cross_val_score(pipe, X_train, y_train, cv=cv, scoring="roc_auc")

print(f"CV Accuracy: {acc_scores.mean():.4f} ± {acc_scores.std():.4f}")
print(f"CV AUC:      {auc_scores.mean():.4f} ± {auc_scores.std():.4f}")

# Final evaluation on held-out test set
pipe.fit(X_train, y_train)
y_pred  = pipe.predict(X_test)
y_proba = pipe.predict_proba(X_test)[:, 1]

print("\nTest Set Report:")
print(classification_report(y_test, y_pred, target_names=["malignant", "benign"]))
print(f"Test AUC: {roc_auc_score(y_test, y_proba):.4f}")

---

M3: Cross-validation output interpretation

A colleague runs cross-validation on three models and shares the results below. Based only on this output, answer the questions that follow.

Model A — RandomForestClassifier:
  Fold scores: [0.9123, 0.9241, 0.9198, 0.9087, 0.9156]
  Mean: 0.9161 ± 0.0053

Model B — KNeighborsClassifier:
  Fold scores: [0.8812, 0.9501, 0.8334, 0.9712, 0.8201]
  Mean: 0.8912 ± 0.0598

Model C — LogisticRegression:
  Fold scores: [0.9998, 0.9997, 0.9999, 0.9998, 0.9997]
  Mean: 0.9998 ± 0.0001

1. Which model would you choose for deployment, and why?
2. What would you investigate about Model B before ruling it out?
3. What would you investigate about Model C before trusting it?
4. Model A's worst fold scores 0.9087. Model B's best fold scores 0.9712. Does that mean Model B can outperform Model A? Explain.

Show answers

1. Choose Model A. It has the highest mean with very low variance. Low variance (0.0053) means the model performs consistently across different data subsets — that consistency is a strong signal of genuine generalisation. Model B has higher variance, meaning performance is unstable. Model C's results are suspicious.

2. Model B's high variance (0.0598) needs investigation. Questions to ask: Is the dataset small, making fold-to-fold variation large? Is there class imbalance causing some folds to have very few positive examples? Is KNN sensitive to the particular samples in each fold because of outliers? You might try a larger k (e.g. k=15 instead of k=3 to reduce variance), more folds (10-fold instead of 5), or a repeated K-fold to get a more stable estimate.

3. Model C's near-perfect scores (0.9998) are a red flag. On a real-world classification problem, near-perfect accuracy suggests: data leakage (a feature is derived from the target), a trivially easy problem (maybe the classes are separable by a single feature), label encoding errors (the target is accidentally included as a feature), or the dataset is synthetic and too clean. Before trusting this result, audit every feature for leakage, check the confusion matrix, and verify the dataset source.

4. No, that does not mean Model B can outperform Model A in general. Model B's best fold (0.9712) was on a particular data subset that happened to favour KNN. Model A's worst fold (0.9087) was its weakest subset. The fold conditions differ — you cannot compare across folds of different models because each fold contains different data. What matters is the expected performance across all possible data — which is estimated by the cross-validation mean. Model A's mean (0.9161) is reliably above Model B's mean (0.8912).

---

Stretch Exercises

S1: Mixed-type pipeline with multiple algorithms

Build a model comparison experiment on a dataset that has both numeric and categorical features. Use ColumnTransformer for preprocessing. Compare at least three algorithms. Select the best based on cross-validation AUC, then report final performance on a test set.

import pandas as pd
import numpy as np
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.model_selection import train_test_split, cross_val_score, StratifiedKFold
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import roc_auc_score, classification_report

# Synthetic HR attrition dataset — employee leaving or not
np.random.seed(42)
n = 1500

hr = pd.DataFrame({
    "age":                  np.random.randint(22, 60, n),
    "monthly_income":       np.random.normal(6500, 2500, n).clip(1000).round(0),
    "years_at_company":     np.random.randint(0, 25, n),
    "years_in_role":        np.random.randint(0, 15, n),
    "num_companies_worked": np.random.randint(0, 9, n),
    "overtime":             np.random.choice(["Yes", "No"], n, p=[0.28, 0.72]),
    "job_satisfaction":     np.random.choice(["Low","Medium","High","Very High"], n),
    "department":           np.random.choice(["Sales","R&D","HR"], n, p=[0.37, 0.47, 0.16]),
    "education_field":      np.random.choice(
                                ["Life Sciences","Medical","Marketing","Technical","Other"],
                                n, p=[0.41, 0.32, 0.11, 0.09, 0.07]),
    "distance_from_home":   np.random.exponential(9, n).round(0).clip(1, 29),
    "attrition":            np.random.choice([0, 1], n, p=[0.84, 0.16]),
})

# Introduce some missing values (realistic)
hr.loc[np.random.choice(hr.index, 50, replace=False), "monthly_income"] = np.nan
hr.loc[np.random.choice(hr.index, 30, replace=False), "job_satisfaction"] = np.nan

# YOUR CODE HERE
# 1. Define numeric_cols and categorical_cols
# 2. Build a ColumnTransformer preprocessor
# 3. Wrap three candidate models in Pipelines
# 4. Cross-validate each with StratifiedKFold(5), metric = roc_auc
# 5. Select the best model
# 6. Refit the best model on full train set, evaluate on test set
# 7. Print: which model won, CV scores, and final test AUC

Show answer

import pandas as pd
import numpy as np
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.model_selection import train_test_split, cross_val_score, StratifiedKFold
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import roc_auc_score, classification_report

np.random.seed(42)
n = 1500
hr = pd.DataFrame({
    "age":                  np.random.randint(22, 60, n),
    "monthly_income":       np.random.normal(6500, 2500, n).clip(1000).round(0),
    "years_at_company":     np.random.randint(0, 25, n),
    "years_in_role":        np.random.randint(0, 15, n),
    "num_companies_worked": np.random.randint(0, 9, n),
    "overtime":             np.random.choice(["Yes", "No"], n, p=[0.28, 0.72]),
    "job_satisfaction":     np.random.choice(["Low","Medium","High","Very High"], n),
    "department":           np.random.choice(["Sales","R&D","HR"], n, p=[0.37, 0.47, 0.16]),
    "education_field":      np.random.choice(
                                ["Life Sciences","Medical","Marketing","Technical","Other"],
                                n, p=[0.41, 0.32, 0.11, 0.09, 0.07]),
    "distance_from_home":   np.random.exponential(9, n).round(0).clip(1, 29),
    "attrition":            np.random.choice([0, 1], n, p=[0.84, 0.16]),
})
hr.loc[np.random.choice(hr.index, 50, replace=False), "monthly_income"] = np.nan
hr.loc[np.random.choice(hr.index, 30, replace=False), "job_satisfaction"] = np.nan

# Step 1: Define features by type
X = hr.drop(columns=["attrition"])
y = hr["attrition"]

numeric_cols     = ["age", "monthly_income", "years_at_company",
                    "years_in_role", "num_companies_worked", "distance_from_home"]
categorical_cols = ["overtime", "job_satisfaction", "department", "education_field"]

# Step 2: Split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

# Step 3: ColumnTransformer
preprocessor = ColumnTransformer([
    ("num", Pipeline([
        ("imputer", SimpleImputer(strategy="median")),
        ("scaler",  StandardScaler()),
    ]), numeric_cols),
    ("cat", Pipeline([
        ("imputer", SimpleImputer(strategy="most_frequent")),
        ("encoder", OneHotEncoder(handle_unknown="ignore", sparse_output=False)),
    ]), categorical_cols),
])

# Step 4: Candidate models
candidates = {
    "Logistic Regression": LogisticRegression(
        max_iter=10000, class_weight="balanced", random_state=42),
    "Random Forest": RandomForestClassifier(
        n_estimators=100, class_weight="balanced", random_state=42),
    "Gradient Boosting": GradientBoostingClassifier(
        n_estimators=100, random_state=42),
}

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
results = {}

print("Model Comparison (CV AUC):")
for name, clf in candidates.items():
    pipe = Pipeline([("preprocessor", preprocessor), ("classifier", clf)])
    scores = cross_val_score(pipe, X_train, y_train, cv=cv, scoring="roc_auc")
    results[name] = {"scores": scores, "mean": scores.mean(), "std": scores.std()}
    print(f"  {name:<25} {scores.mean():.4f} ± {scores.std():.4f}")

# Step 5: Select best
best_name = max(results, key=lambda k: results[k]["mean"])
print(f"\nBest model: {best_name}")

# Step 6: Retrain on full training set, evaluate on test
best_pipe = Pipeline([
    ("preprocessor", preprocessor),
    ("classifier",   candidates[best_name]),
])
best_pipe.fit(X_train, y_train)
y_pred  = best_pipe.predict(X_test)
y_proba = best_pipe.predict_proba(X_test)[:, 1]

print(f"\nFinal Test AUC: {roc_auc_score(y_test, y_proba):.4f}")
print(classification_report(y_test, y_pred, target_names=["Stayed", "Left"]))

---

S2: Design a leakage-free time series split

You are building a model to predict whether a customer will churn in the next 30 days. The dataset has 24 months of records per customer. Describe (in prose + pseudocode) a correct train/test split strategy that:

1. Avoids temporal leakage (future data in training)
2. Avoids customer leakage (the same customer appearing in both train and test)
3. Provides a realistic estimate of how the model would perform when deployed

Show answer

This problem has two leakage axes: time and customer identity.

The correct approach: temporal split with customer-level separation

# Pseudocode

# 1. For each customer-month, define:
#    - features: all data from months [t-12, t-1] for that customer
#    - target: did the customer churn in month [t+1, t+30]?

# 2. Choose a cutoff date for training vs testing
#    Training:  label dates before cutoff (e.g. before month 22)
#    Testing:   label dates at/after cutoff (e.g. months 22–24)
#    This ensures: no future feature data leaks into training labels

# 3. Check for customer overlap
#    A customer active across the boundary appears in both sets.
#    Two options:
#    Option A: exclude customers who span the boundary (cleaner, more conservative)
#    Option B: allow overlap but ensure their training rows
#              use data before cutoff only (more data, needs careful implementation)

# 4. Feature computation
#    For training rows: compute features using data strictly before the label date
#    For testing rows:  compute features using data strictly before the label date
#    Never use post-label data in features

# 5. Preprocessing
#    Fit all scalers/encoders on training rows only
#    Apply to test rows

# Implementation sketch
cutoff = pd.Timestamp("2024-10-01")

train_df = events_df[events_df["label_date"] < cutoff]
test_df  = events_df[events_df["label_date"] >= cutoff]

# Optionally exclude customers in both sets
train_customers = set(train_df["customer_id"])
test_customers  = set(test_df["customer_id"])
overlap = train_customers & test_customers
test_df = test_df[~test_df["customer_id"].isin(overlap)]  # Option A

Why this matters for deployment: The model, when live, will receive customer data up to "today" and predict churn for the next 30 days. The test set simulates exactly this: the model sees features from before the cutoff and predicts outcomes after it. If customers span the boundary, there is a risk the model learns patterns specific to those long-tenure customers that do not generalise.

---

Self-Check

Before moving on, verify you can do each of these without looking at the notes:

• [ ] Explain generalisation in your own words
• [ ] Map any business problem to supervised/unsupervised and classification/regression
• [ ] Write a correct stratified train/test split from memory
• [ ] Identify three forms of data leakage and explain what each one causes
• [ ] Build a Pipeline with an imputer, scaler, and logistic regression
• [ ] Run cross_val_score with StratifiedKFold and interpret the output
• [ ] Name two metrics better than accuracy for imbalanced classification

---

Previous: Scikit-learn Workflow | Next: Regression Algorithms