Exercises: Clustering Techniques

These exercises use realistic data and ask you to make decisions — not just run code. Each section builds on the previous one. The stretch exercise has no single correct answer, which is by design: clustering in practice rarely does.

---

Setup

Run this block first. All exercises use data generated here.

import pandas as pd
import numpy as np
from sklearn.cluster import KMeans, DBSCAN, AgglomerativeClustering
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import silhouette_score, davies_bouldin_score
from sklearn.datasets import make_moons, make_blobs
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt

np.random.seed(42)

# Customer behavioural dataset — 500 customers, 4 features
n = 500
customers = pd.DataFrame({
    "customer_id": range(1, n + 1),
    "monthly_spend": np.concatenate([
        np.random.normal(60, 15, 150),      # casual spenders
        np.random.normal(350, 60, 200),     # regular buyers
        np.random.normal(1800, 250, 120),   # high-value
        np.random.normal(4500, 400, 30)     # VIP / whale customers
    ]),
    "purchase_frequency": np.concatenate([
        np.random.normal(1.2, 0.4, 150),
        np.random.normal(5.5, 1.2, 200),
        np.random.normal(14, 2.5, 120),
        np.random.normal(22, 3, 30)
    ]),
    "tenure_months": np.concatenate([
        np.random.normal(2, 1, 150),
        np.random.normal(14, 4, 200),
        np.random.normal(38, 6, 120),
        np.random.normal(60, 8, 30)
    ]),
    "support_tickets": np.concatenate([
        np.random.normal(3, 1.5, 150),
        np.random.normal(1.5, 0.8, 200),
        np.random.normal(0.8, 0.4, 120),
        np.random.normal(0.3, 0.2, 30)
    ])
}).clip(lower=0)

print(f"Dataset shape: {customers.shape}")
print(customers.describe().round(1))

---

Warm-Up: Scale and Explore

Goal: Get comfortable with the data before clustering.

Task 1.1 — Inspect the distributions

feature_cols = ["monthly_spend", "purchase_frequency", "tenure_months", "support_tickets"]

# Compute coefficient of variation (std / mean) for each feature
# Features with high CV will dominate distance calculations if unscaled
for col in feature_cols:
    cv = customers[col].std() / customers[col].mean()
    print(f"{col}: mean={customers[col].mean():.1f}, std={customers[col].std():.1f}, CV={cv:.2f}")

Question: Which feature has the largest raw standard deviation? Without scaling, would this feature dominate clustering?

Show answer

monthly_spend has the largest standard deviation (several hundred). Without scaling, a difference of 1,000 in monthly spend would be treated as far more significant than a 10-month difference in tenure, even if both represent meaningful variation. StandardScaler brings all features to the same unit before clustering.

---

Task 1.2 — Scale the features

X = customers[feature_cols].values
X_scaled = StandardScaler().fit_transform(X)

# Verify scaling worked
scaled_df = pd.DataFrame(X_scaled, columns=feature_cols)
print(scaled_df.describe().round(2))
# All features should now have mean ≈ 0 and std ≈ 1

---

Task 1.3 — Quick elbow plot

inertias = []
k_range = range(2, 10)

for k in k_range:
    model = KMeans(n_clusters=k, random_state=42, n_init="auto")
    model.fit(X_scaled)
    inertias.append(model.inertia_)

plt.figure(figsize=(8, 4))
plt.plot(list(k_range), inertias, marker="o")
plt.xlabel("k")
plt.ylabel("Inertia")
plt.title("Elbow Plot")
plt.xticks(list(k_range))
plt.tight_layout()
plt.show()

Question: Does the elbow plot clearly suggest one value of k? If not, what does that tell you?

Show answer

The elbow is likely somewhere around k=3 or k=4 but may not be a sharp bend — real data rarely has a perfect elbow. When the curve is smooth, the elbow method gives you a range of candidates, not a definitive answer. This is exactly why silhouette score is a necessary complement.

---

Main Exercise: Customer Segmentation

Goal: Find the best k for this dataset, cluster the customers, and name each segment.

Task 2.1 — Silhouette search

silhouette_results = {}

for k in range(2, 9):
    labels = KMeans(n_clusters=k, random_state=42, n_init="auto").fit_predict(X_scaled)
    score = silhouette_score(X_scaled, labels)
    db = davies_bouldin_score(X_scaled, labels)
    silhouette_results[k] = score
    print(f"k={k}  silhouette={score:.4f}  davies_bouldin={db:.4f}")

best_k = max(silhouette_results, key=silhouette_results.get)
print(f"\nBest k by silhouette: {best_k}")

Question: Do silhouette score and Davies-Bouldin index agree on the best k?

Show answer

For this dataset with 4 underlying groups, both metrics should point toward k=4 (or close to it). Silhouette is highest at k=4; Davies-Bouldin is lowest at k=4. When they agree, you can proceed with more confidence. When they disagree, look at cluster profiles for each candidate k and ask which is more interpretable.

---

Task 2.2 — Final segmentation

best_k = 4  # use your finding from Task 2.1

final_model = KMeans(n_clusters=best_k, random_state=42, n_init="auto")
customers["cluster"] = final_model.fit_predict(X_scaled)

# Profile each cluster
profile = customers.groupby("cluster").agg(
    avg_spend=("monthly_spend", "mean"),
    avg_frequency=("purchase_frequency", "mean"),
    avg_tenure=("tenure_months", "mean"),
    avg_tickets=("support_tickets", "mean"),
    count=("customer_id", "count")
).round(1)

print(profile)

Task 2.3 — Name the segments

Based on the cluster profiles, assign a business name to each cluster. Use names a marketing manager would understand (e.g., "Occasional Buyers", "Loyal Mid-Tier", "VIP", "New Customers").

# Map cluster numbers to names based on your profile analysis
# Example (adjust based on your actual cluster numbers):
segment_names = {
    0: "YOUR_NAME_HERE",
    1: "YOUR_NAME_HERE",
    2: "YOUR_NAME_HERE",
    3: "YOUR_NAME_HERE"
}

customers["segment"] = customers["cluster"].map(segment_names)
print(customers["segment"].value_counts())

Show answer

Typical segments from this data: - High support_tickets, low spend, short tenure → "New/At-Risk Customers" - Moderate spend, moderate frequency, medium tenure → "Regular Buyers" - High spend, high frequency, long tenure → "High-Value Loyal" - Very high spend, very long tenure, very low tickets → "VIP / Whale Customers"

Remember: cluster numbers (0, 1, 2, 3) are arbitrary. What matters is the profile of each group.

---

Task 2.4 — Visualise with PCA

pca = PCA(n_components=2, random_state=42)
X_2d = pca.fit_transform(X_scaled)

plt.figure(figsize=(8, 6))
scatter = plt.scatter(X_2d[:, 0], X_2d[:, 1],
                      c=customers["cluster"], cmap="tab10", alpha=0.6)
plt.colorbar(scatter, label="Cluster")
plt.xlabel(f"PC1 ({pca.explained_variance_ratio_[0]:.1%} variance)")
plt.ylabel(f"PC2 ({pca.explained_variance_ratio_[1]:.1%} variance)")
plt.title("Customer Segments — PCA Projection")
plt.tight_layout()
plt.show()

Question: Do the clusters look well-separated in PCA space? What does it mean if two clusters overlap in this 2D view?

Show answer

Overlap in PCA space does not necessarily mean bad clustering. PCA projects 4D data to 2D, which loses information. Two clusters may overlap in 2D but be well-separated in the original 4D feature space. Always check the silhouette score in the original space — do not judge clustering quality from PCA plots alone.

---

Comparison Exercise: K-Means vs DBSCAN on Non-Spherical Data

Goal: Understand when K-Means fails and DBSCAN succeeds.

Task 3.1 — Create crescent-shaped data

X_moons, true_labels = make_moons(n_samples=400, noise=0.08, random_state=42)
X_moons_scaled = StandardScaler().fit_transform(X_moons)

Task 3.2 — Apply both algorithms

from sklearn.metrics import adjusted_rand_score

# K-Means
kmeans_labels = KMeans(n_clusters=2, random_state=42, n_init="auto").fit_predict(X_moons_scaled)

# DBSCAN — tune eps using the k-distance approach or try eps=0.3
dbscan_labels = DBSCAN(eps=0.3, min_samples=5).fit_predict(X_moons_scaled)

# Evaluate against true labels (we have them here — normally you would not)
kmeans_ari = adjusted_rand_score(true_labels, kmeans_labels)
dbscan_ari = adjusted_rand_score(true_labels, dbscan_labels)

kmeans_sil = silhouette_score(X_moons_scaled, kmeans_labels)
dbscan_noise = list(dbscan_labels).count(-1)
dbscan_clusters = len(set(dbscan_labels)) - (1 if -1 in dbscan_labels else 0)

print(f"K-Means:  ARI={kmeans_ari:.3f}  silhouette={kmeans_sil:.3f}")
print(f"DBSCAN:   ARI={dbscan_ari:.3f}  clusters={dbscan_clusters}  noise={dbscan_noise}")

Task 3.3 — Plot the comparison

fig, axes = plt.subplots(1, 3, figsize=(15, 4))

for ax, labels, title in zip(
    axes,
    [true_labels, kmeans_labels, dbscan_labels],
    ["True Labels", "K-Means (k=2)", "DBSCAN"]
):
    ax.scatter(X_moons_scaled[:, 0], X_moons_scaled[:, 1],
               c=labels, cmap="tab10", alpha=0.7, s=15)
    ax.set_title(title)
    ax.set_xticks([])
    ax.set_yticks([])

plt.suptitle("Crescent Data: K-Means vs DBSCAN")
plt.tight_layout()
plt.show()

Question: K-Means may produce a reasonable silhouette score on this data even though its cluster assignments are wrong. What does that tell you about silhouette score as a metric?

Show answer

Silhouette score measures compactness and separation of the clusters that were formed — not whether those clusters match any real structure. K-Means cuts the crescents in half, creating two compact half-moon shapes. These may be moderately compact (decent silhouette), but they are geometrically wrong. This is why you should always visualise your clustering results and not rely solely on a single metric.

---

Stretch Exercise: Hierarchical Clustering + Dendrogram-Informed K

Goal: Use a dendrogram to select k, then validate with silhouette score, and compare to K-Means on the same data.

Task 4.1 — Sample the customer data and plot a dendrogram

from scipy.cluster.hierarchy import linkage, dendrogram, fcluster

# Use a subsample — full 500 rows makes a crowded dendrogram
sample_idx = np.random.choice(len(X_scaled), size=80, replace=False)
X_sample = X_scaled[sample_idx]

linked = linkage(X_sample, method="ward")

plt.figure(figsize=(14, 5))
dendrogram(linked, color_threshold=5.0, above_threshold_color="grey", no_labels=True)
plt.axhline(y=5.0, color="red", linestyle="--", label="Candidate cut")
plt.xlabel("Samples")
plt.ylabel("Merge distance (Ward)")
plt.title("Dendrogram — 80-sample subset of customer data")
plt.legend()
plt.tight_layout()
plt.show()

Task 4.2 — Extract clusters from the dendrogram

# Experiment with different heights. Start at 5.0 and adjust.
cut_height = 5.0
hier_labels = fcluster(linked, t=cut_height, criterion="distance")
n_hier_clusters = len(set(hier_labels))
print(f"Clusters at height {cut_height}: {n_hier_clusters}")

# Compute silhouette for the hierarchical result
if n_hier_clusters > 1:
    sil = silhouette_score(X_sample, hier_labels)
    print(f"Silhouette: {sil:.4f}")

Task 4.3 — Compare to K-Means on the same sample

# Run K-Means with the k you found from the dendrogram
km_labels_sample = KMeans(n_clusters=n_hier_clusters, random_state=42, n_init="auto").fit_predict(X_sample)
km_sil = silhouette_score(X_sample, km_labels_sample)
print(f"K-Means silhouette (same k): {km_sil:.4f}")

Questions to answer:

1. How many clusters did the dendrogram suggest? Does it match what you found with silhouette search earlier?
2. Which has a higher silhouette score on this sample — hierarchical or K-Means?
3. Given that hierarchical clustering is O(n²) and K-Means is approximately O(n), which would you choose for the full 500-row dataset? What about a 100,000-row dataset?

Show answer

Q1: The dendrogram on this dataset should suggest 3–5 clusters depending on where the longest vertical lines fall. This often aligns with the silhouette search finding from the main exercise.

Q2: K-Means often achieves a slightly higher silhouette score on blob-like data because it explicitly optimises inertia. Hierarchical clustering makes greedy merges and cannot undo a bad early decision. However, hierarchical clustering with Ward linkage is competitive and often gives very similar results to K-Means on this type of data.

Q3: For 500 rows, either is practical. For 100,000 rows, K-Means is the only realistic choice — hierarchical would require ~40 GB of memory to store the distance matrix. A common pattern is: use hierarchical on a 500-row sample to identify k, then apply K-Means to the full dataset with that k.

---

Summary Checklist

Before you move on, make sure you can do all of these from memory:

• [ ] Scale features with StandardScaler before any distance-based algorithm
• [ ] Run the elbow plot and silhouette search to select k for K-Means
• [ ] Explain the difference between silhouette score and inertia
• [ ] Run DBSCAN and interpret the -1 noise label
• [ ] Use the k-distance plot to choose eps for DBSCAN
• [ ] Read a dendrogram and identify a reasonable cut height
• [ ] Profile cluster results with groupby().agg() and give clusters meaningful names
• [ ] Use PCA to visualise high-dimensional clusters (and explain why this is only for visualisation)

---

← Evaluation and Scaling | Next: Feature Engineering →