Skip to content

Exploratory Data Analysis

EDA is not just plotting histograms. It is the process of building a mental model of your data before you touch a classifier. Every plot you make should answer a question. Every question should connect to the business problem: who churns, and why?

Setup: Generate the Dataset

Paste this block at the top of your notebook before anything else.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

rng = np.random.default_rng(42)
n = 1000

tenure = rng.integers(1, 72, n)
age = rng.integers(22, 65, n).astype(float)
segment = rng.choice(["Basic", "Premium", "Enterprise"], n, p=[0.55, 0.35, 0.10])
region = rng.choice(["North", "South", "East", "West"], n)
support_calls = rng.integers(0, 15, n).astype(float)
monthly_fee = np.where(
    segment == "Enterprise", rng.uniform(2000, 5000, n),
    np.where(segment == "Premium", rng.uniform(800, 1500, n),
             rng.uniform(200, 600, n))
)
num_products = rng.integers(1, 6, n)
has_contract = rng.choice([0, 1], n, p=[0.4, 0.6])

churn_prob = (
    0.55
    - tenure * 0.008
    + support_calls * 0.04
    - (segment == "Premium") * 0.12
    - (segment == "Enterprise") * 0.20
    - has_contract * 0.18
    + rng.normal(0, 0.05, n)
)
churn_prob = np.clip(churn_prob, 0.02, 0.95)
churn = (rng.uniform(size=n) < churn_prob).astype(int)

df = pd.DataFrame({
    "customer_id": range(10001, 10001 + n),
    "age": age,
    "tenure_months": tenure,
    "segment": segment,
    "region": region,
    "support_calls": support_calls,
    "monthly_fee": monthly_fee.round(2),
    "num_products": num_products,
    "has_contract": has_contract,
    "churn": churn,
})

df.loc[rng.choice(df.index, 30, replace=False), "age"] = np.nan
df.loc[rng.choice(df.index, 15, replace=False), "support_calls"] = np.nan

print(df.shape)        # (1000, 10)
print(df.dtypes)

Step 1: First Look

df.head()

df.describe()
# Check: tenure_months range 1–71, monthly_fee is trimodal (200–600 / 800–1500 / 2000–5000)

df.info()
# Confirms age and support_calls are float due to planned NaN injection

Step 2: Target Distribution

The first question for any classification problem: how balanced is the target?

churn_counts = df['churn'].value_counts()
churn_rate = df['churn'].mean()

print(churn_counts)
# 0    ~700
# 1    ~300

print(f"Churn rate: {churn_rate:.1%}")
# Churn rate: ~30.0%

fig, ax = plt.subplots(figsize=(6, 4))
bars = ax.bar(['Retained (0)', 'Churned (1)'],
              churn_counts.values,
              color=['#2DD4BF', '#F59E0B'],
              edgecolor='white', linewidth=0.8)

for bar, count in zip(bars, churn_counts.values):
    ax.text(bar.get_x() + bar.get_width() / 2, bar.get_height() + 8,
            f'{count}\n({count/n:.0%})', ha='center', va='bottom', fontsize=11)

ax.set_title('Target Distribution: Churn vs Retained', fontsize=13, pad=12)
ax.set_ylabel('Customer Count')
ax.set_ylim(0, 850)
plt.tight_layout()
plt.show()

Info

A 30% churn rate is moderate class imbalance. A naive classifier that always predicts "retained" would score 70% accuracy. That is your baseline to beat — and you will beat it meaningfully using recall as your primary metric.

Step 3: Missing Values Audit

missing = df.isnull().sum()
missing_pct = (missing / len(df) * 100).round(1)

missing_report = pd.DataFrame({
    'missing_count': missing[missing > 0],
    'missing_pct': missing_pct[missing > 0]
})
print(missing_report)
#                  missing_count  missing_pct
# age                         30          3.0
# support_calls               15          1.5

Decision for age: median imputation. Age is roughly symmetric (22–64); median and mean are close. Median is more robust to any future outliers.

Decision for support_calls: impute with 0. A missing support_calls record most likely means the customer never called — not that the data was lost. Imputing 0 is a domain-informed choice, not a statistical assumption. See feature-engineering for the full imputation pipeline.

Warning

Check whether missingness is random before deciding on imputation strategy. If customers who churned are more likely to have missing support_calls, then imputing 0 could understate their friction signal and hurt recall.

# Check whether missingness correlates with churn
print("Churn rate where age is missing:",
      df[df['age'].isna()]['churn'].mean().round(3))
print("Churn rate where age is not missing:",
      df[df['age'].notna()]['churn'].mean().round(3))
# Expect values close to each other (~0.30) — missingness is random here

Step 4: Univariate Distributions

Tenure (right-skewed)

fig, axes = plt.subplots(1, 2, figsize=(12, 4))

axes[0].hist(df['tenure_months'], bins=30, color='#0D9488', edgecolor='white', linewidth=0.5)
axes[0].set_title('Tenure Distribution')
axes[0].set_xlabel('Tenure (months)')
axes[0].set_ylabel('Count')

# By churn label
for label, color in zip([0, 1], ['#2DD4BF', '#F59E0B']):
    subset = df[df['churn'] == label]['tenure_months']
    axes[1].hist(subset, bins=30, alpha=0.6, color=color,
                 label=f'Churn={label}', edgecolor='white', linewidth=0.3)
axes[1].set_title('Tenure by Churn Label')
axes[1].set_xlabel('Tenure (months)')
axes[1].legend()

plt.tight_layout()
plt.show()

Expected finding: churned customers cluster in the 1–20 month range. Retained customers are spread more evenly, with a bump at high tenure.

Monthly Fee (trimodal by segment)

fig, ax = plt.subplots(figsize=(9, 4))
for seg, color in zip(['Basic', 'Premium', 'Enterprise'], ['#0D9488', '#2DD4BF', '#F59E0B']):
    subset = df[df['segment'] == seg]['monthly_fee']
    ax.hist(subset, bins=25, alpha=0.7, color=color, label=seg, edgecolor='white', linewidth=0.3)
ax.set_title('Monthly Fee Distribution by Segment')
ax.set_xlabel('Monthly Fee (INR)')
ax.set_ylabel('Count')
ax.legend()
plt.tight_layout()
plt.show()

Info

The trimodal shape of monthly_fee is a direct consequence of the segment design — three fee bands. This means monthly_fee and segment carry partially overlapping information. Keep both, but be aware of multicollinearity when interpreting logistic regression coefficients.

Step 5: Bivariate Analysis — Churn Rate by Category

By Segment

churn_by_segment = (
    df.groupby('segment')['churn']
    .agg(['mean', 'count'])
    .rename(columns={'mean': 'churn_rate', 'count': 'n'})
    .sort_values('churn_rate', ascending=False)
)
print(churn_by_segment.round(3))
#             churn_rate    n
# Basic            ~0.40  550
# Premium          ~0.26  350
# Enterprise       ~0.10  100

fig, ax = plt.subplots(figsize=(7, 4))
bars = ax.bar(churn_by_segment.index,
              churn_by_segment['churn_rate'],
              color=['#F59E0B', '#2DD4BF', '#0D9488'])
for bar, rate in zip(bars, churn_by_segment['churn_rate']):
    ax.text(bar.get_x() + bar.get_width() / 2, bar.get_height() + 0.005,
            f'{rate:.0%}', ha='center', va='bottom', fontsize=11)
ax.set_title('Churn Rate by Customer Segment')
ax.set_ylabel('Churn Rate')
ax.set_ylim(0, 0.55)
plt.tight_layout()
plt.show()

By Contract Status

churn_by_contract = df.groupby('has_contract')['churn'].mean()
print(churn_by_contract)
# has_contract
# 0    ~0.45   (no contract — high churn)
# 1    ~0.19   (annual contract — low churn)

# The contract effect is one of the strongest binary signals in the dataset

By Support Call Bucket

df['call_bucket'] = pd.cut(
    df['support_calls'].fillna(0),
    bins=[-1, 2, 6, 15],
    labels=['Low (0–2)', 'Medium (3–6)', 'High (7+)']
)

churn_by_calls = df.groupby('call_bucket', observed=True)['churn'].agg(['mean', 'count'])
print(churn_by_calls.round(3))
#              mean  count
# Low (0–2)   ~0.12    ~330
# Medium (3–6) ~0.31   ~400
# High (7+)   ~0.52    ~270

Success

The call bucket analysis reveals a near-linear dose-response relationship between support friction and churn. This is the key finding that motivates the calls_per_tenure engineered feature in feature-engineering.

By Region

churn_by_region = df.groupby('region')['churn'].mean().sort_values(ascending=False)
print(churn_by_region.round(3))
# Expect all four regions within ~3 percentage points of each other (~0.30)
# Region is a weak predictor — this will be confirmed by feature importance later

Step 6: Numeric Features vs Churn — Box Plots

numeric_features = ['tenure_months', 'support_calls', 'monthly_fee', 'age']

fig, axes = plt.subplots(2, 2, figsize=(12, 8))
axes = axes.flatten()

for ax, col in zip(axes, numeric_features):
    data = [
        df[df['churn'] == 0][col].dropna(),
        df[df['churn'] == 1][col].dropna(),
    ]
    bp = ax.boxplot(data, labels=['Retained', 'Churned'],
                    patch_artist=True,
                    boxprops=dict(facecolor='#1E293B', color='#94A3B8'),
                    medianprops=dict(color='#F59E0B', linewidth=2),
                    whiskerprops=dict(color='#94A3B8'),
                    capprops=dict(color='#94A3B8'),
                    flierprops=dict(marker='o', color='#94A3B8', alpha=0.3))
    bp['boxes'][0]['facecolor'] = '#2DD4BF'
    bp['boxes'][1]['facecolor'] = '#F59E0B'
    ax.set_title(col.replace('_', ' ').title())
    ax.set_ylabel(col)

plt.suptitle('Feature Distributions by Churn Label', fontsize=13, y=1.01)
plt.tight_layout()
plt.show()

What to look for:

  • tenure_months: churned customers have a clearly lower median. The box for churned customers sits in the 5–25 month range; retained sits in 25–55.
  • support_calls: churned customers have a higher median and a wider interquartile range.
  • monthly_fee: the distributions overlap heavily because fee is segment-dependent, and Enterprise (lowest churn) has the highest fees. The raw feature is less informative than it looks.
  • age: the distributions are nearly identical. Age is a weak predictor here.

Step 7: Correlation Heatmap

numeric_df = df[['age', 'tenure_months', 'support_calls',
                  'monthly_fee', 'num_products', 'has_contract', 'churn']].copy()

corr = numeric_df.corr()

fig, ax = plt.subplots(figsize=(8, 6))
mask = np.triu(np.ones_like(corr, dtype=bool))
sns.heatmap(
    corr, mask=mask, annot=True, fmt='.2f',
    cmap='RdYlGn', center=0, vmin=-1, vmax=1,
    linewidths=0.5, ax=ax, annot_kws={'size': 9}
)
ax.set_title('Correlation Matrix — Numeric Features + Target', pad=12)
plt.tight_layout()
plt.show()

Expected findings from the correlation matrix:

Feature pair Correlation Interpretation
tenure_months ↔ churn ~−0.40 Strong negative — long tenure predicts staying
support_calls ↔ churn ~+0.38 Strong positive — high calls predicts leaving
has_contract ↔ churn ~−0.30 Moderate negative — contracts retain customers
age ↔ churn ~−0.03 Negligible — age barely matters here
monthly_fee ↔ churn ~−0.15 Weak negative — Enterprise (high fee) churns less

Key EDA Findings

Summarise what you have learned before moving to feature engineering:

Success

Finding 1 — Tenure is the dominant signal. Customers in their first 6 months churn at roughly 3× the rate of customers past 3 years. Interventions should prioritise new customers.

Success

Finding 2 — Support calls are the strongest actionable signal. High-call customers (7+) churn at ~52%. This is actionable: support quality improvement is a lever the business controls.

Success

Finding 3 — Segment and contract status add meaningful lift. Basic-plan customers with no contract churn at ~45%. Enterprise customers with contracts churn at ~8%. Segment-specific models or features will help.

Info

Finding 4 — Region is noise. All four regions hover at ~30% churn. Region will probably not appear in the top features of your final model. Do not drop it yet — let the model confirm — but do not engineer region-based features.

Previous: dataset-guide | Next: feature-engineering