Submission Checklist — Churn Prediction Project¶
Go through this list before you submit, push to GitHub, or share with a reviewer. Every item is here because a real project failed it at least once.
Code Quality¶
- [ ]
customer_idis dropped before any model training - [ ] Train/test split happens before any fitting — the preprocessor is never fitted on
X_test - [ ] All preprocessing is inside a
Pipelineobject (no stray.fit_transform()calls on test data) - [ ] Every random operation has
random_state=42(or any fixed integer) —train_test_split, all models,RandomizedSearchCV - [ ] The test set is evaluated exactly once, in the final evaluation cell
- [ ] No hardcoded file paths that only work on your machine (the dataset is generated in code — no path needed)
- [ ] No leftover debug cells (
print(df.head())in the middle of the modeling section, unused variable assignments) - [ ] Each notebook cell is runnable in order from top to bottom — restart kernel and run all before submitting
Warning
The most common reason projects fail review: the model looks great on the notebook but the test set was accidentally seen during preprocessing. The giveaway is a test F1 that is higher than the cross-validated training F1. If your test score is more than 5 points higher than your CV score, stop and audit your pipeline for leakage before submitting.
Reproducibility¶
- [ ] Running the notebook from scratch (Kernel → Restart & Run All) produces the same results every time
- [ ] All imports are at the top of the notebook (or in the first cell of each section)
- [ ] Dataset generation cell includes
rng = np.random.default_rng(42)— the seed is explicit - [ ]
requirements.txtlists exact library versions:
Generate it with:
Tip
Pin exact versions. "scikit-learn>=1.0" sounds flexible but breaks when scikit-learn 2.0 changes an API. Reviewers who cannot reproduce your results will not grade your methodology.
Documentation¶
- [ ] Each notebook section has a markdown header (H2 or H3) that matches the project phase
- [ ] Every cleaning decision has a one-sentence justification in a markdown cell above the code
- [ ] EDA findings are written in prose (not just charts) before the feature engineering section
- [ ] Model selection rationale is written in prose — not just a table with numbers
- [ ] The results section uses the template from
05-evaluation-and-report.md— filled in with real numbers, not placeholders
Results Section (copy and fill in)¶
Paste this at the top of your final notebook section and fill in every bracket:
## Results
**Model:** Gradient Boosting Classifier (tuned with RandomizedSearchCV, 40 iterations)
**Test set F1 (churn class):** [X]
**Test set ROC-AUC:** [X]
**Test set accuracy:** [X]%
**Cross-validated F1 (training set):** [X] ± [X]
At threshold 0.50: precision = [X], recall = [X], F1 = [X]
At threshold 0.35: precision = [X], recall = [X], F1 = [X] ← recommended operating point
Top 3 churn drivers (by feature importance):
1. contract_type_Month-to-Month (importance: [X])
2. calls_per_tenure (importance: [X])
3. tenure_months (importance: [X])
Business recommendation: [one paragraph — see 05-evaluation-and-report.md for template]
Limitations:
- [list 2–3 honest limitations]
Next steps if this were a production project:
- [list 2–3 concrete improvements]
GitHub Repo Structure¶
Your repo should look like this before you push:
A single well-organised notebook is correct for a solo project. Do not split into five files to look more "production-like" — reviewers want to follow your reasoning in order.
What a Good README Contains¶
Your README.md should answer five questions a stranger would have when landing on your repo:
# Customer Churn Prediction
## Problem
[One sentence: what are you predicting and why does it matter?]
## Dataset
[One sentence: how is the data generated, how many rows, what is the churn rate?]
## Method
[Two to three sentences: what approach did you take? What models did you try?]
## Results
[Copy your results section — model, test F1, ROC-AUC, top features]
## How to Run
[Four lines maximum]
pip install -r requirements.txt
jupyter notebook notebook.ipynb
# Run all cells in order
# Dataset is generated in Cell 1 — no external files needed
Tip
The README is the first thing a recruiter or interviewer sees. Write it last, when you know your actual results. A README with placeholder brackets is worse than no README.
Interview Story — 2-Minute Version¶
Prepare this answer before any interview. You should be able to deliver it without notes.
I built a customer churn prediction model for a telecom company.
The business goal was to flag at-risk customers before they cancelled so a retention
team could intervene. I chose F1 on the churned class as the primary metric because
the dataset was moderately imbalanced at 30% churn, and accuracy would have been
misleading.
I generated a realistic synthetic dataset, ran EDA that identified contract type and
support call rate as the strongest churn signals, then built a scikit-learn Pipeline
with StandardScaler for numerics and OneHotEncoder for categoricals. I engineered two
interaction features — calls per tenure and charges per product — which both ranked in
the top 5 by feature importance.
I compared Logistic Regression, Random Forest, and Gradient Boosting using 5-fold
cross-validation, tuned the best performer with RandomizedSearchCV, and evaluated
once on a held-out test set: F1 of 0.74 and ROC-AUC of 0.88.
The main business recommendation was to lower the classification threshold from 0.50
to 0.35 to increase recall from 75% to 90%, accepting more false positives given that
a retention offer costs far less than losing a customer.
The biggest limitation is that the dataset is synthetic. The next step would be
validating on real customer data with temporal features like recency of last call.
Success
This answer demonstrates: problem framing, metric choice rationale, EDA-to-model consistency, pipeline hygiene, model comparison process, threshold analysis as a business decision, and honest acknowledgment of limitations. That is a complete data science project story.