Day 05 — Part 2: Exploratory Data Analysis¶
Every ML project starts with EDA, and most failed ML projects failed because EDA was skipped or rushed. Before you train a model, you need to know what your data actually contains: the distributions, the outliers, the missing values, the relationships between features. EDA is where you form hypotheses — and where you catch the data quality issues that would otherwise invalidate months of work downstream.
Session Overview
Difficulty: Intermediate Reading time: ~2.5 hours | Exercises: ~2 hours Prerequisites: Pandas · Matplotlib/Seaborn · basic statistics (Days 02–04)
What You Will Learn¶
By the end of this session you will be able to:
- Inspect a new dataset in 5 minutes: shape, types, missing values, duplicates
- Detect outliers using IQR and z-score methods
- Produce univariate and bivariate summaries for numeric and categorical columns
- Write a 3-part insight (Observation → Interpretation → Action)
- Complete a full EDA and document findings clearly
Session Roadmap¶
| # | Topic | File | Time |
|---|---|---|---|
| 1 | Data Cleaning — shape, dtypes, missing values, type errors | 01-data-cleaning.md |
20 min |
| 2 | Outlier Detection — IQR method, z-scores, visualisation | 02-outlier-detection.md |
20 min |
| 3 | Feature Understanding — cardinality, distributions, correlations | 03-feature-understanding.md |
20 min |
| 4 | Univariate Analysis — histograms, boxplots, value counts | 04-univariate-analysis.md |
20 min |
| 5 | Bivariate Analysis — scatter plots, grouped summaries, crosstabs | 05-bivariate-analysis.md |
20 min |
| 6 | EDA Workflow — the standard 8-step process | 06-eda-workflow.md |
15 min |
| 7 | Mini Case Study — full EDA on a real dataset | 07-mini-case-study.md |
45 min |
Total active learning time: ~2.5 hours
How to Use This Session¶
This session is the payoff for everything in Week 01. Files 01–06 are relatively short — they codify a structured process. The real learning happens in file 07, where you apply the full workflow to a real dataset. Do not skip straight to the case study; the workflow file (06) gives you the checklist that makes the case study tractable.
When working through the case study, write your observations as you go. A single sentence per chart — Observation, Interpretation, Action — is enough to turn exploration into documented analysis.
Before You Start
Download the dataset for the Mini Case Study before starting file 01. The case study is most valuable when you run the analysis yourself rather than reading someone else's output — so have a Jupyter notebook open from the beginning.