Movie Recommendation System¶
Netflix credits recommendations for driving over one-third of what users watch. Amazon attributes 35% of its revenue to its recommendation engine. Spotify's Discover Weekly — built on collaborative filtering — became one of the most-used features in streaming history. Recommendation systems are everywhere, and understanding how they work is one of the most marketable skills a data scientist can have.
In this project you build three recommendation approaches from scratch, using only NumPy, Pandas, and scikit-learn — no specialised RecSys libraries required.
Learning Objectives¶
By the end of this project you will be able to:
- Explain the difference between popularity-based, content-based, and collaborative filtering approaches
- Build a user-item ratings matrix and reason about sparsity
- Implement cosine similarity for both content-based and user-based recommendations
- Evaluate a recommender offline using Hit Rate and Precision@K
- Articulate the cold start problem and name at least two mitigation strategies
- Discuss how matrix factorisation (SVD/NMF) extends what you built here
Skills Practised¶
| Skill | Where it appears |
|---|---|
| Matrix operations | Building the ratings matrix, computing similarity |
| Cosine similarity | Content-based features, user-user similarity |
| Sparse data handling | NaN fill strategies, pivot tables |
| One-hot encoding | Genre features for content-based filtering |
| Offline evaluation | Precision@K, Recall@K, Hit Rate |
| Train-test split for RecSys | Holding out the last rating per user |
Prerequisites¶
Before starting, make sure you are comfortable with:
- Week-02 Day-01 Part-1: Machine Learning Basics (what a model is, train/test split)
- Week-02 Day-02 Part-1: Classification Algorithms (decision boundaries, similarity metrics)
- Week-02 Day-03 Part-1: Feature Engineering (encoding, normalisation)
- NumPy dot products and matrix shapes — you will use these constantly
Project Structure¶
| File | What it covers |
|---|---|
README.md |
Overview, objectives, prerequisites (this file) |
dataset-guide.md |
Generating the synthetic dataset, understanding sparsity |
eda.md |
Exploring rating distributions, genre patterns, the long-tail effect |
feature-engineering.md |
Genre encoding, cosine similarity matrices, mean-centering |
model-building.md |
Three recommendation approaches, full working functions |
evaluation.md |
Offline metrics, comparing approaches, limitations |
interview-questions.md |
Eight interview questions with model answers |
The Three Approaches¶
Approach 1 — Popularity Baseline Recommend the most-rated or highest-rated movies globally. No personalisation. Every user gets the same list. It is a useful baseline because any personalised approach should outperform it.
Approach 2 — Content-Based Filtering Recommend movies that are similar to what a specific user has already rated highly. Similarity is computed from movie features — in this project, genre one-hot encodings. If a user likes Sci-Fi movies, the system finds other Sci-Fi movies they have not yet rated.
Approach 3 — User-Based Collaborative Filtering Find users who have similar rating patterns to the target user. Recommend movies those similar users have rated highly but the target user has not seen. This approach does not need any movie metadata — it learns purely from the pattern of ratings.
Info
A fourth approach — matrix factorisation (SVD, NMF) — is introduced conceptually in model-building.md. It is the foundation of most production recommendation systems, including Netflix's early prize-winning approach.
Why a Synthetic Dataset?¶
Real rating datasets (MovieLens, Netflix Prize) require downloads and have licence restrictions. The synthetic dataset generated in dataset-guide.md is built from a latent-factor model, so it has realistic structure: users with coherent tastes, popular movies, and a long-tail of rarely-rated films. It runs entirely in memory with no external files.