Sentiment Analysis on Tweets¶

Brands send out thousands of tweets and receive tens of thousands in return — every day. Knowing whether that conversation is positive, neutral, or negative at scale is what separates reactive PR from proactive brand strategy. This project builds a complete multi-class text classification pipeline that mimics that exact use case.

Learning Objectives¶

By the end of this project you will be able to:

Build a reusable text preprocessing pipeline using re and scikit-learn
Vectorise raw text with TF-IDF and explain what each parameter controls
Train and compare Naive Bayes, Logistic Regression, and LinearSVC for multi-class text classification
Interpret model outputs using per-class precision, recall, and F1
Diagnose classification errors and explain why neutral sentiment is the hardest class
Describe where BERT fits in the production NLP landscape

Why Sentiment Analysis Matters¶

Use case	What it enables
Brand monitoring	Detect reputation crises before they escalate
Customer feedback	Route negative reviews to support, positive to marketing
Social media analytics	Track campaign sentiment in near real-time
Financial markets	News sentiment predicts short-term price movements
Product launches	Measure reception by hour, not by week

Info

Sentiment analysis is one of the most commercially deployed NLP tasks. It appears in virtually every company that has a public-facing product and a social media presence.

Skills Covered¶

Regular expressions for text cleaning
TfidfVectorizer with unigrams and bigrams
MultinomialNB, LogisticRegression, LinearSVC from scikit-learn
Pipeline composition for clean, leak-free preprocessing
GridSearchCV for hyperparameter tuning on text features
classification_report and confusion matrix interpretation
Coefficient inspection for model explainability

Prerequisites¶

This project builds directly on:

Week-02 Day-04 Part-2 — NLP Basics — tokenisation, bag-of-words, TF-IDF theory
Week-02 Day-02 Part-1 — Classification Algorithms — Logistic Regression, Naive Bayes
Week-02 Day-03 — Model Evaluation — precision, recall, F1, confusion matrices

Project Structure¶

File	What it covers
`README.md`	Project overview, objectives, prerequisites
`dataset-guide.md`	Synthetic dataset generation, class distribution, tweet characteristics
`eda.md`	Text length analysis, word frequencies, bigrams, brand-sentiment patterns
`feature-engineering.md`	Text cleaning pipeline, TF-IDF vectorisation, sklearn Pipeline
`model-building.md`	Baseline, Naive Bayes, Logistic Regression, LinearSVC, hyperparameter tuning
`evaluation.md`	Classification report, confusion matrix, error analysis, business interpretation
`interview-questions.md`	8 interview questions with detailed model answers

Dataset Note¶

This project uses a synthetic tweet-like dataset generated with reproducible code — no API key or download required. The dataset captures the linguistic patterns that make tweet sentiment hard: informal language, hashtags, brand mentions, and short texts.

For real-world follow-up, the best next steps are:

Kaggle Sentiment140 — 1.6 million real tweets, binary sentiment, freely downloadable
Twitter API v2 — requires a developer account; gives access to live tweet streams
HuggingFace datasets — tweet_eval benchmark covers sentiment, irony, emotion

Tip

Always prototype on synthetic or small datasets first. Once your pipeline is clean and your evaluation logic is correct, swapping in real data is a one-line change.

What You Will Build¶

Raw tweet text
      ↓
Text cleaning (lowercase, remove mentions/URLs/punctuation)
      ↓
TF-IDF vectorisation (unigrams + bigrams, top 5000 features)
      ↓
Multi-class classifier (Logistic Regression / Naive Bayes / LinearSVC)
      ↓
Predicted sentiment: positive | neutral | negative

The final model can label thousands of tweets per second on a laptop CPU — sufficient for daily brand health dashboards.

dataset-guide → eda → feature-engineering → model-building → evaluation → interview-questions