NLP Applications¶
Understanding how NLP techniques translate into production systems is what separates candidates who have read about NLP from those who have built with it. These questions target end-to-end system design, evaluation, and the practical decisions that come up in real projects.
Q1: Walk through a complete text classification pipeline from raw text to predictions.¶
Show answer
A production-ready classification pipeline has distinct stages, each with failure modes.
1. Data collection and labelling: Gather raw text samples. Label them (manually or via weak supervision). Check class distribution — imbalance here will affect every downstream decision.
2. Preprocessing: Clean and normalise: decode encoding, strip HTML, remove URLs, lowercase (if appropriate), tokenise.
3. Feature extraction: Classical: TF-IDF with ngram_range=(1,2), min_df=3, max_features=50000 Modern: sentence-transformers or fine-tuned BERT embeddings
4. Model selection and training: Classical features → Logistic Regression or Linear SVM Embeddings → Logistic Regression, MLP, or fine-tune the encoder end-to-end
5. Evaluation: Stratified train/val/test split. Report F1 (macro for balanced classes, weighted for imbalanced). Check confusion matrix for systematic errors.
6. Error analysis: Sample 50 misclassified examples. Categorise the error types. Improve preprocessing or add training examples for the most common error category.
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
texts = ["great product", "terrible service", "okay experience", "loved it", "never buying again"]
labels = [1, 0, 1, 1, 0]
X_train, X_test, y_train, y_test = train_test_split(
texts, labels, test_size=0.2, stratify=labels, random_state=42
)
pipeline = Pipeline([
('tfidf', TfidfVectorizer(ngram_range=(1, 2), min_df=1)),
('clf', LogisticRegression(C=1.0, max_iter=500))
])
pipeline.fit(X_train, y_train)
preds = pipeline.predict(X_test)
print(classification_report(y_test, preds))
Q2: How do you approach sentiment analysis? What are the key decisions?¶
Show answer
Sentiment analysis is one of the most common NLP tasks in industry — customer reviews, social media monitoring, product feedback.
Binary vs multi-class: - Binary (positive/negative): simpler, more training data available, usually higher accuracy - Multi-class (5-star rating prediction): harder. Star 3 reviews are genuinely ambiguous. Often model this as regression (predict a score), not classification.
Key decisions:
Negation handling: "Not good" must not be treated the same as "good". Avoid stop word removal for sentiment. Use bigrams to capture negation patterns.
Domain matters more than algorithm: Embeddings trained on movie reviews (Stanford Sentiment Treebank) do not transfer well to product reviews or clinical text. Fine-tune on in-domain data.
Aspect-based sentiment: "The camera is excellent but the battery life is poor" → positive for camera, negative for battery. Standard sentence-level classifiers assign one label to the whole sentence and lose this nuance.
from transformers import pipeline
sentiment = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english")
reviews = [
"The product works exactly as described.",
"Great build quality but the software is buggy.",
"Absolutely terrible. Do not buy."
]
results = sentiment(reviews)
for review, result in zip(reviews, results):
print(f"{result['label']:8} ({result['score']:.2f}) — {review[:50]}")
Common mistake: using accuracy as the primary metric when classes are imbalanced (often far more negative reviews than positive ones in some domains). Always report F1.
Q3: What is the BIO tagging scheme in NER, and why is it used?¶
Show answer
BIO stands for Beginning, Inside, Outside. It converts named entity recognition from a span-detection problem into a token classification problem — every token gets exactly one label.
- B-TYPE: Beginning of an entity of type TYPE
- I-TYPE: Inside (continuation of) an entity of type TYPE
- O: Outside — not part of any entity
Why not a simpler scheme? Adjacent entities of the same type cannot be distinguished without B and I markers.
"John Smith" → B-PER I-PER (one entity: John Smith)
"John Smith" → B-PER B-PER (two separate entities: John, Smith)
Evaluation: NER is evaluated at entity level, not token level. A prediction is correct only if the entire span and the entity type both match the ground truth. Partial matches count as errors.
import spacy
from spacy.training import Example
# spaCy uses IOB2 (equivalent to BIO) internally
nlp = spacy.load("en_core_web_sm")
doc = nlp("Microsoft acquired LinkedIn in 2016.")
for token in doc:
print(f"{token.text:12} {token.ent_iob_}-{token.ent_type_ if token.ent_type_ else 'O'}")
# Microsoft B-ORG
# acquired O-O
# LinkedIn B-ORG
# in O-O
# 2016 B-DATE
Q4: What is the difference between extractive and abstractive summarisation?¶
Show answer
Extractive summarisation: Select the most important sentences from the source document verbatim and concatenate them as the summary. No new text is generated.
- Approach: rank sentences by importance (TF-IDF centroid similarity, TextRank graph algorithm, or a trained classifier)
- Output is always grammatically correct (it's original text)
- Cannot compress or paraphrase — the summary can only be as short as the shortest relevant sentence
Abstractive summarisation: Generate new text that expresses the key ideas, potentially using different words, combining information across sentences, or introducing phrases not present in the source.
- Approach: encoder-decoder models (BART, T5, Pegasus) trained on document-summary pairs
- Can produce shorter, more fluid summaries
- Risk: hallucination — generated content may not be grounded in the source
from transformers import pipeline
summarizer = pipeline("summarization", model="facebook/bart-large-cnn")
article = """
Scientists at MIT announced a breakthrough in fusion energy research.
The team achieved a net energy gain for the first time, producing more energy
than was required to initiate the fusion reaction. This milestone, decades in the
making, could accelerate the development of commercial fusion power plants.
"""
summary = summarizer(article, max_length=60, min_length=20, do_sample=False)
print(summary[0]['summary_text'])
Evaluation with ROUGE: ROUGE (Recall-Oriented Understudy for Gisting Evaluation) measures n-gram overlap between generated and reference summaries. ROUGE-1 measures unigram overlap; ROUGE-2 measures bigram overlap; ROUGE-L measures longest common subsequence.
Q5: What is ROUGE, and what are its limitations as an evaluation metric?¶
Show answer
ROUGE measures overlap between a generated text and one or more reference texts (human-written). It is the standard automatic metric for summarisation and translation.
ROUGE-N Recall = (n-gram matches) / (n-grams in reference)
ROUGE-N Precision = (n-gram matches) / (n-grams in hypothesis)
ROUGE-N F1 = harmonic mean of recall and precision
from rouge_score import rouge_scorer
scorer = rouge_scorer.RougeScorer(['rouge1', 'rouge2', 'rougeL'], use_stemmer=True)
reference = "The cat sat on the mat near the window."
hypothesis = "A cat was sitting on the mat."
scores = scorer.score(reference, hypothesis)
for metric, score in scores.items():
print(f"{metric}: P={score.precision:.3f}, R={score.recall:.3f}, F={score.fmeasure:.3f}")
Limitations of ROUGE: - Insensitive to paraphrase: "The vehicle crashed" and "The car had an accident" have low ROUGE despite identical meaning - Reference-dependent: quality depends on having good human references; a correct but differently worded summary scores poorly - Does not measure factual accuracy: a summary that preserves all key n-grams but introduces a factual error scores well - Poor for abstractive systems: extractive summaries trivially score well since they copy source text
In practice, ROUGE is used as a cheap filter and ranking tool, not as a definitive quality measure. Human evaluation remains necessary for production decisions.
Q6: What is semantic search, and how does it differ from keyword search?¶
Show answer
Keyword search (BM25, TF-IDF retrieval): Matches documents that contain the same terms as the query. Fast, no training required, interpretable. Fails when query and document use different vocabulary for the same concept.
Example: query "heart attack symptoms" fails to retrieve documents about "myocardial infarction signs" even though they are the same topic.
Semantic search (dense retrieval): Encodes queries and documents into dense vectors using a sentence encoder. Retrieves documents by nearest-neighbour search in embedding space.
- Query and relevant documents end up nearby in vector space even if they share no words
- Requires an encoder (SBERT, e5, BGE) and an approximate nearest-neighbour index (FAISS, Hnswlib)
from sentence_transformers import SentenceTransformer
import numpy as np
from sklearn.metrics.pairwise import cosine_similarity
model = SentenceTransformer('all-MiniLM-L6-v2')
documents = [
"Myocardial infarction occurs when blood flow to the heart is blocked.",
"Python is a popular programming language.",
"Chest pain and shortness of breath are warning signs of cardiac events."
]
query = "What are the symptoms of a heart attack?"
doc_embeddings = model.encode(documents)
query_embedding = model.encode([query])
scores = cosine_similarity(query_embedding, doc_embeddings)[0]
ranked = sorted(zip(scores, documents), reverse=True)
for score, doc in ranked:
print(f"{score:.3f} {doc[:60]}")
When to use each: - Keyword search: exact term matching needed (legal text, product codes, identifiers), low latency requirements, no training data - Semantic search: natural language queries, paraphrase tolerance needed, multilingual retrieval
Q7: How does BERT get fine-tuned for extractive question answering?¶
Show answer
Extractive QA takes a question and a context passage and returns the answer as a span — a start and end position within the passage (the answer must be copied from the context, not generated).
SQuAD format: - Context: "The Eiffel Tower is located in Paris, France." - Question: "Where is the Eiffel Tower?" - Answer: "Paris, France" (span from the context)
How BERT is fine-tuned: 1. Input: [CLS] question tokens [SEP] context tokens [SEP] 2. Two linear layers are added on top of BERT — one predicts the start token index, one predicts the end token index 3. Fine-tuned on (question, context, start_position, end_position) tuples
from transformers import pipeline
qa = pipeline("question-answering", model="deepset/roberta-base-squad2")
context = """
The transformer architecture was introduced by Vaswani et al. in 2017
in the paper "Attention is All You Need". It became the foundation
for BERT, GPT, and most modern NLP systems.
"""
question = "Who introduced the transformer architecture?"
result = qa(question=question, context=context)
print(f"Answer: {result['answer']}")
print(f"Score: {result['score']:.3f}")
# Answer: Vaswani et al.
When extractive QA is insufficient: If the answer is not a literal span in the provided context, extractive QA fails. Abstractive QA (generating the answer with an encoder-decoder like T5) handles multi-hop reasoning and synthesis across passages.
Q8: What is BLEU, and how do you evaluate machine translation quality?¶
Show answer
BLEU (Bilingual Evaluation Understudy) measures the precision of n-grams in the hypothesis (machine translation output) against one or more reference translations.
BLEU = BP × exp(Σ w_n × log precision_n)
where BP = brevity penalty (penalises short translations)
w_n = weights for each n-gram order (typically uniform)
A BLEU score of 1.0 is a perfect match with a reference translation (impossible in practice). Scores above 0.4 generally indicate good translation quality on common benchmarks.
from nltk.translate.bleu_score import sentence_bleu, SmoothingFunction
reference = [["the", "cat", "is", "on", "the", "mat"]]
hypothesis = ["the", "cat", "sits", "on", "the", "mat"]
smooth = SmoothingFunction().method1
score = sentence_bleu(reference, hypothesis, smoothing_function=smooth)
print(f"BLEU: {score:.3f}")
Limitations of BLEU: - Insensitive to meaning — a grammatically correct but semantically wrong translation can score well - Does not measure fluency or adequacy independently - Requires reference translations — expensive to produce for new language pairs - Correlates poorly with human judgements for high-quality translations
Other translation metrics: - chrF: character n-gram F-score — more sensitive to morphological variation - METEOR: considers synonyms and stemming - COMET: neural metric trained on human judgements — currently the best correlation with human evaluation
Q9: How do you evaluate NLP models when you cannot use standard metrics?¶
Show answer
Standard metrics (BLEU, ROUGE, F1) fail in several common situations: - Open-ended generation where multiple answers are equally correct - Tasks where the ground truth labels are noisy or incomplete - New tasks without established benchmarks
Approaches when standard metrics are insufficient:
Human evaluation: The gold standard but expensive. Design a clear rubric (fluency 1–5, relevance 1–5, factual accuracy yes/no). Use multiple annotators and measure inter-annotator agreement (Cohen's kappa or Krippendorff's alpha).
LLM-as-judge: Use a capable LLM (GPT-4) to evaluate outputs against a rubric. Correlates reasonably with human judgement for some tasks. Biased toward outputs that match the evaluator LLM's own style — use cautiously and validate against human judgements on a sample.
A/B testing: Serve two model variants to users and measure downstream metrics (click-through rate, task completion, engagement). The most reliable signal for production systems.
Pointwise vs pairwise evaluation: - Pointwise: rate each output on an absolute scale (harder for annotators to be consistent) - Pairwise: show two outputs and ask which is better (more reliable inter-annotator agreement)
from sklearn.metrics import cohen_kappa_score
# Inter-annotator agreement for NER labels
annotator_1 = [1, 0, 1, 1, 0, 1, 0, 0]
annotator_2 = [1, 0, 1, 0, 0, 1, 1, 0]
kappa = cohen_kappa_score(annotator_1, annotator_2)
print(f"Cohen's kappa: {kappa:.3f}")
# kappa > 0.6: substantial agreement
# kappa > 0.8: almost perfect agreement
Q10: When should you use a pre-trained LLM via API vs fine-tune a smaller model vs use classical methods?¶
Show answer
This is a system design question that comes up in every applied NLP interview. The answer depends on data, compute, latency, and interpretability constraints.
Use a pre-trained LLM via API (GPT-4, Claude, Gemini) when: - You have no labelled training data - The task is general-purpose and well within the LLM's capabilities - Rapid prototyping is the goal - Volume is low enough that API costs are acceptable - Customisation via prompt engineering is sufficient
Fine-tune a smaller model (BERT, RoBERTa, DistilBERT, LLaMA) when: - You have hundreds to thousands of labelled examples in your domain - The domain language differs significantly from general web text (medical, legal, financial) - Latency requirements rule out large API-hosted models - Cost at scale makes API calls prohibitive - You need reproducibility and control over the model
Use classical methods (TF-IDF + Logistic Regression, Naive Bayes) when: - You need full interpretability (regulated industries, explainability requirements) - You have very limited compute (edge deployment, embedded systems) - The task is simple (keyword classification, spam detection with known keywords) - You need a fast baseline before investing in neural approaches
Decision heuristic for interviews: 1. Start with a classical baseline — it sets the floor and often reveals the hardest examples 2. If baseline is insufficient, fine-tune a small pre-trained model 3. If data is scarce or the task is novel, use LLM with few-shot prompting 4. If production cost and latency matter, fine-tune a small model to match LLM quality
Q11: What is text similarity, and when do you use embedding cosine similarity vs a cross-encoder?¶
Show answer
Text similarity quantifies how semantically alike two pieces of text are. The two main approaches have a fundamental speed-quality tradeoff.
Bi-encoder (embedding cosine similarity): - Encode each text independently into a single vector - Similarity = cosine similarity between vectors - Pre-compute document embeddings offline → sub-millisecond retrieval at query time - Quality: good for retrieving rough candidates, less accurate for fine-grained ranking
Cross-encoder: - Concatenate the two texts as [CLS] text_A [SEP] text_B [SEP] and run through a transformer - The model sees the full interaction between the two texts — attention can compare specific words and phrases - Quality: significantly higher accuracy than bi-encoder - Speed: must process every (query, document) pair at query time — cannot pre-compute
from sentence_transformers import SentenceTransformer, CrossEncoder
from sklearn.metrics.pairwise import cosine_similarity
query = "What is the capital of France?"
docs = ["Paris is the capital of France.", "France is a country in Europe.", "Berlin is the capital of Germany."]
# Bi-encoder: fast retrieval
bi_model = SentenceTransformer('all-MiniLM-L6-v2')
q_emb = bi_model.encode([query])
d_embs = bi_model.encode(docs)
bi_scores = cosine_similarity(q_emb, d_embs)[0]
print("Bi-encoder scores:", bi_scores.round(3))
# Cross-encoder: accurate re-ranking
cross_model = CrossEncoder('cross-encoder/ms-marco-MiniLM-L-6-v2')
pairs = [(query, doc) for doc in docs]
cross_scores = cross_model.predict(pairs)
print("Cross-encoder scores:", cross_scores.round(3))
Production pattern: use bi-encoder to retrieve top-K candidates (fast), then cross-encoder to re-rank those K candidates (accurate). This gives the speed of retrieval with the quality of interaction modelling.
Q12: How do you handle multilingual NLP tasks?¶
Show answer
Multilingual NLP is the norm in production systems that serve global users. There are two main strategies.
Per-language models: - Train or fine-tune a separate model for each language - Maximum performance per language when training data is available - Maintenance overhead: N models for N languages - Not viable for low-resource languages with insufficient training data
Multilingual models: - One model handles all languages: mBERT (104 languages), XLM-RoBERTa, mT5 - Trained on concatenated multilingual corpora - Cross-lingual transfer: fine-tune on English labelled data, the model generalises to other languages - Performance gap vs monolingual models narrows as model size increases
from transformers import pipeline
classifier = pipeline(
"text-classification",
model="cardiffnlp/twitter-xlm-roberta-base-sentiment"
)
texts = [
"This product is amazing!", # English
"Ce produit est incroyable!", # French
"Este producto es increíble!" # Spanish
]
results = classifier(texts)
for text, result in zip(texts, results):
print(f"{result['label']:8} ({result['score']:.2f}) — {text}")
Practical considerations: - Tokenisation affects all languages differently — BPE vocabularies trained primarily on English underrepresent other scripts - Language detection as a first step is often useful to route inputs to the best available model - For very low-resource languages, machine translation to English then English-language model is a viable fallback
Q13: What evaluation metrics are used for each major NLP task?¶
Show answer
Knowing which metric to report for which task is tested directly in interviews — candidates who name "accuracy" as the metric for every NLP task signal shallow experience.
| Task | Primary Metric | Notes |
|---|---|---|
| Text classification (balanced) | Macro F1 | Averages F1 equally across classes |
| Text classification (imbalanced) | Weighted F1 or PR-AUC | Accuracy is misleading |
| Named entity recognition | Entity-level F1 | Partial span matches = wrong |
| Sentiment analysis | Macro F1 | Or Pearson correlation if predicting a score |
| Summarisation | ROUGE-1, ROUGE-2, ROUGE-L | Supplement with human evaluation |
| Machine translation | BLEU, chrF, COMET | COMET correlates best with human judgements |
| Language modelling | Perplexity | Lower = better |
| Question answering (extractive) | Exact Match (EM) and F1 | F1 allows partial span credit |
| Semantic search / retrieval | NDCG@K, MRR, Recall@K | Rank-based metrics |
| Text generation (open-ended) | Human evaluation, BERTScore | No good automatic metric exists |
from sklearn.metrics import f1_score, classification_report
y_true = [0, 1, 2, 0, 1, 2, 0, 1]
y_pred = [0, 0, 2, 0, 1, 1, 0, 1]
print("Macro F1:", f1_score(y_true, y_pred, average='macro').round(3))
print("Weighted F1:", f1_score(y_true, y_pred, average='weighted').round(3))
print()
print(classification_report(y_true, y_pred))
The common mistake is reporting a single number without specifying the averaging strategy for multi-class problems. "F1 = 0.85" is ambiguous; "macro F1 = 0.85" is precise.
Q14: What is Retrieval-Augmented Generation (RAG), and when does it outperform standard fine-tuning?¶
Show answer
RAG (Lewis et al., 2020) combines a retriever and a generator. At inference time: 1. The user query is used to retrieve relevant documents from an external knowledge base 2. Retrieved documents are concatenated with the query as context 3. A language model generates the answer conditioned on both the query and retrieved context
Why RAG outperforms standard fine-tuning in specific scenarios:
Knowledge that changes over time: Fine-tuned model weights are frozen — they cannot incorporate new information without retraining. RAG's knowledge base can be updated independently (add new documents, remove outdated ones) without touching the model.
Long-tail factual knowledge: Fine-tuned models hallucinate on rare facts because those facts appear infrequently during training. RAG grounds generation in retrieved source documents — the model can cite specific passages.
Auditability: RAG systems can show which documents were retrieved and used. Fine-tuned models provide no such transparency.
RAG system flow:
User query: "What is the current return policy?"
↓
Retriever: finds top-3 relevant chunks from policy database
↓
Prompt: "Using the following documents: [chunk1] [chunk2] [chunk3]
Answer the question: What is the current return policy?"
↓
Generator: produces grounded, cited answer
When fine-tuning is better than RAG: - You need to change the model's style, tone, or behaviour (not just knowledge) - Latency requirements prohibit retrieval at inference time - The task requires reasoning over many facts simultaneously (RAG context windows are limited)