🤖 05 — Transformers Overview¶
Transformers are modern deep learning architectures for language tasks.
They power systems like:
- BERT
- GPT-style models
- translation systems
- summarizers
- semantic search
Core Ideas¶
| Concept | Meaning |
|---|---|
| Token | piece of text |
| Embedding | numeric representation |
| Attention | mechanism for focusing on relevant tokens |
| Pretraining | learning from huge text corpora |
| Fine-tuning | adapting to a specific task |
When to Use Classic NLP vs Transformers¶
| Situation | Good Choice |
|---|---|
| small dataset, simple classifier | TF-IDF + Logistic Regression |
| high accuracy text task | pretrained transformer |
| limited compute | classic NLP baseline |
| semantic understanding | transformer embeddings |
Landmark Paper¶
Next¶
➡️ 06-exercises