Attention Is All You Need, 2017

Eight Google authors replaced recurrence with attention and quietly launched the modern AI era.

32 min · Reviewed 2026

The Paper That Rewired NLP

In June 2017, eight authors from Google Brain and Google Research, led by Ashish Vaswani, posted Attention Is All You Need on arXiv. The paper proposed the Transformer, an architecture with no recurrence and no convolutions. Just attention, feed-forward layers, and residual connections.

The immediate target was machine translation, where Transformers beat the best recurrent models on English-to-German while training several times faster. Within two years, the paper would be one of the most cited in AI history.

Why Transformers took over

Parallel training: unlike RNNs, attention processes all positions at once, making full use of GPUs
Long-range dependencies: attention sees the whole sequence, not just a compressed state
Scaling: performance improved smoothly with more parameters, more data, and more compute
Generality: the same architecture worked for text, images, audio, code, and proteins

The Transformer became the common substrate for everything that followed. BERT from Google in 2018 used the encoder. GPT from OpenAI used the decoder. Vision Transformers, Whisper, AlphaFold, Stable Diffusion, and the current generation of chat models all rest on it.

We propose a new simple network architecture, the Transformer, based solely on attention mechanisms.
— Vaswani et al., 2017

The big idea: one clean architectural choice, attention without recurrence, unlocked a decade of scaling. Every LLM you use today traces directly to this paper.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-history-attention-2017-creators

What is the main idea of "Attention Is All You Need, 2017"?
1. Eight Google authors replaced recurrence with attention and quietly launched the modern AI era.
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "Attention Is All You Need, 2017"?
1. self-attention
2. Transformer
3. Vaswani
4. sequence modeling
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. Parallel training: unlike RNNs, attention processes all positions at once, making full use of GPUs
4. Treat the AI output as automatically correct
What should a careful learner remember about "What self-attention does"?
1. Use AI to draft or organize ideas about Transformer, then verify before acting.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. Use AI for drafting and comparison, but verify before publishing or relying on it.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about Transformer be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about Transformer.
Which action would help you apply "Attention Is All You Need, 2017" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Treat the AI output as automatically correct
4. Long-range dependencies: attention sees the whole sequence, not just a compressed state

← Back to interactive lesson

Tendril · Creators · AI Foundations

Attention Is All You Need, 2017

Eight Google authors replaced recurrence with attention and quietly launched the modern AI era.

32 min · Reviewed 2026

The Paper That Rewired NLP

Why Transformers took over

Parallel training: unlike RNNs, attention processes all positions at once, making full use of GPUs
Long-range dependencies: attention sees the whole sequence, not just a compressed state
Scaling: performance improved smoothly with more parameters, more data, and more compute
Generality: the same architecture worked for text, images, audio, code, and proteins

We propose a new simple network architecture, the Transformer, based solely on attention mechanisms.
— Vaswani et al., 2017

The big idea: one clean architectural choice, attention without recurrence, unlocked a decade of scaling. Every LLM you use today traces directly to this paper.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-history-attention-2017-creators

What is the main idea of "Attention Is All You Need, 2017"?
1. Eight Google authors replaced recurrence with attention and quietly launched the modern AI era.
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "Attention Is All You Need, 2017"?
1. self-attention
2. Transformer
3. Vaswani
4. sequence modeling
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. Parallel training: unlike RNNs, attention processes all positions at once, making full use of GPUs
4. Treat the AI output as automatically correct
What should a careful learner remember about "What self-attention does"?
1. Use AI to draft or organize ideas about Transformer, then verify before acting.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. Use AI for drafting and comparison, but verify before publishing or relying on it.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about Transformer be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about Transformer.
Which action would help you apply "Attention Is All You Need, 2017" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Treat the AI output as automatically correct
4. Long-range dependencies: attention sees the whole sequence, not just a compressed state

← Back to interactive lesson