How Large Language Models Actually Work

Underneath all the magic, an LLM is a system that takes your text, breaks it into tokens, and predicts what tokens come next based on patterns it learned from huge amounts of text. Understanding this one fact explains why AI is brilliant at some things and weirdly bad at others.

Some examples

• LLMs predict the next token, then the next, then the next — that's basically the whole trick.
• 'Training' means showing the model billions of text examples and adjusting its weights.
• 'Fine-tuning' is extra training to make it follow instructions or be safer.
• RLHF (reinforcement learning from human feedback) is how models learn what humans prefer.

Try it!

Open any chatbot and ask it to explain its own architecture in one paragraph. Notice what it gets right and what's vague.