How AI Chops Up Words Into Tiny Pieces

AI doesn't read words the way we do. It chops sentences into small pieces called 'tokens'. A token might be a whole word, part of a word, or even just a few letters. This helps the AI handle any language.

Some examples

• The word 'unhappy' might become two tokens: 'un' and 'happy'.
• Short, common words like 'the' are usually one token.
• Made-up words like 'flibbertigibbet' get chopped into many tokens.
• Spaces and punctuation are tokens too!

Try it!

Try saying these words and clap for each chunk: 'play-ing', 'un-der-stand', 'birth-day'. AI does something like this, but with its own special chunks.

Practice this safely

Try this with a low-stakes example and a trusted adult nearby. The goal is to notice how AI talks about word chunks, not to let it make the decision for you.

1. 1Ask AI to explain word chunks in plain language, then underline anything that sounds uncertain or too broad.
2. 2Give it one detail from "How AI Chops Up Words Into Tiny Pieces" and ask for two possible next steps plus one reason each step might be wrong.
3. 3Check tokens against a trusted source, teacher, adult, expert, or original document before you use it.