How AI Chops Up Words Into Tiny Pieces

AI doesn't read words the way we do. It chops sentences into small pieces called 'tokens'. A token might be a whole word, part of a word, or even just a few letters. This helps the AI handle any language.

Some examples

• The word 'unhappy' might become two tokens: 'un' and 'happy'.
• Short, common words like 'the' are usually one token.
• Made-up words like 'flibbertigibbet' get chopped into many tokens.
• Spaces and punctuation are tokens too!

Try it!

Try saying these words and clap for each chunk: 'play-ing', 'un-der-stand', 'birth-day'. AI does something like this, but with its own special chunks.

When you type a sentence to an AI, it cuts your words into tiny puzzle pieces called tokens. Some tokens are whole words, and some are just parts of words like 'play' and 'ing'.

Some examples

• The word 'unhappy' might become two tokens: 'un' and 'happy'.
• 'Cat' is usually just one token because it is short.
• Long words like 'hippopotamus' often get cut into 3 or 4 tokens.
• Even spaces and punctuation can be their own tokens.

Try it!

Type a normal sentence to an AI, then type the same sentence with NO spaces between words. Notice if the answer changes.

You see a smiley face. AI sees a special code. Emojis are tokens too — and AI can read them, write them back, and even use them on purpose to set the mood.

Some examples

• The pizza emoji is its own token AI knows.
• AI can sort emojis by feeling: happy ones, sad ones, sporty ones.
• Some rare emojis make AI confused.
• AI can pick emojis to match a story you wrote.

Try it!

Ask AI: 'Tell me a story using only 5 emojis.' See what it picks!

Here's why "AI Sees Emojis as Tiny Code Pieces" matters: Learning about AI is one of the most important skills you can build for the future! To AI, a smiley emoji is just another little chunk in its puzzle — and knowing how to apply this gives you a concrete advantage.

• Learn what "tokens" means and why it's important
• Learn what "emojis" means and why it's important
• Learn what "encoding" means and why it's important

1. 1Find out more about AI Sees Emojis as Tiny Code Pieces by asking an AI a question about it
2. 2Talk to a grown-up about what you learned
3. 3Write down one new thing you learned today

Did you know AI is bad at counting letters in a word? Ask it 'how many R's in strawberry?' and it might say 2 instead of 3. AI sees words in chunks called tokens, not one letter at a time.

Some examples

• 'Strawberry' might look like 'straw' + 'berry' to AI.
• AI doesn't see 'cat' as c-a-t — it sees one chunk.
• Counting letters is harder for AI than counting apples.
• Spelling backwards confuses AI a lot.

Try it!

Ask AI to count the R's in 'strawberry'. Try 'banana' too. Did it get them right?

AI does not read words like you do — it changes each word into numbers and does math on them.

Some examples

• The word 'cat' might turn into a long list of numbers.
• Words that mean similar things end up with similar numbers.
• AI compares number-lists to guess what comes next.
• That's why AI can finish your sentence even if it has never seen it before.

Try it!

Try saying two words like 'puppy' and 'dog' out loud. They mean almost the same thing — to AI, their number-lists look almost the same too. Now try 'puppy' and 'rocket' — very different number-lists!

AI keeps a dictionary of word-pieces called tokens, and it builds any word by snapping pieces together like LEGO.

Some examples

• The word 'unhappy' might be two pieces: 'un' and 'happy'.
• A made-up word like 'florbnado' becomes a long string of tiny pieces.
• Common words like 'the' are usually one whole piece.
• That's why AI can read words it has never seen before.

Try it!

Pick a long word like 'butterfly' and try splitting it three different ways: 'butter+fly', 'but+ter+fly', or 'b+u+t+t+e+r+f+l+y'. AI does this same kind of splitting!