The big idea
Humans rated AI answers thumbs up or thumbs down. AI learned to give more answers humans liked. This is called RLHF.
Some examples
- RLHF stands for Reinforcement Learning from Human Feedback.
- Workers read AI answers all day and rated them.
- Good answers got copied. Bad answers got avoided.
- That is why modern AI sounds nicer and clearer than older AI.
Try it!
When AI gives a great answer, click thumbs up. That feedback can help train future AI versions.
End-of-lesson check
15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-explorers-foundations-AI-and-the-RLHF-thumbs-up-r10a5
Why did workers spend their day reading and rating AI answers?
- To check if AI was using bad words
- To fix spelling mistakes in AI answers
- To find funny jokes the AI wrote
- To rate them as good or bad for training
What happened to AI answers that got many "thumbs down"?
- They were avoided in future training
- They were sent to more users
- They were copied to make more AI systems
- They were made bigger on screen
Why does modern AI sound nicer than older AI?
- AI started reading story books
- Human feedback taught AI to be friendlier
- AI learned to use more emojis
- AI became older and wiser
What is "human feedback" in AI training?
- When humans write code for AI
- When humans rate AI answers as good or bad
- When humans build computer servers
- When humans type questions for AI
If you give AI a "thumbs down" for a confusing answer, what do you help the AI learn?
- To use more technical words
- To answer faster
- To give longer answers
- To avoid confusing answers in the future
What is the main purpose of clicking thumbs up on an AI answer?
- To save the answer for later
- To tell other users the answer is good
- To make the AI happy
- To help train future AI versions
Which statement best describes how RLHF works?
- Humans rate AI answers and AI learns from those ratings
- AI reads books and learns on its own
- AI guesses answers and users check if they're true
- AI watches videos to learn new things
What did workers do with good AI answers during RLHF training?
- They copied them as examples to learn from
- They sent them to other companies
- They turned them into pictures
- They deleted them
What would happen if no humans ever gave feedback to AI?
- AI would become smarter on its own
- AI would only learn from books
- AI would learn nothing new
- AI would stop working
How does clicking thumbs up help make AI better?
- It fixes the AI's code directly
- It gives the AI money to buy upgrades
- It tells the AI what question to answer next
- It provides examples of good answers for training
What is the "big idea" of this lesson?
- Humans rated AI answers to help AI improve
- AI learned to speak different languages
- AI can read minds
- Computers can feel emotions
Which of these is an example of human feedback?
- A user asking AI a question
- A developer writing AI computer code
- A computer running AI programs
- A user giving AI a star rating for its answer
Why is it important that humans, not just computers, rate the AI answers?
- Computers don't like rating things
- Computers can't click buttons
- Humans can tell what sounds nice and helpful to other humans
- Humans are faster than computers
What kind of answers did the AI learn to give more often after RLHF?
- Answers that humans marked as thumbs up
- Answers that were very short
- Answers with many spelling mistakes
- Answers that took a long time to generate
If an AI keeps getting thumbs down for rude answers, what will likely happen?
- The AI will learn to be less rude
- The AI will become more rude
- The AI will start ignoring users
- The AI will delete its memory
