Lesson 202 of 1455
Confidence Intervals
A point estimate is a guess. A confidence interval is an honest guess with its uncertainty attached. Honest Numbers Come In Pairs When a model scores 72 percent on a benchmark, that is a point estimate.
Builders · AI Foundations · ~15 min read
Honest Numbers Come In Pairs
When a model scores 72 percent on a benchmark, that is a point estimate. The real question is: what range of values is plausible? That range is the confidence interval.
What 95% CI actually means
A 95% confidence interval is an interval built by a procedure that, in the long run, captures the true value 95 percent of the time. '72%, 95% CI [68, 76]' means roughly: if you repeated the experiment many times, 95% of the computed intervals would contain the true score.
Estimating CIs quickly
- 1For a percentage: CI ≈ score ± 1.96 × sqrt(p(1-p)/n)
- 2For the mean: CI ≈ mean ± 1.96 × std/sqrt(n)
- 3For anything messy: bootstrap (resample and recompute many times)
Bootstrapping gives a CI for any metric, even ugly ones
# Bootstrap CI for model accuracy import numpy as np results = [] # 1 for correct, 0 for wrong boots = [] for _ in range(10000): sample = np.random.choice(results, size=len(results), replace=True) boots.append(np.mean(sample)) low, high = np.percentile(boots, [2.5, 97.5]) print(f"95% CI: [{low:.3f}, {high:.3f}]")“A point estimate without an interval is a guess that forgot to mention its uncertainty.”
Key terms in this lesson
The big idea: every number has a halo of uncertainty around it. Always ask for the halo.
End-of-lesson quiz
Check what stuck
8 questions · Score saves to your progress.
Lesson help
Questions are best handled with a grown-up here.
For this age range, Tendril keeps freeform AI chat paused until parent/guardian consent and child-safe moderation are fully verified. Use the quiz, notes, and related lessons below, or ask a parent, guardian, teacher, or librarian to work through the question with you.
Progress saved locally in this browser. Sign in to sync across devices.
Related lessons
Keep going
Builders · 30 min
The Supervised Learning Loop
Most modern AI is trained on a loop of guess, check, and adjust. Understand the loop and you understand the heart of machine learning.
Builders · 30 min
Tokens and Embeddings: How AI Reads Words
AI does not read letters. It reads tokens, which live as vectors in a space of meaning. Learn how text becomes numbers you can do math on.
Builders · 35 min
Neural Networks, Actually Explained
You have heard the term a thousand times. Now let's actually look inside: neurons, weights, activations, and what happens in a single pass.