Tendril
Loading lesson…
AI-generated code that compiles, runs, and produces wrong answers is the most dangerous class of bug. Learn the disguises plausible-but-wrong code wears and the verification habits that catch it.
Crashing code is a gift. It tells you something is wrong. The dangerous bug is the one where everything runs fine and the answer is wrong. AI is exceptionally good at producing this class of bug because it optimizes for plausibility. Plausibility is not correctness.
| Disguise | Example | How it bites |
|---|---|---|
| Off-by-one | `for i in range(len(arr) - 1)` (skips last) | Last item silently omitted |
| Wrong default | `dict.get(key, [])` returns `[]`, code expects `None` | Empty-list arithmetic, not the crash you'd expect |
| Reversed condition | `if user.is_admin == False:` flipped during refactor | Permission check inverted in production |
| Float vs int | `sum / len(items)` returns int in Python 2 idiom on Python 3 floats | Quietly wrong averages |
| Time zone | `datetime.now()` instead of `datetime.now(timezone.utc)` | All timestamps off by a server's local offset |
For any nontrivial AI-generated function, run it on three inputs before trusting it: the happy path, the edge case (empty/null/zero), and a weird case (wrong type, very large, unicode). If you cannot quickly construct three inputs, the function is too vague to test and probably wrong somewhere.
# AI gave you this function for computing percentiles
def percentile(values, p):
values = sorted(values)
k = (len(values) - 1) * p / 100
return values[int(k)]
# Three-input verification:
print(percentile([1, 2, 3, 4, 5], 50)) # 3 — looks right
print(percentile([1, 2, 3, 4, 5], 100)) # 5 — looks right
print(percentile([], 50)) # IndexError — broken
# The model never tested empty input. Almost no AI-written
# function does on the first pass. This is the most common bug class.Three inputs caught a bug the model proudly handed you in 30 seconds.If you don't trust the AI-generated implementation, write a slower obviously-correct version and compare. Run both on a hundred random inputs. If they ever disagree, the AI version is wrong. This catches off-by-ones and edge-case omissions almost every time.
import random
def percentile_oracle(values, p):
"""Slow, obviously correct, used only as ground truth."""
values = sorted(values)
n = len(values)
if n == 0:
return None
idx = max(0, min(n - 1, int(round(p / 100 * (n - 1)))))
return values[idx]
for _ in range(1000):
vals = [random.random() for _ in range(random.randint(1, 50))]
p = random.uniform(0, 100)
if percentile(vals, p) != percentile_oracle(vals, p):
print("DISAGREE:", vals, p)
break1000 random inputs, two implementations. Disagreement means somebody is wrong — usually the fast one.# After the AI writes a function, immediately:
"Now act as a hostile reviewer. Find three inputs where this function
behaves wrong, dangerously, or surprisingly. For each, show the input,
the actual output, and the expected output."
# About 60% of the time, the model finds its own bug.
# The other 40%, the bugs it lists ARE bugs but it didn't realize it wrote them.Adversarial self-review is a free verification pass. Add it to every code-generation prompt.The bug that compiles is worse than the bug that crashes.
— An old Lisp hacker, more relevant than ever
The big idea: AI optimizes for code that looks right, not code that is right. Build a habit of three-input verification, adversarial self-review, and differential testing for anything important. The compiled-and-running bug is the one that ships, and the one that hurts most.
15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-coding-debug-confidently-wrong-creators
What is the core idea behind "Confidently Wrong — When the AI Writes Plausible Nonsense"?
Which term best describes a foundational idea in "Confidently Wrong — When the AI Writes Plausible Nonsense"?
A learner studying Confidently Wrong — When the AI Writes Plausible Nonsense would need to understand which concept?
Which of these is directly relevant to Confidently Wrong — When the AI Writes Plausible Nonsense?
Which of the following is a key point about Confidently Wrong — When the AI Writes Plausible Nonsense?
Which of these does NOT belong in a discussion of Confidently Wrong — When the AI Writes Plausible Nonsense?
What is the key insight about "Confidence is not correctness" in the context of Confidently Wrong — When the AI Writes Plausible Nonsense?
What is the key insight about "Property-based testing pairs perfectly with AI code" in the context of Confidently Wrong — When the AI Writes Plausible Nonsense?
Which statement accurately describes an aspect of Confidently Wrong — When the AI Writes Plausible Nonsense?
What does working with Confidently Wrong — When the AI Writes Plausible Nonsense typically involve?
Which of the following is true about Confidently Wrong — When the AI Writes Plausible Nonsense?
Which best describes the scope of "Confidently Wrong — When the AI Writes Plausible Nonsense"?
Which section heading best belongs in a lesson about Confidently Wrong — When the AI Writes Plausible Nonsense?
Which section heading best belongs in a lesson about Confidently Wrong — When the AI Writes Plausible Nonsense?
Which section heading best belongs in a lesson about Confidently Wrong — When the AI Writes Plausible Nonsense?