Probabilistic Systems: Why LLMs Do Not Act Like Code

Writing software on top of an LLM is not like writing software on top of a database. Treat it as a stochastic system or it will bite you.

45 min · Reviewed 2026

LLMs Are Samplers

An LLM does not return an answer. It returns a probability distribution over possible next tokens. Sampling from that distribution produces text. Even identical prompts will produce different text unless you control the randomness explicitly.

The sampling knobs

Parameter	Effect
temperature	Scales logits. 0 = greedy, higher = more random
top_p (nucleus)	Keep smallest set of tokens whose cumulative probability exceeds p
top_k	Keep only the top k tokens
repetition_penalty	Down-weight tokens already in context
seed	Pin the pseudo-random generator for reproducibility

Why this breaks classical software instincts

Unit tests that check exact output strings are brittle
Caching by prompt hash works only at temperature 0 and even then not perfectly
Error handling must account for plausible wrong answers, not just failures
Rate limits and costs scale with token counts, which vary per call

# Robust evaluation of a probabilistic system import statistics import anthropic client = anthropic.Anthropic() def evaluate(prompt, n=10): outputs = [] for _ in range(n): resp = client.messages.create( model="claude-opus-4-7", max_tokens=200, messages=[{"role": "user", "content": prompt}], ) outputs.append(resp.content[0].text) # Measure pass@k, agreement, or similar return outputs results = evaluate("Classify: 'great product'", n=10) print(f"Unique outputs: {len(set(results))}")Sample multiple times and reason statistically. One call is anecdote, ten calls are data.

Strategies for taming randomness

Constrain output format (JSON schema, regex, tool calls)
Use structured generation libraries like Outlines, Instructor, or native JSON mode
Ensemble multiple samples and majority-vote
Use a cheaper verifier model to grade an expensive generator
Cache known-good outputs and fall through to LLM only on cache miss

Testing probabilistic systems

Behavioral tests: semantic properties (answer contains X, matches regex, passes a grader)
Distributional tests: 95 percent of samples must score above threshold
A/B tests in production with guardrails
Canary prompts that track drift between model versions

An LLM in production is a distribution, not a function. Design accordingly.
— A platform engineer

The big idea: LLMs are stochastic systems. The tooling, testing, and architecture patterns that assume deterministic code need to be re-thought from the ground up when you put one in the loop.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-creators-probabilistic-systems

What is the main idea of "Probabilistic Systems: Why LLMs Do Not Act Like Code"?
1. Writing software on top of an LLM is not like writing software on top of a database. Treat it as a stochastic system or it will bite you.
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "Probabilistic Systems: Why LLMs Do Not Act Like Code"?
1. temperature
2. probabilistic
3. determinism
4. sampling
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. Unit tests that check exact output strings are brittle
4. Treat the AI output as automatically correct
What should a careful learner remember about "Temperature 0 is not fully deterministic"?
1. Use AI to draft or organize ideas about probabilistic, then verify before acting.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. Use AI for drafting and comparison, but verify before publishing or relying on it.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about probabilistic be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about probabilistic.
Which action would help you apply "Probabilistic Systems: Why LLMs Do Not Act Like Code" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Treat the AI output as automatically correct
4. Caching by prompt hash works only at temperature 0 and even then not perfectly

← Back to interactive lesson

Tendril · Creators · AI Foundations

Probabilistic Systems: Why LLMs Do Not Act Like Code

Writing software on top of an LLM is not like writing software on top of a database. Treat it as a stochastic system or it will bite you.

45 min · Reviewed 2026

LLMs Are Samplers

The sampling knobs

Parameter	Effect
temperature	Scales logits. 0 = greedy, higher = more random
top_p (nucleus)	Keep smallest set of tokens whose cumulative probability exceeds p
top_k	Keep only the top k tokens
repetition_penalty	Down-weight tokens already in context
seed	Pin the pseudo-random generator for reproducibility

Why this breaks classical software instincts

Unit tests that check exact output strings are brittle
Caching by prompt hash works only at temperature 0 and even then not perfectly
Error handling must account for plausible wrong answers, not just failures
Rate limits and costs scale with token counts, which vary per call

# Robust evaluation of a probabilistic system import statistics import anthropic client = anthropic.Anthropic() def evaluate(prompt, n=10): outputs = [] for _ in range(n): resp = client.messages.create( model="claude-opus-4-7", max_tokens=200, messages=[{"role": "user", "content": prompt}], ) outputs.append(resp.content[0].text) # Measure pass@k, agreement, or similar return outputs results = evaluate("Classify: 'great product'", n=10) print(f"Unique outputs: {len(set(results))}")Sample multiple times and reason statistically. One call is anecdote, ten calls are data.

Strategies for taming randomness

Constrain output format (JSON schema, regex, tool calls)
Use structured generation libraries like Outlines, Instructor, or native JSON mode
Ensemble multiple samples and majority-vote
Use a cheaper verifier model to grade an expensive generator
Cache known-good outputs and fall through to LLM only on cache miss

Testing probabilistic systems

Behavioral tests: semantic properties (answer contains X, matches regex, passes a grader)
Distributional tests: 95 percent of samples must score above threshold
A/B tests in production with guardrails
Canary prompts that track drift between model versions

An LLM in production is a distribution, not a function. Design accordingly.
— A platform engineer

The big idea: LLMs are stochastic systems. The tooling, testing, and architecture patterns that assume deterministic code need to be re-thought from the ground up when you put one in the loop.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-creators-probabilistic-systems

What is the main idea of "Probabilistic Systems: Why LLMs Do Not Act Like Code"?
1. Writing software on top of an LLM is not like writing software on top of a database. Treat it as a stochastic system or it will bite you.
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "Probabilistic Systems: Why LLMs Do Not Act Like Code"?
1. temperature
2. probabilistic
3. determinism
4. sampling
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. Unit tests that check exact output strings are brittle
4. Treat the AI output as automatically correct
What should a careful learner remember about "Temperature 0 is not fully deterministic"?
1. Use AI to draft or organize ideas about probabilistic, then verify before acting.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. Use AI for drafting and comparison, but verify before publishing or relying on it.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about probabilistic be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about probabilistic.
Which action would help you apply "Probabilistic Systems: Why LLMs Do Not Act Like Code" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Treat the AI output as automatically correct
4. Caching by prompt hash works only at temperature 0 and even then not perfectly

← Back to interactive lesson