Probabilistic Systems: Why LLMs Do Not Act Like Code

Writing software on top of an LLM is not like writing software on top of a database. Treat it as a stochastic system or it will bite you.

45 min · Reviewed 2026

LLMs Are Samplers

An LLM does not return an answer. It returns a probability distribution over possible next tokens. Sampling from that distribution produces text. Even identical prompts will produce different text unless you control the randomness explicitly.

The sampling knobs

Parameter	Effect
temperature	Scales logits. 0 = greedy, higher = more random
top_p (nucleus)	Keep smallest set of tokens whose cumulative probability exceeds p
top_k	Keep only the top k tokens
repetition_penalty	Down-weight tokens already in context
seed	Pin the pseudo-random generator for reproducibility

Why this breaks classical software instincts

Unit tests that check exact output strings are brittle
Caching by prompt hash works only at temperature 0 and even then not perfectly
Error handling must account for plausible wrong answers, not just failures
Rate limits and costs scale with token counts, which vary per call

# Robust evaluation of a probabilistic system
import statistics
import anthropic

client = anthropic.Anthropic()

def evaluate(prompt, n=10):
    outputs = []
    for _ in range(n):
        resp = client.messages.create(
            model="claude-opus-4-7",
            max_tokens=200,
            messages=[{"role": "user", "content": prompt}],
        )
        outputs.append(resp.content[0].text)
    # Measure pass@k, agreement, or similar
    return outputs

results = evaluate("Classify: 'great product'", n=10)
print(f"Unique outputs: {len(set(results))}")Sample multiple times and reason statistically. One call is anecdote, ten calls are data.

Strategies for taming randomness

Constrain output format (JSON schema, regex, tool calls)
Use structured generation libraries like Outlines, Instructor, or native JSON mode
Ensemble multiple samples and majority-vote
Use a cheaper verifier model to grade an expensive generator
Cache known-good outputs and fall through to LLM only on cache miss

Testing probabilistic systems

Behavioral tests: semantic properties (answer contains X, matches regex, passes a grader)
Distributional tests: 95 percent of samples must score above threshold
A/B tests in production with guardrails
Canary prompts that track drift between model versions

An LLM in production is a distribution, not a function. Design accordingly.
— A platform engineer

The big idea: LLMs are stochastic systems. The tooling, testing, and architecture patterns that assume deterministic code need to be re-thought from the ground up when you put one in the loop.

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-creators-probabilistic-systems

What is the core idea behind "Probabilistic Systems: Why LLMs Do Not Act Like Code"?
1. Writing software on top of an LLM is not like writing software on top of a database. Treat it as a stochastic system or it will bite you.
2. A picture model used by both a sticker app and a coloring app.
3. AI might not know who won a game last night.
4. The internet has the latest scores. AI might not.
Which term best describes a foundational idea in "Probabilistic Systems: Why LLMs Do Not Act Like Code"?
1. top-p
2. temperature
3. sampling
4. determinism
A learner studying Probabilistic Systems: Why LLMs Do Not Act Like Code would need to understand which concept?
1. temperature
2. sampling
3. top-p
4. determinism
Which of these is directly relevant to Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. temperature
2. top-p
3. determinism
4. sampling
Which of the following is a key point about Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. Unit tests that check exact output strings are brittle
2. Caching by prompt hash works only at temperature 0 and even then not perfectly
3. Error handling must account for plausible wrong answers, not just failures
4. Rate limits and costs scale with token counts, which vary per call
Which of these does NOT belong in a discussion of Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. Caching by prompt hash works only at temperature 0 and even then not perfectly
2. Error handling must account for plausible wrong answers, not just failures
3. A picture model used by both a sticker app and a coloring app.
4. Unit tests that check exact output strings are brittle
Which statement is accurate regarding Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. Use structured generation libraries like Outlines, Instructor, or native JSON mode
2. Ensemble multiple samples and majority-vote
3. Constrain output format (JSON schema, regex, tool calls)
4. Use a cheaper verifier model to grade an expensive generator
Which of these does NOT belong in a discussion of Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. Ensemble multiple samples and majority-vote
2. A picture model used by both a sticker app and a coloring app.
3. Use structured generation libraries like Outlines, Instructor, or native JSON mode
4. Constrain output format (JSON schema, regex, tool calls)
What is the key insight about "Temperature 0 is not fully deterministic" in the context of Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. Floating-point nondeterminism on GPUs, batched inference, and kv-cache rounding can still produce different outputs for …
2. A picture model used by both a sticker app and a coloring app.
3. AI might not know who won a game last night.
4. The internet has the latest scores. AI might not.
What is the key insight about "Beware hidden non-determinism" in the context of Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. A picture model used by both a sticker app and a coloring app.
2. Model versions shift, system prompts evolve, and upstream APIs change tokenizers.
3. AI might not know who won a game last night.
4. The internet has the latest scores. AI might not.
What is the key insight about "Review date" in the context of Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. A picture model used by both a sticker app and a coloring app.
2. AI might not know who won a game last night.
3. Reviewed in 2026. Treat fast-changing product names, prices, availability, and policy details as examples to verify befo…
4. The internet has the latest scores. AI might not.
Which statement accurately describes an aspect of Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. A picture model used by both a sticker app and a coloring app.
2. AI might not know who won a game last night.
3. The internet has the latest scores. AI might not.
4. An LLM does not return an answer. It returns a probability distribution over possible next tokens.
What does working with Probabilistic Systems: Why LLMs Do Not Act Like Code typically involve?
1. The big idea: LLMs are stochastic systems. The tooling, testing, and architecture patterns that assume deterministic code need to be re-thou…
2. A picture model used by both a sticker app and a coloring app.
3. AI might not know who won a game last night.
4. The internet has the latest scores. AI might not.
Which best describes the scope of "Probabilistic Systems: Why LLMs Do Not Act Like Code"?
1. It is unrelated to foundations workflows
2. It focuses on Writing software on top of an LLM is not like writing software on top of a database. Treat it as a s
3. It applies only to the opposite beginner tier
4. It was deprecated in 2024 and no longer relevant
Which section heading best belongs in a lesson about Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. A picture model used by both a sticker app and a coloring app.
2. AI might not know who won a game last night.
3. The sampling knobs
4. The internet has the latest scores. AI might not.

← Back to interactive lesson

Tendril · Creators · AI Foundations

Probabilistic Systems: Why LLMs Do Not Act Like Code

Writing software on top of an LLM is not like writing software on top of a database. Treat it as a stochastic system or it will bite you.

45 min · Reviewed 2026

LLMs Are Samplers

The sampling knobs

Parameter	Effect
temperature	Scales logits. 0 = greedy, higher = more random
top_p (nucleus)	Keep smallest set of tokens whose cumulative probability exceeds p
top_k	Keep only the top k tokens
repetition_penalty	Down-weight tokens already in context
seed	Pin the pseudo-random generator for reproducibility

Why this breaks classical software instincts

Unit tests that check exact output strings are brittle
Caching by prompt hash works only at temperature 0 and even then not perfectly
Error handling must account for plausible wrong answers, not just failures
Rate limits and costs scale with token counts, which vary per call

# Robust evaluation of a probabilistic system
import statistics
import anthropic

client = anthropic.Anthropic()

def evaluate(prompt, n=10):
    outputs = []
    for _ in range(n):
        resp = client.messages.create(
            model="claude-opus-4-7",
            max_tokens=200,
            messages=[{"role": "user", "content": prompt}],
        )
        outputs.append(resp.content[0].text)
    # Measure pass@k, agreement, or similar
    return outputs

results = evaluate("Classify: 'great product'", n=10)
print(f"Unique outputs: {len(set(results))}")Sample multiple times and reason statistically. One call is anecdote, ten calls are data.

Strategies for taming randomness

Constrain output format (JSON schema, regex, tool calls)
Use structured generation libraries like Outlines, Instructor, or native JSON mode
Ensemble multiple samples and majority-vote
Use a cheaper verifier model to grade an expensive generator
Cache known-good outputs and fall through to LLM only on cache miss

Testing probabilistic systems

Behavioral tests: semantic properties (answer contains X, matches regex, passes a grader)
Distributional tests: 95 percent of samples must score above threshold
A/B tests in production with guardrails
Canary prompts that track drift between model versions

An LLM in production is a distribution, not a function. Design accordingly.
— A platform engineer

The big idea: LLMs are stochastic systems. The tooling, testing, and architecture patterns that assume deterministic code need to be re-thought from the ground up when you put one in the loop.

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-creators-probabilistic-systems

What is the core idea behind "Probabilistic Systems: Why LLMs Do Not Act Like Code"?
1. Writing software on top of an LLM is not like writing software on top of a database. Treat it as a stochastic system or it will bite you.
2. A picture model used by both a sticker app and a coloring app.
3. AI might not know who won a game last night.
4. The internet has the latest scores. AI might not.
Which term best describes a foundational idea in "Probabilistic Systems: Why LLMs Do Not Act Like Code"?
1. top-p
2. temperature
3. sampling
4. determinism
A learner studying Probabilistic Systems: Why LLMs Do Not Act Like Code would need to understand which concept?
1. temperature
2. sampling
3. top-p
4. determinism
Which of these is directly relevant to Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. temperature
2. top-p
3. determinism
4. sampling
Which of the following is a key point about Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. Unit tests that check exact output strings are brittle
2. Caching by prompt hash works only at temperature 0 and even then not perfectly
3. Error handling must account for plausible wrong answers, not just failures
4. Rate limits and costs scale with token counts, which vary per call
Which of these does NOT belong in a discussion of Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. Caching by prompt hash works only at temperature 0 and even then not perfectly
2. Error handling must account for plausible wrong answers, not just failures
3. A picture model used by both a sticker app and a coloring app.
4. Unit tests that check exact output strings are brittle
Which statement is accurate regarding Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. Use structured generation libraries like Outlines, Instructor, or native JSON mode
2. Ensemble multiple samples and majority-vote
3. Constrain output format (JSON schema, regex, tool calls)
4. Use a cheaper verifier model to grade an expensive generator
Which of these does NOT belong in a discussion of Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. Ensemble multiple samples and majority-vote
2. A picture model used by both a sticker app and a coloring app.
3. Use structured generation libraries like Outlines, Instructor, or native JSON mode
4. Constrain output format (JSON schema, regex, tool calls)
What is the key insight about "Temperature 0 is not fully deterministic" in the context of Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. Floating-point nondeterminism on GPUs, batched inference, and kv-cache rounding can still produce different outputs for …
2. A picture model used by both a sticker app and a coloring app.
3. AI might not know who won a game last night.
4. The internet has the latest scores. AI might not.
What is the key insight about "Beware hidden non-determinism" in the context of Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. A picture model used by both a sticker app and a coloring app.
2. Model versions shift, system prompts evolve, and upstream APIs change tokenizers.
3. AI might not know who won a game last night.
4. The internet has the latest scores. AI might not.
What is the key insight about "Review date" in the context of Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. A picture model used by both a sticker app and a coloring app.
2. AI might not know who won a game last night.
3. Reviewed in 2026. Treat fast-changing product names, prices, availability, and policy details as examples to verify befo…
4. The internet has the latest scores. AI might not.
Which statement accurately describes an aspect of Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. A picture model used by both a sticker app and a coloring app.
2. AI might not know who won a game last night.
3. The internet has the latest scores. AI might not.
4. An LLM does not return an answer. It returns a probability distribution over possible next tokens.
What does working with Probabilistic Systems: Why LLMs Do Not Act Like Code typically involve?
1. The big idea: LLMs are stochastic systems. The tooling, testing, and architecture patterns that assume deterministic code need to be re-thou…
2. A picture model used by both a sticker app and a coloring app.
3. AI might not know who won a game last night.
4. The internet has the latest scores. AI might not.
Which best describes the scope of "Probabilistic Systems: Why LLMs Do Not Act Like Code"?
1. It is unrelated to foundations workflows
2. It focuses on Writing software on top of an LLM is not like writing software on top of a database. Treat it as a s
3. It applies only to the opposite beginner tier
4. It was deprecated in 2024 and no longer relevant
Which section heading best belongs in a lesson about Probabilistic Systems: Why LLMs Do Not Act Like Code?
1. A picture model used by both a sticker app and a coloring app.
2. AI might not know who won a game last night.
3. The sampling knobs
4. The internet has the latest scores. AI might not.

← Back to interactive lesson