Lesson 1769 of 2116
Output Format Engineering: Schemas, Length Control, and Reliability, Part 2
Replace 'please return JSON' instructions with structured-output features so downstream code never has to parse around model whims.
Lesson map
What this lesson covers
Learning path
The main moves in order
- 1The premise
- 2AI and output format contracts
- 3The premise
- 4The Spec Pattern: Tell the AI What 'Done' Means
Concept cluster
Terms to connect while reading
Section 1
The premise
Asking the model nicely for JSON works most of the time and fails at 3am; structured-output schemas turn format from a wish into a contract enforced by the API.
What AI does well here
- Define a JSON schema for the expected output
- Use the provider's structured-output mode (function/tool calling)
- Validate on receipt and retry with the validation error
- Version the schema as the prompt evolves
What AI cannot do
- Guarantee semantic correctness of fields
- Replace business validation (e.g., 'this email exists')
- Eliminate the need for a post-receipt sanity check
Key terms in this lesson
Section 2
AI and output format contracts
Section 3
The premise
Free-form output is fragile downstream. A clear schema, ideally enforced by the API's structured output mode, prevents most parsing bugs.
What AI does well here
- Convert prose instructions into a JSON schema.
- Suggest field types and enums.
- Add a 'reason' or 'confidence' field where useful.
What AI cannot do
- Stop the model from filling fields with junk.
- Validate semantic correctness.
- Replace runtime schema validation.
Section 4
The Spec Pattern: Tell the AI What 'Done' Means
Section 5
The premise
Most bad outputs come from missing success criteria, not weak models. State what 'done' looks like before asking for anything.
What AI does well here
- Hit a clearly stated format and length budget.
- Self-check against criteria you spelled out.
What AI cannot do
- Infer your unstated quality bar.
- Decide which trade-offs you prefer.
Section 6
Format Contracts: JSON Schemas in the Prompt
Section 7
The premise
Free-form text is hard to consume downstream; a schema in the prompt plus a validator on the output makes integration tractable.
What AI does well here
- Match a schema you state explicitly.
- Repair an output when given the validator error.
What AI cannot do
- Always emit valid JSON without validation.
- Invent the right schema for your downstream system.
Section 8
Stripping Hedge Words From Model Output
Section 9
The premise
Default model output is full of 'it depends' and 'generally speaking'. You can instruct it to commit, with a separate channel for genuine uncertainty.
What AI does well here
- Drop filler hedges when told to.
- Mark genuine uncertainty in a structured field instead.
What AI cannot do
- Be more certain than its training warrants.
- Tell you when its commitment is wrong.
Section 10
Setting Length Budgets That the Model Actually Hits
Section 11
The premise
Models are mediocre at exact word counts but good at structural budgets. Specify both the structure and an approximate length.
What AI does well here
- Hit structural targets like '5 bullets, each <= 12 words'.
- Approximate paragraph counts and section headings.
What AI cannot do
- Hit an exact word count reliably.
- Self-trim when the content does not fit.
Section 12
Scaffolded Reasoning: Give the AI a Worked Solution Skeleton
Section 13
The premise
Models reason better when the reasoning shape is pre-built. A skeleton with labeled slots forces complete coverage.
What AI does well here
- Fill named slots in a template you supply.
- Stay within sections you've defined.
- Maintain consistent formatting across many outputs.
- Avoid skipping sections when each is required.
What AI cannot do
- Invent a useful structure when your skeleton is wrong.
- Push back on a flawed template even when it produces nonsense.
Section 14
Format Locking: Pin AI Output Shape with JSON Schemas
Section 15
The premise
'Return JSON' is fragile; pasting an actual schema with required fields and types is reliable.
What AI does well here
- Match a JSON schema you provide closely.
- Include all required fields when schema lists them.
- Use enums you define rather than free strings.
- Stay within types you specified.
What AI cannot do
- Validate JSON itself — invalid output still happens.
- Guarantee schema compliance without a programmatic validator.
Section 16
AI Output Format Control: JSON, Schemas, and Structured Generation
Section 17
The premise
AI structured output reliability comes from a stack: schema declaration, format examples, constrained decoding when available, and validation with retry on failure.
What AI does well here
- Producing valid JSON when given a clear schema
- Following format examples shown in the prompt
- Honoring constrained-decoding grammars at runtime
- Self-correcting when shown a validation error
What AI cannot do
- Guarantee schema conformance without constrained decoding
- Invent reasonable values for required fields with no signal
Tutor
Curious about “Output Format Engineering: Schemas, Length Control, and Reliability, Part 2”?
Ask anything about this lesson. I’ll answer using just what you’re reading — short, friendly, grounded.
Progress saved locally in this browser. Sign in to sync across devices.
Related lessons
Keep going
Creators · 40 min
Output Format Engineering: Schemas, Length Control, and Reliability, Part 1
If you're parsing model output in code, format reliability matters as much as content quality. Here's how to architect prompts and validators that produce parseable output even from imperfect models.
Builders · 40 min
Output Format Control: JSON, Tables, Schemas, and Structure
Tell AI the shape of the answer (table, bullets, JSON) and you stop wasting time reformatting.
Explorers · 40 min
Format Your Answers: Lists, Tables, Length, and Layout, Part 2
You can ask AI for short, medium, or long answers — your choice.