Hermes For Structured JSON Output: Schemas That Work

When you need data, not prose, an open-weight model has to play by a schema. Hermes is one of the more reliable choices — but only if you prompt it carefully.

9 min · Reviewed 2026

Why JSON-from-LLMs is harder than it looks

Asking a model for JSON is easy. Asking it for a JSON object that always matches your schema is hard. Frontier API models offer schema-strict modes; open-weight models often need help. Hermes is responsive to good instructions, and when paired with grammar-constrained decoding (available in llama.cpp / Ollama), it can be very reliable.

Schemas that work in practice

Keep the schema flat where possible — fewer levels of nesting means fewer chances for the model to drop a brace.
Use enum lists for categorical fields — 'category' should be one of a fixed list, not free text.
Always include an 'id' field that echoes input — easier to map outputs back.
Add a 'confidence' field when you can — useful for routing low-confidence cases to a human.
Provide ONE example of the exact output you want, formatted as the schema, in the prompt.

Prompt skeleton: SYSTEM: You will receive an input. Return ONLY a JSON object matching this schema. Do not add commentary, do not wrap in code fences: { "id": string, // echo input id "category": one of ["a","b","c"], "summary": string (max 30 words), "confidence": number 0.0-1.0 } Example output for an input id="x1": {"id":"x1","category":"b","summary":"","confidence":0.78} Now process the input below.An example output beats three sentences of explanation about the schema.

Grammar-constrained decoding

llama.cpp supports a grammar feature that physically prevents the model from emitting tokens that violate a JSON schema. When available, it is the strongest reliability tool in your kit — schema violations become impossible, not unlikely. Both Ollama and LM Studio expose access to this feature.

Approach	Reliability	Setup effort	Trade-off
Plain prompt with example	Good	Low	Occasional drift on edge cases
Prompt + retry on parse failure	Better	Low	Slower on bad runs
Grammar-constrained decoding	Best	Medium	Schema must be expressible as a grammar
Full schema-validating loop	Excellent	Higher	Most code to maintain

Applied exercise

Pick a real classification or extraction task you do.
Define a flat JSON schema for the output.
Prompt Hermes with the skeleton above and run on 25 inputs.
Compute schema-failure rate. If above 5%, try grammar-constrained decoding and recompute.

The big idea: structured output from open-weight models is solvable. Use grammar constraints when you can, validate always, and never trust the model to remember the schema mid-stream.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-hermes-structured-json-creators

What is the main idea of "Hermes For Structured JSON Output: Schemas That Work"?
1. When you need data, not prose, an open-weight model has to play by a schema.
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "Hermes For Structured JSON Output: Schemas That Work"?
1. JSON schema
2. structured output
3. grammar-constrained decoding
4. validation
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. Keep the schema flat where possible — fewer levels of nesting means fewer chances for the model to drop a brace.
4. Treat the AI output as automatically correct
What should a careful learner remember about "Validate, don't pray"?
1. Use AI to draft or organize ideas about structured output, then verify before acting.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. Use AI for drafting and comparison, but verify before publishing or relying on it.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about structured output be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about structured output.
Which action would help you apply "Hermes For Structured JSON Output: Schemas That Work" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Treat the AI output as automatically correct
4. Use enum lists for categorical fields — 'category' should be one of a fixed list, not free text.

← Back to interactive lesson

Tendril · Creators · Model Families