Production Agent Patterns: Queues, Retries, Idempotency

Why production is different

In a prototype, a crash is fine — you rerun. In production, a crash means a user's pizza never got ordered and a $4 LLM call got burned. Production agents must be durable, idempotent, observable, and cost-capped. Most teams discover this after shipping a demo.

The five production requirements

Durable state pattern

Agents that run longer than a few seconds shouldn't live in memory. Checkpoint after every step. Options in 2026:

• Vercel Workflow DevKit (WDK) — step-based, crash-safe, powered by Queues.
• LangGraph + PostgresSaver — durable state machines.
• Temporal — mature workflow engine; strong for multi-day flows.
• Inngest — event-driven steps with retries and concurrency controls.
• Roll your own — Postgres + a state column + a worker loop.

// Vercel Workflow DevKit — modern "use workflow" directive // Models are addressed via the AI Gateway alias format. import { step } from 'workflow'; import { generateText } from 'ai'; export async function researchAgent(goal: string) { 'use workflow'; const plan = await step('plan', async () => { const { text } = await generateText({ model: 'anthropic/claude-opus-4.7', prompt: `Break into sub-questions:\n${goal}`, }); return text.split('\n'); }); const findings = []; for (const q of plan) { const answer = await step(`research:${q.slice(0, 20)}`, async () => { return await searchAndSummarize(q); }, { retries: 3, timeout: '60s' }); findings.push({ q, answer }); } return await step('synthesize', async () => { const { text } = await generateText({ model: 'anthropic/claude-opus-4.7', prompt: `Write a cited answer:\n${JSON.stringify(findings)}`, }); return text; }); }Every step() is durable. If the process dies, execution resumes from the last completed step. Built-in retries + timeouts.

Idempotency — the underrated superpower

Any step that touches the outside world (send email, charge card, create ticket) needs an idempotency key. When the step retries, the external system recognizes the key and doesn't duplicate the action.

// Idempotent Stripe charge const chargeId = `task:${taskId}:step:${stepName}`; const charge = await stripe.paymentIntents.create( { amount: 5000, currency: 'usd', customer: custId }, { idempotencyKey: chargeId } ); // Same chargeId on retry returns the same charge — no double-billing.Every external API call gets an idempotency key derived from workflow state. Most APIs (Stripe, SendGrid, Twilio) support this natively.

Retry policy

Observability essentials

• Trace every LLM call with prompt, response, token counts, cost — OpenTelemetry + LangSmith/Braintrust/Vercel Observability.
• Log every tool call with args, result, latency.
• Record every workflow event (start, step complete, retry, fail).
• Cost dashboards per workflow/agent/user.
• Alerts on cost spikes, error spikes, latency regressions.

Cost and step caps

const MAX_STEPS = 50; const MAX_COST_USD = 2.00; let stepCount = 0; let costUsd = 0; while (!isDone(state)) { if (++stepCount > MAX_STEPS) { throw new Error('Step cap exceeded — possible loop.'); } const { result, usage } = await runOneStep(state); costUsd += usage.inputTokens * 3/1_000_000 + usage.outputTokens * 15/1_000_000; if (costUsd > MAX_COST_USD) { throw new Error(`Cost cap exceeded: $${costUsd.toFixed(2)}`); } state = applyResult(state, result); }Non-negotiable ceilings. Cheaper to fail a task than to let a loop burn $400 of Opus calls at 3 AM.

Next: the security dimension. An agent is a new attack surface.