Python async/await — Waiting Without Blocking

Why async matters for AI code

Every call to Claude or GPT takes 1–10 seconds. If your app makes 50 calls sequentially, it takes minutes. With asyncio, you fire all 50 at once and wait only as long as the slowest one. This single pattern is the difference between a toy script and a production app.

import asyncio import httpx async def fetch(client: httpx.AsyncClient, url: str) -> dict: response = await client.get(url, timeout=10) response.raise_for_status() return response.json() async def main() -> None: urls = [ "https://api.github.com/users/anthropic", "https://api.github.com/users/openai", "https://api.github.com/users/vercel", ] async with httpx.AsyncClient() as client: results = await asyncio.gather(*(fetch(client, u) for u in urls)) for data in results: print(data["login"], "-", data.get("bio", "")) asyncio.run(main())asyncio.gather runs all coroutines concurrently. Three API calls, one total wait.

The two words: async and await

• async def creates a coroutine function — calling it returns a coroutine object, it does not run yet
• await pauses until a coroutine finishes, freeing the event loop to run others
• asyncio.run starts the event loop; asyncio.gather runs many coroutines at once

Concurrency with rate limiting (the real world)

import asyncio from anthropic import AsyncAnthropic client = AsyncAnthropic() semaphore = asyncio.Semaphore(5) # max 5 concurrent requests async def summarize(text: str) -> str: async with semaphore: response = await client.messages.create( model="claude-opus-4-7", max_tokens=200, messages=[{"role": "user", "content": f"One sentence summary:\n{text}"}], ) return response.content[0].text async def main(): articles = ["text 1", "text 2", "text 20"] summaries = await asyncio.gather(*(summarize(a) for a in articles)) for s in summaries: print(s) asyncio.run(main())Semaphore caps concurrent requests so you don't hit the provider's rate limit.

Error handling across many coroutines

results = await asyncio.gather( *(fetch(client, u) for u in urls), return_exceptions=True, # don't fail-fast ) for url, result in zip(urls, results): if isinstance(result, Exception): print(f"Failed {url}: {result}") else: print(f"OK {url}")return_exceptions=True collects failures instead of aborting the whole batch — crucial for LLM calls.

Mini-exercise

1. Write an async function that calls the Anthropic API with a prompt
2. Call it concurrently on a list of 10 prompts using gather
3. Add a semaphore limiting to 3 concurrent calls
4. Measure total time with time.perf_counter — should be ~3x faster than sequential

Big idea: most AI code spends 99% of its time waiting. Async is how you stop waiting in series and start waiting in parallel.