Red-Teaming Your AI-Generated Code

AI agents optimize for making the thing work. They rarely optimize for secure. Studies from Stanford and others have repeatedly shown AI-generated code is more likely to contain security vulnerabilities than human-written code, and developers using AI are more confident in that code. That combination is the problem.

Classes of bugs to hunt for

• Injection: SQL, shell, XSS, prompt — untrusted input reaching a dangerous sink
• AuthN/AuthZ: missing auth checks, wrong ownership assumptions, IDOR
• Secrets handling: API keys in logs, hardcoded tokens, exposed .env
• Dependency supply chain: invented packages, typosquats, unpinned versions
• Server-side request forgery: fetching user-supplied URLs without validation
• Prompt injection: user content reaching another LLM call without sanitization

A practical red-team prompt

You are a security red-teamer. Review the code below for:

1. Injection vulnerabilities (SQL, shell, XSS, prompt).
2. Authentication/authorization gaps.
3. Secrets exposure in logs, responses, or error messages.
4. Dependency issues (hallucinated packages, unpinned versions).
5. Any other class of vulnerability you recognize.

For each finding:
- Severity (critical/high/medium/low)
- File and line
- Attack scenario in one sentence
- Specific fix

Do not suggest stylistic or non-security improvements.

[paste diff or file]

The dependency supply chain trap

AI models regularly hallucinate package names that do not exist. Worse, attackers have begun publishing malicious packages under common hallucinated names — a pattern called slopsquatting. Before installing any dependency an AI recommends, verify it exists on the real registry and check download counts and recent commit activity.

# Verify an npm package before installing
npm view some-package

# Check download history and maintainers
npm view some-package time maintainers versions

# Audit existing dependencies for known CVEs
npm audit --production

# Pin exact versions in production
npm install --save-exact some-package

Automated scanners to run in CI

Prompt injection is the new XSS

If your app feeds user content to an LLM — anywhere — treat that content as untrusted input that can issue instructions. Sanitize, quarantine, and never let tool-calling agents receive raw user input without scoping. The attack surface grows with every tool you add.

The weekly habit

1. 1Run SAST and dependency scans in CI on every PR
2. 2Do a manual red-team prompt review on any new auth, payments, or user-input code
3. 3Subscribe to CVE alerts for your stack
4. 4Rotate API keys quarterly, even if no breach
5. 5Keep a security.md file describing what data you hold and how to report issues

“Attackers have agents too. The only defense is assuming yours is being tested right now.”

The big idea: AI makes shipping easy, which makes shipping insecure code easier. Red-teaming is no longer optional — it's the habit that separates toys from products.