Peer-Review Prep: Steelmanning Your Own Paper

Journal reviewers are overworked. They often focus on the easiest weaknesses to articulate: methods, stats, unclear writing, missing citations. You can have an LLM simulate this exact pass in 10 minutes.

The hostile-reviewer prompt

1. 1Run the hostile pass BEFORE you submit, not after rejection
2. 2Fix the easy stuff first (citations, clarity) — this alone saves rounds of revision
3. 3For the major revision issue, write a defense OR revise the paper
4. 4Save the LLM's review — if the real reviewers agree, you were on the right track

Response-to-reviewer drafts

After real peer review, LLMs are excellent at drafting polite, structured responses to reviewer comments. Ask for a 'response memo that addresses each comment numbered, with a one-line summary of the change and a pointer to the revised section.'

The big idea: catch your own weaknesses before reviewers do. A 10-minute AI pre-review buys you a round of revisions.

AI use in peer review raises confidentiality and quality concerns the field is just now codifying; reviewers should match the strictest applicable journal policy.

What AI does well here

• Disclose any AI use to the editor before completing the review
• Use only firm-deployed AI that contractually does not retain manuscript content
• Use AI for translation or terminology check, not for the substantive evaluation
• Document what you did with AI for transparency if asked

What AI cannot do

• Substitute AI evaluation for your own substantive review (you signed up to provide expert judgment)
• Upload manuscripts to consumer AI products (violates manuscript confidentiality)
• Hide AI use from the editor — discovery damages your reviewer reputation

AI can draft paper rebuttal-letter sections that map each reviewer comment to a specific revision and a response paragraph.

What AI does well here

• Mirror reviewer comments into a numbered response structure.
• Render the substantive-revision call-outs crisply.

What AI cannot do

• Decide whether the responses are scientifically adequate.
• Replace the corresponding author's judgment.

Editors love letters that quote each comment and respond beneath it. AI can format the letter; you write the substance.

What AI does well here

• Format Reviewer X / Comment N / Response / Changes-in-MS structure.
• Soften defensive language without conceding the point.
• Cross-reference page and line numbers when you supply them.

What AI cannot do

• Decide which reviewer comments to push back on.
• Make changes to the manuscript itself.
• Verify your responses are factually correct.

AI can take a journal scope and draft a peer review rubric with criteria, weights, and decision categories.

What AI does well here

• Produce consistent criterion descriptors per score point
• Generate decision-letter templates per recommendation

What AI cannot do

• Replace editorial judgment about field norms
• Substitute for reviewer expertise

Reviewer 2 stings; AI is the unflappable colleague who drafts a calm response while you cool down.

What AI does well here

• Reframe defensive impulses into evidence-based replies
• Structure point-by-point responses
• Suggest changes that genuinely address the critique
• Soften tone while keeping substance

What AI cannot do

• Decide which battles are worth fighting
• Carry your authority with the editor