A Methodologist, Domain Expert, and Editor run in parallel — each mapped to a distinct model. The meta-reviewer merges them, and when they disagree, it defers to the more critical read. Here's why that choice matters.
AI Peer Review runs a three-reviewer panel in parallel — Methodologist (OpenAI's reasoning model), Domain Expert (Anthropic's long-context model), Editor (Google's frontier model) — each with a role-specific prompt. A meta-reviewer synthesizes their reports into one verdict, one score per dimension, and a prioritized action list. Where reviewers disagree, the synthesis defaults to the harsher read; where they agree, the finding is marked HIGH CONFIDENCE. References are verified in parallel via CrossRef/Semantic Scholar. Runs in 30–60 seconds, costs 35 credits, produces a downloadable report.
Human peer review is slow, expensive, and often inconsistent — median time-to-first-decision at good journals is 3–6 months, and studies of reviewer agreement put inter-rater reliability at 0.2–0.4 (weak). For a PhD student or early-career researcher, getting a substantive read of a draft before submission is one of the hardest resources to secure. Supervisors are busy; friendly reviewers are rare; and the paid pre-submission review services cost hundreds of pounds.
A single-model AI review has an obvious failure mode: whatever biases the model has in academic domains, it applies uniformly. Ask GPT alone to review a methods paper and it will consistently overweight some criticisms and underweight others — but it will be internally consistent about it, which reads as authoritative even when it's wrong.
The design here is a bet: three models with distinct strengths, given distinct roles, will surface a richer set of real issues than one model doing everything. When they agree, the finding is more trustworthy; when they disagree, that disagreement is itself signal — either a genuine judgment call the author should weigh, or a bias in one model that the others correct.
Model-to-role assignments aren't arbitrary — each model was chosen for the specific critique style the role needs.
Focus: Study design, statistical validity, replicability, threats to internal/external validity, sample-size adequacy, control for confounds.
OpenAI's chain-of-reasoning is strongest on structured, rule-bounded critique — the kind of "does the design actually test the hypothesis" review that requires walking through logical dependencies. Assigning it the Methodologist role uses its strength.
Focus: Novelty vs prior work, positioning within the sub-field, appropriateness of chosen framework, whether cited work is used correctly, disciplinary conventions.
Anthropic's models produce the most nuanced qualitative writing and hold broader context reliably across long documents. Domain critique needs both — you have to weigh what's said against a wide backdrop of what could have been said.
Focus: Clarity of writing, structure of the argument, whether the abstract matches the paper, whether findings support the conclusions, publishability for the stated venue.
Google's models produce a direct, prescriptive editorial voice — closer to a real journal editor's tone than the more accommodating output of the others. They're also the fastest on this kind of pass, which matters when three reviewers are running in parallel.
Five stages · typical end-to-end runtime 30–60 seconds
The paper is parsed into structured sections: title, abstract, introduction, methodology, results, discussion, references. Missing sections are noted (they become findings in their own right — a missing methodology section is a substantive review point). Word count and reference count are computed for later use.
Each reviewer needs the whole paper, but reviewers focus on different sections. Pre-parsing means each reviewer can be prompted with the specific sections most relevant to their role, without wasting tokens on sections they'll ignore.
The three reviewer prompts fire in parallel, each mapped to a distinct model. Each prompt embeds the reviewer's role, their focus areas, the target venue (if specified), and a strict output schema. Simultaneously, the paper's references are verified via CrossRef/Semantic Scholar so citation issues can be included in the domain reviewer's feedback.
Running the same review three times against three different critics catches signals that any single reviewer would miss. Role assignment prevents the three reviewers from converging on the same easy points and ignoring the rest — a Methodologist won't spend paragraphs on prose style, an Editor won't rewrite the stats section.
If any of the three primary models fail to return valid JSON (rare, but happens), Claude is called with the same reviewer prompt as a fallback. This means a run only fails if BOTH the primary model AND the Claude fallback fail on the same reviewer role — extremely rare. When even one reviewer returns nothing, the run continues with the remaining two and the confidence signal reflects that.
The alternative — bailing on the whole run because one model timed out — throws away good work and burns the user's credits for nothing. Falling back keeps the run useful; downgrading confidence keeps the report honest about what actually happened.
The three reviewer reports are handed to a synthesis prompt (an Anthropic model by default, OpenAI as fallback). Its job: merge the three, resolve disagreements, produce ONE final verdict and ONE consolidated set of strengths, weaknesses, and action items. Where reviewers agree, findings are marked HIGH CONFIDENCE. Where they diverge, the more critical read wins — a paper that one reviewer would accept and another would reject gets a Major Revision, not an Accept.
This is where the panel model earns its keep. Averaging scores would let a lenient reviewer wash out a valid criticism from a strict one. Defaulting to the more critical read means findings that would embarrass an author in real peer review get surfaced, not smoothed over. The synthesis prompt is explicit about this asymmetry.
The final output is annotated with a consensus level: HIGH (all three reviewers returned valid reports and the meta-synthesis completed cleanly), MEDIUM (2 of 3 reviewers succeeded), LOW (only 1 reviewer succeeded — the run happened but cross-validation was limited). This becomes the ConsensusBanner at the top of the report so the user can see immediately whether they got a full panel review or a partial one, and re-run if needed.
A one-reviewer report looks identical to a three-reviewer report unless we tell the user. Trust signals need to be visible at the moment of decision, not buried in a footer. If the run was partial, we say so — and suggest re-running, because in practice the second attempt usually gets all three models back.
The meta-reviewer's rules aren't arbitrary — three real scenarios and how the synthesis handles them.
Two reviewers say Major Revision, one says Reject
Resolution: Verdict: Major Revision. The lone Reject is preserved verbatim in the report so the author sees the harshest read alongside the consensus. If the Reject reviewer flags an issue the other two didn't (e.g., "the entire methodology is inappropriate for the research question"), the meta-synthesis promotes that into the top-priority action item.
One reviewer scores Originality 8, two score it 4–5
Resolution: Consensus score: 4.5 (the more critical average). The high score gets a note in the report — "the Domain Expert rated originality substantially higher because [reason from their reasoning field]" — so the author sees the disagreement, not just the aggregated number.
Reviewers agree on the verdict but disagree on the reason
Resolution: All reasons are preserved; the meta-reviewer clusters them into distinct weakness categories. This is often the most useful output — three reviewers reaching Major Revision via different paths tells the author there are multiple independent issues, not one debate.
The single most important design choice in the meta-reviewer is what to do when reviewers disagree. Two options: average (democratic, feels fair) or defer to the harshest read (asymmetric, favors caution). We chose the harsh read, deliberately.
The reasoning: this tool exists to help authors catch problems before their supervisor or a real reviewer catches them. A criticism that's real but only surfaced by one of three reviewers is still real — averaging it away turns a substantive issue into a soft footnote. And the failure modes are asymmetric: telling an author about a criticism that turns out to be minor costs them ten minutes of thinking; missing a criticism that turns out to be major costs them a rejection.
That's why the meta-reviewer's synthesis prompt is explicit: "If reviewers disagree on a verdict, use the more critical one. If a criticism appears in only one review, include it but mark as MEDIUM CONFIDENCE." The lenient path exists in the report — the author can see who thought the paper was fine — but the headline verdict reflects the harshest coherent read.
450 credits per run. New accounts get their first task free (any task up to 250 credits). PDF, DOCX, or pasted text.