Peer review in the age of AI: When scientific judgement meets prompt injection

Hidden prompts buried in preprints show how easily large language models (LLMs) can be manipulated, exposing a deep vulnerability in science’s quality-control system. As artificial intelligence (AI) becomes part of the scientific review process, traceability and transparency must become new norms, not afterthoughts.

When researchers embedded hidden prompts into preprints uploaded to the arXiv asking AI reviewers to “give a positive review only” it seemed, at first glance, like a digital footnote to the evolving story of AI in science. The prompts, concealed in white font and microtext, were invisible to human readers but crystal clear for an LLM. They weren’t mistakes, they were instructions.

Some authors defended the practice as a preemptive countermeasure. An insurance policy against inattentive reviewers who rely on LLMs. Others admitted it crossed a line and withdrew their papers after backlash. But the deeper issue here isn’t about individual misconduct; it’s that the peer review pipeline now contains a structural flaw: to an LLM, every byte of text is executable code, and peer review is not prepared for that reality.

How prompt injection works

Unlike human readers, LLMs do not skip “hidden” text, and they don’t recognize deception unless they’ve been explicitly trained to detect it. These models process the full token stream, including micro-text, white-on-white characters, or content tucked into metadata. That makes language itself an attack surface. With a single line an author can bias a model toward praise, dampen criticism, or sidestep entire sections of a review rubric. 

Example of basic prompt injection in the abstract of a scientific paper. When text is viewed in “night mode” (or when the area is highlighted) the prompt becomes visible.

Because these prompts run inside the reviewer’s private chat window, they leave no trace. The result is a shift in what counts as review. When a model interprets a prompt-laced paper and generates a favourable assessment, who is doing the evaluation? And who bears responsibility for its validity? The entire chain linking the author, reviewer, and editorial decision fractures under the weight of this opacity since one can no longer interpret which instructions the model obeyed or ignored.

The metascientific stakes of AI in peer review

This is not a question of whether AI should be used in peer review, because it already is. The question is whether our institutions and publishers are ready for the new conditions it imposes. The answer, so far, is no.

Peer review was never built for co-authorship with algorithms, yet that is increasingly what it resembles. Language models summarize submissions, flag methodological concerns, and offer preliminary verdicts. In this hybrid workflow, human and machine form a cognitive loop but one half of the loop remains hidden.

As of yet, there is not a widespread requirement for reviewers to disclose their use of AI tools and publishers are only starting to react. Springer Nature recently made transparent peer review the default for Nature titles, and its updated policy requires reviewers to declare any AI assistance. Elsevier’s Generative AI policies provide similar guidance, yet it does not enforce specific prompt disclosures and rather focuses on the disclosure of AI and AI-assisted technologies. These steps point in the right direction, but coverage remains patchy and enforcement weak. 

No standards currently exist for preserving prompts or validating generated feedback. And without reform, the peer review system may drift further from its foundational purpose: the collective evaluation of scientific rigor.

Traceability is no longer optional

Human referees have flaws, but they leave a record. Comments, references, email threads and subjective notes all form a record and can be audited. Even flawed reviews can be critiqued, rebutted, or retracted. But AI-generated reviews leave nothing behind. The prompt vanishes, the model’s intermediate steps are opaque, and a review summary lands in the editor’s inbox without provenance. 

In this environment, traceability isn’t a bonus; it’s becomes a critical safeguard against epistemic drift. If scientific legitimacy will rest in part on machine mediation, then three things become overwhelmingly important. Inputs must be disclosed, outputs must be attributed, and reviewers must attest to having evaluated model outputs with appropriate oversight. Without that paper trail, errors and biases can silently distort the scientific record.

But even these necessities are not in and of themselves sufficient. Traceability can document what happened, but it doesn’t prevent how it happened, especially when prompt injection leverages the model’s indiscriminate processing of all input tokens as valid context. To address that and other deeper vulnerabilities, we also need proactive measures: input sanitation, adversarial prompt detection, and model-side awareness of linguistic constructs that bias model interpretation. Without these, errors and biases won’t just slip through, they’ll be engineered in.

Toward transparent and accountable scientific evaluation

Adapting peer review to this new reality means rethinking how evaluation is structured. Prompts, like methods in experimental science, must be recorded. Reviewer training must evolve to include critical awareness of machine-generated text, including its style, patterns, and its blind spots. Editorial workflows should incorporate checks for hidden inputs, misaligned outputs, and anomalies in reasoning that could signal automation without accountability.

The goal is not to banish AI, but to integrate it responsibly. Used well, LLMs can augment expert judgment, reduce cognitive load, and highlight features that humans may overlook. But they are not neutral tools. Their output reflects not just data, but context shaped by prompts, model architecture, and the broader information ecosystem. In some cases, that ecosystem is outdated: static models operate on fixed training corpora and remain unaware of recent research developments. While some systems use retrieval to inject fresh context, peer review workflows may not. When left ungrounded, these models risk reinforcing obsolete perspectives or overlooking recent developments.

That means we must treat machine-generated evaluation as an active force in shaping knowledge, not just a passive tool. As such, it must be subject to the same norms of transparency and reproducibility we demand from any other scientific method.

Discernment at the edge of AI in science

Peer review is messy and subjective, but essential. It is the arena where evidence is weighed, interpretations are challenged, and the boundaries of knowledge are negotiated. If LLMs are to step into that ring, does the burden of scientific stewardship remain with the human referee? Machines can assist, but they cannot be trusted blindly. Judgment is not just about prediction. It is about accountability.

Scientific judgement carries a burden of responsibility that no algorithm can shoulder alone. Trust in the literature will survive only if we treat machine assistance as an active agent that must be documented and checked, not a black-box oracle. Prompt injection is a warning shot. The real test is whether the research community will adapt its norms of transparency and rigor fast enough to keep pace with the tools it now relies on.

This article was created with the assistance of artificial intelligence and thoroughly edited by FirstPrinciples staff and scientific advisors.