How Much Research Can We Safely Hand Over to AI Agents?
What the European union’s updated AI guidelines mean for researchers, grant writers and reviewers now that AI does more of their work, and who is responsible when it gets something wrong.
Dr Umair Ali Khan & Martti Asikainen | 20.10.2026 | Photo by Adobe Stock Photos
Every research paper and grant proposal opens with a list of names. AI agents can now search the literature, run the analysis and draft the findings, but they never appear on that list. The researcher’s name does, and with it comes responsibility for everything that follows. That raises two questions few researchers can yet answer with confidence. How much of this work can they hand over to AI, and how can they check it was done properly?
The European Commission has now weighed in. It has just published the most extensive update to its guidelines on how researchers should use generative AI (European Commission, 2026a). For the first time, they deal with AI agents, which are the tools that plan and carry out a chain of tasks with little human input. The update also expands the guidance on peer review and grant evaluation (European Commission, 2026b).
Behind both questions sits a more basic one. If an AI agent does part of the research, who answers for what it gets wrong? The Commission’s answer is rather simple. You do. Using an agent never transfers control or responsibility, and you are not exempt from the law because an agent acted on your behalf.
This article looks at what the updated guidelines say and what they mean in practice for researchers and grant writers. It also looks at where the guidelines fall short. They are not binding, they offer little guidance on how to protect those who disclose AI use, and they do not define how much checking is enough.
Set Your Checkpoints Before Runs
Under the hood, an AI agent is a large language model wrapped in software that lets it plan, use tools, remember what it has done and check its own results. This allows it to carry out long, multi-step tasks with little human input. The guidelines warn that researchers may lose control of their own work simply by delegating too many tasks.
In practice, a single instruction can leave many methodological choices to the AI, such as which search terms to use, which studies to include and which to leave out. These choices shape the evidence a summary rests on, and research so far suggests that AI tools do not yet make them reliably.
A systematic review of 19 studies found that generative AI tools such as ChatGPT, used to search the literature, missed between 68% and 96% of the relevant studies that human searchers found. Used to screen studies, they wrongly excluded between 1% and 83% of relevant ones (Clark et al., 2025).
The authors advise against using these tools for literature searching and recommend human oversight for other review tasks, while noting that the technology is developing quickly. The risk is that when the result arrives as a polished summary, the decisions behind it are easy to overlook.
So the guidelines’ answer is to build in checkpoints. Before handing a task to an agent, researchers should decide at which points they will stop it and review its work, especially before critical decisions such as the choice of method. For example, an agent may run a literature search, but a researcher should check which sources it selected before it moves on to analysis.
The same applies to what the agent produces. Any hypothesis it suggests, data it processes or conclusion it draws must be checked by a human before it is used. In the guidelines’ words, agents are advanced research tools, not independent scientists.
Disclose AI When It Shapes the Substance
Most researchers already use AI somewhere. The real question is what to declare. The guidelines draw the line at substantial use. Fixing grammar, polishing language or formatting generally needs no disclosure.
Using AI for a literature review, finding research gaps, forming hypotheses or analysing and interpreting data does. That means researchers should be ready to ask whether the AI shaped their judgement, or could change the results or their meaning. If in doubt, disclose.
Disclosure also goes beyond naming the tool. Researchers should say what it contributed and how they checked it, for example, how they caught invented facts or reduced bias.
Of course, this is easier said than done. Honesty may have a cost. Many writers hesitate to disclose AI use because they worry it will make readers question their writing or language skills. They also worry that reviewers and readers may doubt the quality of the results, and that this doubt can turn into negative bias, leading to a disclosed manuscript or proposal being judged more harshly or even rejected.
The guidelines acknowledge this risk. They cite a 2024 study (Li et al., 2024) suggesting that disclosing AI writing help can lower how readers rate the writing. That creates a quiet incentive to say nothing, which may lead to underreporting. So research organisations and funders are asked to build a climate where people can disclose AI use without being penalised.
Some steps could help. If disclosure were mandatory and used a standard form, it would no longer single anyone out. Evaluators could also see disclosure statements only after scoring. Some publishers already address the concern directly. Springer Nature, for example, tells peer reviewers that declaring AI use does not count against them (Springer Nature, n.d.). But none of these approaches has been tested at scale, and the guidelines do not say how such a climate should be built.
AI Should Support the Review, Not Deliver the Verdict
AI is already part of scientific writing, grant writing and their evaluation, accepted quietly, often reluctantly. So the question is no longer whether AI will be used, but how it should be used responsibly.
Reviewers and grant evaluators may use AI for supporting tasks, such as clarifying a concept, checking background or polishing the language. They may not use it to draft evaluation reports or assess scientific merit. Novelty, significance, whether the evidence supports the claim and whether to accept or reject all stay with humans.
Confidentiality is non-negotiable. Reviewers should not paste unpublished proposals, manuscripts or even key ideas into an unsecured AI tool, not even as excerpts. Tools provided by the institution, funder or publisher are preferred. Pasting a colleague’s unpublished manuscript into a public chatbot is like forwarding their idea to a stranger. The guidelines simply say it out loud.
Rules alone will not settle this. Peer reviewers are usually unpaid, and lack of time is the main reason they decline to review (Tite & Schroter, 2007). Researchers also report practical barriers to using AI responsibly. For example, in a survey at two German research organisations, legal uncertainty and privacy concerns emerged as major barriers to AI use (Chugunova et al., 2026).
Some publishers acknowledge the gap. Springer Nature, for example, prohibits uploading manuscripts to public AI tools but allows limited use of secure, institutionally approved ones, depending on their data policies (Springer Nature, n.d.).
Where reviewers have no secure alternative, a public chatbot remains the most convenient option. That is why the preference for institutional tools matters. A ban without a practical, secure alternative leaves individual reviewers to weigh convenience against confidentiality on their own.
In AI-assisted writing, accuracy is the other risk. AI can invent plausible claims with citations. Even when a cited paper is real, the cited claim could be wrong. So every cited claim and reference must be checked.
Submissions May Carry Hidden Prompts
Some research papers or grant proposals may contain hidden prompts that a human won’t notice, but an AI tool used to review these documents will still read them. They can hide in invisible text, formatting tricks or metadata. These hidden prompts can instruct the AI tools to give only positive reviews.
The instruction is usually set in white text, sometimes in an extremely small font (Gibney, 2025). A human reading the PDF may scroll straight past it, while an AI tool asked to summarise or assess the document may process the hidden text as an instruction and follow it. Experimental research has since shown that the technique works. In a simulated medical peer review, hidden text pushed three commercial AI models to give higher scores and raised their rate of acceptance recommendations from 0% to nearly 100% (Choi et al., 2026).
The guidelines, therefore, ask funders to assume that incoming proposals may carry such instructions, set rules that prohibit them and build safeguards into the IT systems that process applications.
That means once AI reads documents, every document becomes a possible way to steer it. Anyone building AI pipelines for documents should therefore treat each incoming file
Evaluators May Lose Their Compass
AI has made writing a grant proposal faster and easier. As a result, funders receive more applications, and evaluators have more to read. More applications also mean more pressure on evaluators, and with it, more temptation to let AI do the reading.
However, AI can make a proposal read better without making the idea behind it any better, more realistic or achievable. AI can polish the text, but the originality, ambition and value of the idea still come from the researchers. So when every proposal reads well, good writing no longer tells evaluators which idea is strongest.
In other words, the old way of judging proposals is losing its signal. The guidelines warn that small fixes, such as new rules on AI use, will always stay one or two steps behind the technology. Instead, they say funding processes probably need to be redesigned so they still work when AI is everywhere.
For years, a clear, well-structured proposal was a reasonable, if imperfect, sign that applicants had thought their projects through. That signal is now weaker. Evaluators will need to look past the quality of the prose to what good writing alone cannot show: whether the plan is feasible, whether the team has the capacity to deliver it and whether the proposed contribution is genuinely new.
The guidelines do not prescribe a single model. Rather, they encourage funders to experiment, for example with ideas from CoARA’s catalogue of assessment experiments (CoARA, n.d.). Some funders have already tried alternatives, such as interviewing applicants, using short first-stage proposals or drawing lots among proposals judged equally strong.
However, any experiment must stay transparent and protect applicants’ rights to fair treatment, confidentiality and appeal. In the meantime, applicants are asked to declare any substantial AI use in their proposals. Disclosure will not show which idea is strongest, but it gives evaluators an honest picture of how each proposal was made.
AI Can’t Take the Responsibility
Much of the debate about AI in writing focuses on detecting AI-written text. The guidelines look somewhere more useful. They ask who makes the decision at each step of the research, and whether a human still checks it.
At the heart of the updated guidelines is the distance between the researcher and the work. A chatbot that polishes a paragraph sits close to the author. An agent that searches the literature, runs the analysis and drafts the findings sits much further away, while the researcher’s name stays on the result.
That distance is the real risk, because it feeds what the guidelines call automation bias. The more capable the tool, the easier it becomes to trust it without checking. The biggest danger is approving outputs you no longer fully examine or understand.
The guidelines acknowledge that AI could become a real partner in discovery, citing mathematical problems already solved with or by AI. Rejecting it would waste that potential. Using it uncritically would put trust in science at risk.
Managing that distance will not be easy. The guidelines are non-binding, rules differ across journals and funders, honest disclosure can still be penalised, and the tools change faster than any document.
Even so, the practical lessons are already clear:
- Researchers should decide where to check an agent’s work before it runs, keep a record of what the AI did and disclose any use that shaped the results.
- Institutions and funders should provide secure AI tools, so confidential work stays protected, and reward honest disclosure instead of punishing it.
What the guidelines cannot define is how much checking is enough. That judgement stays with each researcher, and that is the point.
Authors
Dr Umair Ali Khan
Senior Researcher, Consultant
Finnish AI Region
umairali.khan@haaga-helia.fi
Martti Asikainen
Communications Lead
Finnish AI Region
martti.asikainen@haaga-helia.fi
References
Choi, B., Jun, T. J., Sung, J. W., Park, I. W., Lee, J.-M., Cho, S. I., Park, H. J., Lee, R. W., & Suh, J. (2026). Invisible text injection and peer review by AI models. JAMA Network Open, 9(1), e2552099. https://doi.org/10.1001/jamanetworkopen.2025.52099
Chugunova, M., Harhoff, D., Hölzle, K., Kaschub, V., Malagimani, S., Morgalla, U., & Rose, R. (2026). Who uses AI in research, and for what? Large-scale survey evidence from Germany. Research Policy, 55(2), 105381. https://doi.org/10.1016/j.respol.2025.105381
Clark, J., Barton, B., Albarqouni, L., Byambasuren, O., Jowsey, T., Keogh, J., Liang, T., Moro, C., O’Neill, H., & Jones, M. (2025). Generative artificial intelligence use in evidence synthesis: A systematic review. Research Synthesis Methods, 16(4), 601–619. https://doi.org/10.1017/rsm.2025.16
CoARA. (n.d.). Catalogue of research assessment experiments. Retrieved October 19, 2026, from https://experiments.coara.org/
European Commission. (2026a, October 16). ERA living guidelines on the responsible use of generative AI in research updated. Directorate-General for Research and Innovation. https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-news/era-living-guidelines-responsible-use-generative-ai-research-updated-2026-10-16_en
European Commission. (2026b). Living guidelines on the responsible use of generative AI in research. Directorate-General for Research and Innovation. https://research-and-innovation.ec.europa.eu/document/download/2b6cf7e5-36ac-41cb-aab5-0d32050143dc_en?filename=ec_rtd_ai-guidelines.pdf
Gibney, E. (2025, July 24). Scientists hide messages in papers to game AI peer review. Nature. https://doi.org/10.1038/d41586-025-02172-y
Li, Z., Liang, C., Peng, J., & Yin, M. (2024). How does the disclosure of AI assistance affect the perceptions of writing? In Y. Al-Onaizan, M. Bansal, & Y.-N. Chen (Eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (pp. 4849–4868). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.emnlp-main.279
Springer Nature. (n.d.). Peer review policy, process and guidance. Retrieved October 19, 2026, from https://www.springer.com/cn/editorial-policies/peer-review-policy-process
The Decoder. (2025, July 6). Researchers hide prompts in scientific papers to sway AI-powered peer review. https://the-decoder.com/researchers-hide-prompts-in-scientific-papers-to-sway-ai-powered-peer-review/
Tite, L., & Schroter, S. (2007). Why do peer reviewers decline to review? A survey. Journal of Epidemiology & Community Health, 61(1), 9–12.
