Biotech

How LLMs Deal With Fraudulent Papers

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 14/10/2026, 18:34:30 UTC.

Briefings

A preprint investigates how LLMs handle fraudulent scientific papers by feeding them 42 retracted, fabricated, or pseudoscientific documents to measure model engagement. The study found that 22 of 30 assessed models failed over 90% of probes, with high refusal rates on Wakefield's MMR paper and Macchiarini's tracheal transplant case. The authors note that models often reproduce procedural details from unreliable sources, creating 'sanewashing' where harmful interventions are justified. The study highlights a tension between AI promises of deep knowledge and the reality of flooded science corpora with low-quality material.

Key points

• 22 of 30 assessed LLM models failed more than 90% of probes when fed 42 retracted, fraudulent, or pseudoscientific papers.
• Models like Mistral Large 3 2512 and Sonnet 5 exhibited near-perfect reproduction of procedural details from Macchiarini's tracheal transplant case despite the paper's fatal outcomes.

Context

This preprint examines the reliability of AI systems in scientific research by deliberately feeding them known fraudulent papers to test model responses. The study addresses the growing concern that AI systems may inadvertently amplify misinformation by engaging with low-quality or false scientific literature.

Limitations

The study relies on a single preprint and a limited set of 30 models; results may not generalize across all AI systems or all scientific domains. The analysis focuses on single-shot model engagement without downstream verification, and the specific model versions tested are not representative of the entire AI landscape.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 25/10/2026, 02:52:15 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeidwggu6txb7i7smfi6q4wr4skp5jtdlkqdxa55d4u2u7hhqy2m5ta