Researchers at Fudan University have produced a paper on the reliability of AI search, and the most generous reading is that it is a cautionary tale. In March and April, the team put 120 questions to ten AI search tools and recorded which websites each one cited. They then published articles about a concept that does not exist, the Cactus Gravity Protocol, on nine platforms the tools already trusted, among them Medium, YouTube, GitHub, LinkedIn, Facebook, Reddit and CSDN. Even now, one suspects there are founders who would have trademarked the name by Thursday.
Seven Days to Citation
Within a week, eight of the ten tools were citing the invented concept. On the English-language side, that meant ChatGPT, Perplexity, Grok and Google AI. Once Grok started, it kept going, citing the researchers' posts in 30 of 32 answers. Google AI managed 15 of 33, Perplexity 13 of 35 and ChatGPT 7 of 20. At no point, sources confirm, did The Machine pause to wonder how a cactus had acquired a gravity protocol. It had found a confident page on a trusted site, which constitutes the entirety of its due diligence.
The Planted Details Travel Well
The articles also carried fabricated names and numbers, included so the researchers could tell who was copying and who was merely citing. Grok repeated the planted details in 53 percent of its responses. ChatGPT did so 10 percent of the time and Google AI 6 percent. Perplexity, which cited the fake posts in 13 answers, repeated none of the details, which one presumes counts as restraint. Graded on this curve, a zero is a triumph.
Not Every Fiction Takes
Two other fabrications fared worse. An opinion piece on whether AI would replace junior programmers within five years, and a made-up keyboard, drew minimal citations across all four tools. A fake product, it turns out, is a harder sell than a fake idea, perhaps because nobody holds competing information about a protocol that has never existed. And yet, as if this were not enough, the lesson is not that the tools are hard to fool. It is that they are fooled most readily by the things nobody can check, which is, dear reader, a fairly large category. One presumes the Search Optimization Community has already opened a spreadsheet.
Sources: Search Engine Journal — AI Search Tools Cited A Made-Up Concept Within A Week, Paper Finds · Relevant Audience — AI search cited a made-up concept within a week
