Concepts covered by different strategies for the query “What is an example of inequality?” “. The x-axis shows topics discovered in search engine results (Section 5), the y-axis shows search engines. Blue boxes indicate that the topic appeared in the output of this engine. Search engines differ in the concepts they evoke. Credit: arXiv (2025). DOI: 10.48550/arxiv.2510.11560
As anyone who uses the Internet knows, the way we find information has fundamentally changed. For the past three decades, search engines have provided ranked lists of links in response to our queries, and it was our job to sift through them to find what we wanted. Now, major search engines use generative AI tools to provide a single, consistent answer, often embedded within a few links. But how does this approach compare to the traditional method? In a comprehensive new study, scientists compared these two approaches to see what we gain and lose.
Comparing AI with traditional searches
To understand this, researchers from Ruhr University Bochum and the Max Planck Institute for Software Systems compared traditional Google Search with four generative search engines: Google AI Overview (AIO), Gemini, GPT-4o-Search and GPT-4o with Search Tool. The team ran thousands of queries covering six main domains, including general knowledge, politics, science and shopping.
Next, they performed a detailed comparison of the two research styles based on three key indicators. First, they analyzed the diversity of sources by comparing the websites used by the AI to the main traditional search links. Second, they measured knowledge dependence to see how much the AI relied on its own internal memory rather than searching for new information on the web.
They then examined conceptual coverage to determine whether each AI’s final responses covered a broader range of ideas than traditional main search results. Finally, they ran the experiment again two months later to see how quickly the AI’s sources and responses changed over time.
Clear differences appear
As the team pointed out in their article published on the arXiv preprint server, there are obvious differences between the two. AI cast a wider net by pulling information from a more diverse set of websites than traditional searches, but the links it used often weren’t among the top search results. However, just because the AI uses more sources does not mean its answers are more complete. In fact, this was often not the case.
“Differences in source selection and internal knowledge use can subtly change the perspectives and facts users are exposed to, even when the overall coverage of the topic appears similar,” the study authors wrote.
AI responses also tend to be less stable over time, changing significantly after two months.
The researchers also discovered differences between the AI models. For example, GPT-4o with Search Tool relied on its internal memory, while Google AI Overview and Gemini constantly pulled information from a larger, newer set of external sites.
Overall, the team couldn’t determine whether one type of search was better than the other, although there is a clear trade-off when using AI. Users receive a diverse, summarized answer, but lose the reliable sources and consistent results they get from traditional searches. In particular, the authors highlighted the need for new criteria and standards to properly evaluate the answers returned by the AI.
Written for you by our author Paul Arnold, edited by Gaby Clark, and fact-checked and revised by Robert Egan, this article is the result of painstaking human work. We rely on readers like you to keep independent science journalism alive. If this reporting interests you, consider making a donation (especially monthly). You will get a without advertising account as a thank you.
More information:
Elisabeth Kirsten et al, Characterizing web search in the era of generative AI, arXiv (2025). DOI: 10.48550/arxiv.2510.11560
arXiv
© 2025 Science X Network
Quote: The Great Divide in Search: How AI Searches and Traditional Web Searches Differ (October 29, 2025) retrieved October 29, 2025 from
This document is subject to copyright. Except for fair use for private study or research purposes, no part may be reproduced without written permission. The content is provided for informational purposes only.
