Google's Vice President of Product for Search, Robby Stein, recently offered critical insights into how the company's evolving AI Mode assesses content quality and helpfulness. During an interview, Stein clarified that Google's AI search experience is built upon decades of established search quality signals, rather than being a complete overhaul. He addressed concerns about AI hallucinations and, crucially for SEO professionals, outlined five key factors that determine content's visibility and success within the new AI-powered search environment.
Combating AI Hallucinations in Google Search
Addressing a common concern about AI — its tendency to "hallucinate" or generate incorrect information — Stein emphasized that Google's AI Mode isn't starting from scratch. Instead, its quality systems are deeply rooted in the 25 years of experience gained from classic Google Search. The mechanisms that determine link relevance and content quality are directly encoded into the AI model, leveraging Google's extensive history of understanding what users find valuable and trustworthy.
"These models are non-deterministic and they hallucinate occasionally… how do you protect against that? How do you make sure the core experience of searching on Google remains consistent and high quality?" the interviewer asked.
Stein responded, "Yeah, I mean, the good news is this is not new. While AI and generative AI in this way is frontier, thinking about quality systems for information is something that’s been happening for 20, 25 years."
He continued, "And so all of these AI systems are built on top of those. There’s an incredibly rigorous approach to understanding, for a given question, is this good information? Are these the right links? Are these the right things that a user would value?"
Stein further explained that the model uses Google Search as a tool, building on its historical understanding of helpful resources. "What are the things that people who are doing that have been relying on on Google for all these years? We kind of know what those resources are we can show you right there. And so I think that helps a lot."
He also noted improvements in model instruction following: "the models over time have also become just better at instruction following as well. And so you can actually just define, hey, here are my primitives, here are my design guidelines. Don’t do this, do this." While acknowledging that mistakes can still occur, Stein expressed confidence in the models' improved quality, making such errors "much less likely to happen now."
Stein's explanation underscores that Google's AI Mode is not a radical departure but an evolution, deeply integrated with the established relevance, trust, and usefulness signals of classic search. This continuity is crucial for managing the risk of hallucinations, ensuring AI answers are grounded in reliable information sources that users have historically valued.
Google's Approach to Evaluating Helpfulness in AI Mode
The discussion then shifted to how Google evaluates helpfulness within its dynamic AI Mode. Stein clarified that the core principles for determining quality remain consistent with classic search, even as the interface evolves.
"And Robbie, as search is evolving, as the results are changing and really, again, becoming dynamic, what signals are you looking at to know that the user is not only getting what they want, but that is the best experience possible for their search?" the interviewer inquired.
Stein detailed a "whole battery of things" Google uses to study helpfulness:
"We look at, like we really study helpfulness and if people find information helpful. And you do that through evaluating the content kind of offline with real people. You do that online by looking at the actual responses themselves. And are people giving us thumbs up and thumbs downs? Are they appreciating the information that’s coming? And then you kind of like, you know, are they using it more? Are they coming back? Are they voting with their feet because it’s valuable to you."
He stressed the importance of triangulating these signals, as relying on any single metric can be misleading. Stein highlighted a specific metric unique to search: repeated attempts for the same query.
"We have a very specific metric that manages people trying to use it again and again for the same thing. We know that’s a bad thing because it means that they can’t find it. You got to be really careful."
This multi-faceted approach to evaluating helpfulness in AI Mode combines human evaluation, explicit user feedback (like thumbs up/down), and long-term behavioral patterns. Crucially, Stein pointed out that increased usage alone isn't always a positive signal; repeated searches for the same information indicate a failure to satisfy the user. This reinforces Google's commitment to user satisfaction, detecting friction and confusion as much as positive engagement, maintaining continuity with its classic search quality philosophy.
Five Essential SEO Factors for AI Search
Finally, Stein addressed the critical question for SEO professionals: Do traditional SEO best practices still apply to content ranking in Google's AI Mode? His answer was a resounding yes, emphasizing that the underlying quality signals remain consistent. He then outlined five key factors Google uses to determine if a website meets its quality and helpfulness standards in the AI-powered search environment.
Stein explained the AI model's core mechanic: "The model takes your question and reasons about it, tries to understand what you’re trying to get out of this. It then generates a fan-out of potentially dozens of queries that are being Googled under the hood. That’s approximating what information people have found helpful for those questions."
He reiterated the strong link to historical quality work: "There’s a very strong association to the quality work we’ve done over 25 years. Is this piece of content about this topic? Has someone found it helpful for the given question?"
Stein then succinctly stated, "The short of it is the same things apply." He listed the five crucial factors:
- Is your content directly answering the user’s question?
- Is it high quality?
- Does it load quickly?
- Is it original?
- Does it cite sources?
He concluded, "If people click on it, value it, and come back to it, that content will rank for a given question and it will rank in the AI world as well."
These factors highlight that fundamental SEO principles — focusing on user intent, delivering high-quality, original, fast-loading, and well-sourced content — remain paramount for achieving visibility and ranking, not just in traditional search but also within Google's evolving AI search experience.
Watch the Interview:
For the full context, you can watch the interview with Robby Stein, starting around the 1 hour and 23-minute mark:
