Perplexity AI: Unpacking How AI Search & AEO Work

In a recent interview, Jesse Dwyer of Perplexity AI shed light on the evolving landscape of AI search and what SEO professionals should focus on for Answer Engine Optimization (AEO). Dwyer's insights reveal a significant shift from traditional search methods, emphasizing personalization and a groundbreaking approach to content indexing.

AI Search Today: The Rise of Personalization

Dwyer highlighted a crucial shift: personalization is fundamentally transforming search. Unlike classic search, where a query typically yields consistent results, AI search, particularly with tools like Perplexity and ChatGPT, can deliver varied answers based on a user's personal context. Dwyer explained, "I’d have to say the biggest/simplest thing to remember about AEO vs SEO is it’s no longer a zero-sum game. Two people with the same query can get a different answer on commercial search if the AI tool they’re using loads personal memory into the context window (Perplexity, ChatGPT)."

He further clarified, "A lot of this comes down to the technology of the index (why there actually is a difference between GEO and AEO). But yes, it is currently accurate to say (most) traditional SEO best practices still apply.”

The takeaway from Dwyer’s response is that search visibility is no longer about a single, consistent search result. Personal context plays a significant role in AI answers, meaning two users can receive notably different responses to the same query, potentially drawing from different underlying content sources. While the underlying infrastructure still relies on a classic search index, SEO continues to be vital in determining whether content is eligible for retrieval at all. Perplexity AI, for instance, is said to use a form of PageRank, a link-based method for assessing website popularity and relevance, offering a hint at what SEOs should prioritize. However, as Dwyer elaborated, what is retrieved is vastly different from classic search.

When asked to confirm if classic search reliably shows the same ten sites for a given query, while AI search provides a different answer for each user due to its contextual nature, Jesse affirmed, “That’s accurate, yes.”

Sub-document Processing: The Core Difference in AI Search

Jesse continued by detailing the behind-the-scenes processes that generate answers in AI search, particularly focusing on index technology.

“As for the index technology, the biggest difference in AI search right now comes down to whole-document vs. ‘sub-document’ processing. Traditional search engines index at the whole document level. They look at a webpage, score it, and file it. When you use an AI tool built on this architecture (like ChatGPT web search), it essentially performs a classic search, grabs the top 10–50 documents, then asks the LLM to generate a summary. That’s why GPT search gets described as ‘four Bing searches in a trenchcoat’ — the joke is directionally accurate, because the model is generating an output based on standard search results.

This is why we call the optimization strategy for this GEO (Generative Engine Optimization). That whole-document search is essentially still algorithmic search, not AI, since the data in the index is all the normal page scoring we’re used to in SEO. The AI-first approach is known as ‘sub-document processing.’ Instead of indexing whole pages, the engine indexes specific, granular snippets (not to be confused with what SEOs know as ‘featured snippets’). A snippet, in AI parlance, is about 5-7 tokens, or 2-4 words, except the text has been converted into numbers (by the fundamental AI process known as a ‘transformer,’ which is the T in GPT). When you query a sub-document system, it doesn’t retrieve 50 documents; it retrieves about 130,000 tokens of the most relevant snippets (about 26K snippets) to feed the AI.

Those numbers aren’t precise, though. The actual number of snippets always equals a total number of tokens that matches the full capacity of the specific LLM’s context window. (Currently they average about 130K tokens). The goal is to completely fill the AI model’s context window with the most relevant information, because when you saturate that window, you leave the model no room to ‘hallucinate’ or make things up. In other words, it stops being a creative generator and delivers a more accurate answer. This sub-document method is where the industry is moving, and why it is more accurate to be called AEO (Answer Engine Optimization).

Obviously, this description is a bit of an oversimplification. But the personal context that makes each search no longer a universal result for every user is because the LLM can take everything it knows about the searcher and use that to help fill out the full context window, which is a lot more info than a Google user profile.

The competitive differentiation of a company like Perplexity, or any other AI search company that moves to sub-document processing, takes place in the technology between the index and the 26K snippets. With techniques like modulating compute, query reformulation, and proprietary models that run across the index itself, we can get those snippets to be more relevant to the query, which is the biggest lever for getting a better, richer answer. By the way, this is less relevant to SEOs, but this whole concept is also why Perplexity’s search API is so legitimate. For devs building search into any product, the difference is night and day.”

Dwyer contrasts two fundamentally different indexing and retrieval approaches:

• Whole-document indexing, where pages are retrieved and ranked as complete units.
• Sub-document indexing, where meaning is stored and retrieved as granular fragments.

In the first approach, AI sits atop traditional search and summarizes ranked pages. In the second, the AI system retrieves fragments directly and never reasons over full documents. He also emphasized that answer quality is constrained by context-window saturation, meaning accuracy emerges from filling the model’s entire context window with relevant fragments. When retrieval successfully saturates that window, the model has little capacity to invent facts or hallucinate.

Lastly, Dwyer noted that “modulating compute, query reformulation, and proprietary models” are part of Perplexity’s unique approach for retrieving snippets that are highly relevant to the search query, forming their competitive edge in the evolving AI search landscape.