Google Search Advocate John Mueller has unequivocally dismissed the emerging idea of serving Markdown files directly to AI crawlers, labeling it “a stupid idea.” This strong pushback comes as some developers explore this method to reduce token usage for large language model (LLM) bots, sparking a significant debate within the technical SEO community.

The Markdown-for-Bots Proposal

The concept involves using middleware, such as with Next.js, to detect AI user agents like GPTBot and ClaudeBot. When these bots access a page, the middleware would intercept the request and deliver a raw Markdown file instead of the full HTML or React payload. Proponents claim early benchmarks show a remarkable 95% reduction in token usage per page, which they argue could boost a site's ingestion capacity for retrieval-augmented generation (RAG) bots.

Mueller's Strong Opposition

Mueller, however, raised immediate technical concerns on Reddit. He questioned whether LLM crawlers could even recognize Markdown as anything other than a plain text file, asking:

“Are you sure they can even recognize MD on a website as anything other than a text file? Can they parse & follow the links? What will happen to your site’s internal linking, header, footer, sidebar, navigation? It’s one thing to give it a MD file manually, it seems very different to serve it a text file when they’re looking for a HTML page.”

His comments on Bluesky were even more direct. Responding to technical SEO consultant Jono Alderson, who highlighted how flattening pages into Markdown strips away meaning and structure, Mueller retorted:

“Converting pages to markdown is such a stupid idea. Did you know LLMs can read images? WHY NOT TURN YOUR WHOLE SITE INTO AN IMAGE?”

Alderson himself emphasized that collapsing a page into Markdown sacrifices crucial context and structure, viewing it as a short-term convenience rather than a sustainable strategy.

Broader Concerns and Unverified Claims

Other experts in the Reddit discussion echoed these concerns. One commenter questioned if such an effort might inadvertently limit crawling rather than enhance it, noting a lack of evidence that LLMs are trained to prioritize less resource-intensive documents. While the original developer defended the theory, suggesting LLMs are better at parsing Markdown due to their heavy training on code repositories, this claim remains unverified.

Mueller's Consistent Stance on AI Content

This isn't the first time Mueller has addressed this topic. He previously responded to a query from Lily Ray regarding separate Markdown or JSON pages for LLMs, maintaining the same position: focus on clean HTML and structured data instead of creating bot-only content copies. This advice aligns with SE Ranking's analysis of 300,000 domains, which found no correlation between having an llms.txt file and a domain's citation frequency in LLM answers. Furthermore, Mueller has likened llms.txt to the outdated keywords meta tag, a format major platforms do not officially use for ranking or citations.

Looking Ahead: Adhering to Best Practices

As of now, public platform documentation offers no indication that bot-only formats, such as Markdown versions of web pages, improve search ranking or AI citations. Mueller's consistent objections across multiple discussions, coupled with SE Ranking's data, reinforce this view. Until an AI platform officially publishes specifications requesting Markdown versions of web pages, the established best practices remain: maintain clean HTML, minimize unnecessary JavaScript that obstructs content parsing, and leverage structured data where platforms have documented schemas.