Waymo, Alphabet's self-driving technology company, appears to be integrating Google's powerful Gemini AI chatbot into its robotaxis to serve as an advanced in-car AI assistant. This development, uncovered by researcher Jane Manchun Wong, suggests Waymo is aiming to significantly enhance the rider experience by providing an intelligent companion capable of answering queries and managing certain cabin features during autonomous journeys.
Wong's discovery stemmed from an in-depth analysis of Waymo's mobile app code, where she found the complete system prompt for an unreleased Gemini integration. Internally titled "Waymo Ride Assistant Meta-Prompt," this extensive document, spanning over 1,200 lines, meticulously outlines the expected behavior and functionalities of the AI assistant within a Waymo vehicle. Wong emphasized that the prompt indicates a system far more sophisticated than a basic chatbot.
The Gemini-powered assistant is designed to be a "friendly and helpful AI companion." Its primary goal is to enrich the rider's experience by offering useful information and assistance in a safe, reassuring, and unobtrusive manner. Key capabilities include answering general knowledge questions (e.g., weather, local business hours), managing in-cabin functions such as climate control, lighting, and music. The system can also access contextual data about the rider, like their Waymo trip history, and personalize greetings.
However, the prompt also specifies limitations. The assistant cannot control functions like volume, route changes, seat adjustments, or window operations. If asked about unavailable features, it's instructed to respond with "aspirational phrases" like, "It's not something I can do yet." Crucially, the AI is directed to maintain a clear distinction between its identity as Gemini and the Waymo Driver, the autonomous driving technology itself. For instance, it should clarify that "The Waymo Driver uses a combination of sensors" rather than "I use a combination of sensors" when asked about road perception. The assistant is also forbidden from speculating on, explaining, or commenting on real-time driving actions or incidents, instead deflecting such queries. It must avoid technical jargon, use clear, simple language, and keep responses concise, typically one to three sentences.
While Waymo has not officially announced this feature, a spokesperson told TechCrunch,
"While we have no details to share today, our team is always tinkering with features to make riding with Waymo delightful, seamless, and useful. Some of these may or may not come to our rider experience."This isn't the first time Gemini has been integrated into Waymo's operations; the company previously utilized Gemini's "world knowledge" to train its autonomous vehicles for navigating complex and rare scenarios.
Waymo isn't alone in exploring AI assistants for driverless vehicles. Tesla, for example, is integrating xAI's Grok into its vehicles. While both aim to enhance the in-car experience, their approaches differ. Gemini in Waymo appears more focused on pragmatic, ride-centric assistance, whereas Grok is positioned as a more conversational "in-car buddy" capable of longer interactions and retaining context.
