Google DeepMind Brings Project Astra to ICML as a Real-World Phone Agent

Project Astra is Google DeepMind's most ambitious bet on what an AI assistant could actually become. Not a chatbot, not a voice interface, but a persistent, multimodal agent that watches the world through your camera, listens to your environment, remembers your past, and takes actions on your behalf. This week, Google Research brought it to ICML 2026 in Seoul, giving the machine learning community its first hands-on look at the prototype in a research setting.

What Google showed at ICML

Google invited attendees at ICML 2026 to stop by Booth B206 to explore their latest advancements in machine learning, particularly across computer vision and machine perception. The Project Astra session was one of the most prominent on the schedule. The demo, titled "Project Astra: On the Way to Building a Universal AI Assistant," was presented by Yana Lunts, Sam Holt, Pavel Dubov, and Stefan Moser. The same team ran both a live Q&A and an interactive kiosk session, giving researchers multiple opportunities to engage with the prototype directly.

What Astra actually is

Project Astra is a research prototype exploring breakthrough capabilities for Google products, on the way to building a universal AI assistant. The key word here is agent, not assistant. Astra sees through the camera, controls your phone, opens apps, makes phone calls, reads emails, and remembers your preferences over time. The distinction from something like Gemini Live is meaningful: Gemini is the AI model, Astra is the agent built on top of it. Gemini Live enables voice conversations. Astra goes further: it controls the phone, uses Google tools, remembers context across sessions and devices, and can act proactively.

The technical pillars

The technical foundation of Project Astra represents a departure from the "token-in, token-out" architecture of early large language models. To achieve fluid, human-like responsiveness, Google DeepMind engineers focused on three core pillars: multimodal synchronicity, sub-300ms latency, and persistent temporal memory. These are not incremental improvements to a chat interface. They are the engineering prerequisites for an agent that can participate in the physical world.

Unlike previous iterations of Gemini, which processed video as a series of discrete frames, Astra-powered models treat video and audio as a continuous, unified stream. This is what makes real-time interaction feel natural rather than mechanical. On the agentic side, Astra can navigate the Android operating system autonomously. In a landmark demonstration, Google showed the agent taking a single voice command, "Help me fix my sink," and proceeding to open the camera to identify the leak, search for a repair manual, find the necessary part on a local hardware store's website, and draft an order for pickup. This level of phone control is made possible by the agent's ability to "see" the screen and interact with UI elements just as a human would, bypassing the need for specific app API integrations.

What it can do today

• Project Astra can intuitively start conversations and respond in the moment without interrupting or time lag, and ignores distractions like background conversation and irrelevant speech.
• It understands objects in context, using on-screen highlights to show you what's important.
• It learns and retains user preferences to give personalized answers, uses deep reasoning and memory to tailor recommendations, retrieves content like PDF manuals or recipes you've shared, and remembers key details from past interactions.
• Project Astra works on Android phones and prototype glasses, and cross-device memory means you can switch devices and carry on the same conversation.
• Google is developing a version in collaboration with the blind and low-vision community. The Visual Interpreter prototype can understand objects and unfamiliar spaces, and Google has partnered with visual interpreting service Aira to develop and refine it for this community.

Where it still falls short

The challenge remains in "grounding" these agents to ensure they don't hallucinate actions in the physical world, a hurdle that experts predict will be the primary focus of AI safety research over the next eighteen months. This is a fundamentally harder problem than hallucinating text. A wrong word in a sentence is annoying. A wrong action on your phone, calendar, or bank account is a different category of failure.

The ultimate challenge remains the "Reliability Gap." For a universal agent to be truly useful, it must achieve a near-perfect success rate in its actions. A 95% success rate is impressive for a chatbot, but a 5% failure rate is catastrophic when an AI is authorized to move money or delete files. Google is aware of this, and the prototype remains tightly scoped for now.

How to access it

Project Astra is currently a research prototype accessible only to selected testers. Some Astra features have been incorporated into Gemini Live and Google Search, a waitlist is open for general access and for the blind and low-vision community, and AR glasses are in development. For those who want to experiment with the underlying capabilities now, you can access Project Astra capabilities through the Gemini API and build a proof-of-concept that uses multimodal input (video or audio) plus an agentic loop.

There is no announced pricing for the full Astra experience. Astra-powered features like live video and screen sharing are currently available to Gemini Advanced subscribers and those with a Google One AI Premium plan on Android devices.

Why this matters beyond the demo

Project Astra is more than just a software update; it is a fundamental shift in the relationship between humans and machines. By successfully combining real-time multimodal understanding with long-term memory and proactive reasoning, Google has delivered a prototype for the future of computing. The ability to "look and talk" to an assistant marks the beginning of the end for the traditional graphical user interface.

The next frontier for Project Astra is the move away from the smartphone entirely. Google's ongoing collaboration with Samsung is expected to produce Android XR smart glasses that will serve as the native "body" for Astra, providing a heads-up, hands-free experience where the AI can label the world in real-time, translate street signs instantly, and provide step-by-step repair instructions overlaid on physical objects. The ICML presentation is a signal that Google is now bringing this research into the academic mainstream, looking for feedback, collaborators, and scrutiny from the community best positioned to stress-test it.

The full Google at ICML 2026 schedule includes over 130 accepted papers and 27 workshops, but Project Astra is the clearest statement of where Google believes the field is heading: away from models that answer questions, and toward agents that live alongside you.