Google Research· Research

Accelerating Gemini Nano models on Pixel with frozen Multi-Token Prediction

Eden Cohen, Research Product Manager, and Michelle Ramanovich, Research Manager, Google Platforms and Devices We introduce a method to retrofit Multi-Token Prediction onto frozen production models, accelerating on-device inference without the inefficiencies of separate drafters. Quick links Keyword blog Share Copy link × Having powerful Large Language Models (LLMs) right in your pocket is now a reality with on-device models like Gemini Nano and Gemma . This technology enables everyday features on your phone — such as instantly summarizing a flurry of notifications or proofreading an important text message — all without sending your private data off device. But to make these features useful for everyday users, they need to happen very efficiently. Delivering this kind of speed on a mobile device is a significant challenge. Unlike vast server environments, mobile phones operate under a strict energy budget and hard memory (RAM) limits. Furthermore, standard language models generate text "autoregressively" — meaning they process and output just one word (or token) at a time. …

Announced

26 Jun 2026 18:30 UTC