Google Research· Research

ConvApparel: Measuring and bridging the realism gap in user simulators

Ofer Meshi and Sally Goldman, Research Scientists, Google Research We introduce ConvApparel, a new human-AI conversation dataset and a comprehensive evaluation framework designed to quantify the "realism gap" in LLM-based user simulators and improve the training of robust conversational agents. Quick links Paper Share Copy link × Modern conversational AI agents can typically handle complex, multi-turn tasks like asking clarifying questions and proactively assisting users. However, they frequently struggle with long interactions, often forgetting constraints or generating irrelevant responses. Improving these systems requires continuous training and feedback, but relying on the "gold standard" of live human testing is prohibitively expensive, time-consuming, and notoriously difficult to scale. As a scalable alternative, the AI research community has increasingly turned to user simulators — LLM-powered agents explicitly instructed to roleplay as human users. However, modern LLM-based simulators can still suffer from a significant r ealism gap …

Announced

09 Apr 2026 11:22 UTC