Skip to main content
Back

NC State's Tomcat Agent Reads Vague Human Commands as Well as People

A new benchmark tests whether LLM agents can decode vague human instructions in a shared task, and top models match human performance.

  • New Instruction Inference benchmark tests LLM theory of mind in a collaborative gridworld with vague commands.
  • Tomcat agent uses either few-shot chain-of-thought or commonsense prompting to infer a human's unspoken goal.
  • GPT-4o with Fs-CoT hit 85% intent accuracy and 75% plan optimality, edging out 52 human participants.
  • DeepSeek-R1 Fs-CoT matched human performance despite documentation warning against few-shot prompts.
  • Gemma-3-27B lagged significantly behind humans on all metrics, suggesting a capacity floor for multi-step ToM.
  • Code, prompts, and dataset available at github.com/fardinsaad/Tomcat-LLM.

When a teammate says Can you get that? while gesturing at a table full of tools, you figure out what they mean without a formal spec. Getting an AI agent to do the same requires what researchers call Theory of Mind, the ability to model what another party is thinking. A team from North Carolina State University and TU Delft built a task and an agent to test whether today's large language models can pull it off in a live collaborative setting, and the results land surprisingly close to human performance.

The paper introduces Instruction Inference, a benchmark where an agent has to figure out a human collaborator's unspoken goal from vague commands, along with Tomcat, an LLM-based agent that reasons about that goal step by step. Existing ToM benchmarks mostly rely on narrative or conversational scenarios; Instruction Inference instead evaluates whether an agent can infer a principal's intended goal from clear and unclear instructions and act appropriately in a shared task environment.

The gap between literal and pragmatic

Most instruction-following benchmarks assume the human says exactly what they mean. Real collaboration works differently. People use indirect language, leave out details they assume are obvious, and rely on shared context. Traditional autonomous agents, lacking that context, tend to interpret commands literally and fail when those commands are incomplete or ambiguous.

Diagram showing an LLM agent processing unclear instructions using pragmatic context and theory of mind

The authors ground their design in Grice's cooperative principle from linguistics, which says communication follows maxims about quantity, quality, relation, and manner. When speakers break these maxims, listeners infer what they really meant. Incomplete instructions omit critical information required to determine the intended action, violating the Quantity maxim. Ambiguous instructions yield two or more possible interpretations, violating the Manner maxim. The team focuses on these two categories because they commonly arise in cooperative settings where collaborators assume shared context and task understanding.