Skip to main content
Back

RheinMain's Two-Agent System Lets Claude-Opus 4.6 Score 85.96 on Factory AI

A two-agent architecture for interrogating value stream map simulations hits 86% accuracy by keeping raw data out of the planner's context window.

Paper, Full Access
  • New two-agent architecture separates planning from data analysis to answer questions over value stream map simulations.
  • Orchestrator never sees raw data; a summarization subworkflow processes each data element in an isolated, fresh context.
  • Four tools enforce progressive discovery: node listing, attribute extraction, taxonomy navigation, and delegated summarization.
  • Claude-Opus 4.6 hits 85.96/100 with 0.86 standard deviation; open-weights GLM-5 reaches 83.88 using the fewest tokens.
  • Roughly 15% of questions defeat every model, with failures traced to navigation errors from semantically similar metric descriptions.
  • Implemented in DSPy; evaluated with a GPT-4.1 judge validated against two human VSM domain experts.

Value stream maps are how factories visualize the flow of materials, information, and work from raw input to finished product. Once you turn one into a discrete event simulation, it becomes a firehose of numbers: hundreds of pre-calculated KPIs, multi-variate time series, and structural parameters spread across a JSON payload that can span megabytes. Asking a language model something as apparently simple as Are any supermarkets undersupplied? is deceptively hard, because the model has to figure out which of dozens of near-identical inventory metrics is actually the right one to consult.

A team from RheinMain University and SimPlan AG has proposed an agentic architecture that tackles this by splitting the work between two agents that never share raw data. On their benchmark, Claude-Opus 4.6 reaches 85.96/100 and the open-weights GLM-5 trails at 83.88, with the top model producing a standard deviation of just 0.86 across four runs.

Why code-writing agents fall over here

The prevailing paradigm for LLM agents in operations research is code-generating agents that write Python to query raw data and compute metrics. That works when the numbers you need have to be derived from clean tabular sources. In VSM simulations, the numbers already exist, sometimes in eleven different flavors, and the actual challenge is picking the right one.

The authors frame this as a multi-hop retrieval problem over two intertwined data structures: the static graph describing the factory (nodes for warehouses, processes, customers, edges for material and information flow) and the dynamic output logs from a discrete event simulation that can cover hours to weeks of operation. A single VSM contains 10 to 30 nodes, though highly complex ones scale past 100, with hundreds of data elements organized into 11 high-level categories.