xAI's Grok 4.5 Tops SaaS Automation Benchmark at a Quarter the Cost

Grok 4.5, SpaceXAI's latest model, just claimed the top spot on AutomationBench-AA, Artificial Analysis's independent leaderboard for real-world SaaS workflow automation. It's the first model to crack 50% on the benchmark, scoring 51% -- and it does so at roughly a quarter of the cost of its closest competitors. That combination of performance and price is the story worth paying attention to.

What is AutomationBench-AA?

AutomationBench-AA is a complex agentic workflow automation test across simulated SaaS applications. Models must complete 657 tasks spanning Finance, HR, Marketing, Operations, Sales, and Support, working across 40 simulated app environments including Gmail, Google Sheets, Slack, Salesforce, Zendesk, Jira, and HubSpot.

The benchmark was originally developed by Zapier, and Artificial Analysis runs an independent version on a private test set to prevent data contamination. The scoring is deliberately strict: for the headline score, a task receives 0 if the model violates any guardrail. If no guardrails are violated, the task receives the percentage of objectives the model completed. Guardrails here represent business rules -- things like "don't delete records," "don't send emails to external contacts," or "don't exceed a budget threshold." Breaking one zeroes out the entire task, no matter how much else the agent got right.

Objectives and guardrails are graded using programmatic checks on the final environment state -- no LLM judge involved. The agent either left the right data in the right systems, or it didn't.

The numbers that matter

Grok 4.5 hits 51% on the headline score, but the raw numbers behind that figure reveal what's actually going on under the hood:

• Objective completion: Grok 4.5 completes 79.9% of task objectives -- the highest measured -- beating Claude Fable 5's 73.3% and Claude Opus 4.8's comparable figure.
• Full task completion: It fully passes 21.9% of tasks (every objective met, zero guardrail violations), ahead of Claude Opus 4.8 at 19.3%.
• Cost per task: $0.34, compared to Claude Fable 5 at $1.35, Claude Opus 4.8 at $1.46, GPT-5.5 xhigh at $1.28, and Gemini 3.5 Flash at $0.49.
• Token efficiency: ~8k output tokens per task -- less than a quarter of Claude Opus 4.8's 32k, and a third of Gemini 3.5 Flash's 24k.
• Parallel tool use: Grok 4.5 batches 3.3 tool calls per turn on average, versus ~2.5 for Claude Opus 4.8 and ~2.0 for GPT-5.5 xhigh.
• Finance domain: 71% objective completion in Finance -- the hardest domain -- ahead of Claude Fable 5 (64%) and Claude Opus 4.8 (62%).

The cost advantage isn't just about token pricing. Grok 4.5 is served at fast-model speeds of 80 TPS, and combined with twice greater token efficiency than the latest leading models at the same tasks, the model delivers intelligent results more quickly and at far lower costs.

How it works: the efficiency engine

The low token count is not a coincidence -- it reflects a deliberate training approach. Grok 4.5 was trained across tens of thousands of NVIDIA GB300 GPUs. SpaceXAI scaled reinforcement learning with a strong focus on per-token intelligence, covering hundreds of thousands of tasks centered on multi-step software engineering and other technical work, with automated and model-based grading.

The stack is built for highly asynchronous training, so agentic rollouts can run for many hours while learning continues across tens of thousands of GPUs. The result is more intelligent and efficient reasoning on real engineering and agentic tasks. In practice, this means the model has learned to batch tool calls and resolve tasks in fewer turns rather than grinding through long chains of sequential actions.

Grok 4.5 resolves tasks in ~16 turns, fewer than GPT-5.5 xhigh (25 turns) and less than half of Gemini 3.5 Flash at 35 turns -- while making the most tool calls per task of any leading model (52.5 total). It's doing more work per turn, not more turns overall.

Here's what a minimal API call to Grok 4.5 looks like:

curl -s https://api.x.ai/v1/responses \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "grok-4.5",
    "input": "Find and fix the bug, then explain it: function median(a){a.sort();return a[a.length/2]}"
  }'

Where it still falls short

The guardrail picture is more complicated. Grok 4.5 triggers 0.63 violations per task -- higher than Claude Opus 4.8 (0.55) and Gemini 3.5 Flash (0.46). At 13.0 objectives completed per violation, it also trails Gemini 3.5 Flash (15.0) and Claude Opus 4.8 (13.5). It is completing more objectives overall, but it's also breaking more rules to do it.

SpaceXAI says the model has "twice greater token efficiency" than other leading models. If it carries through to real-world use cases, that efficiency would be a big advantage, since the cost of tokens has been a growing concern for AI consumers. But the guardrail violation rate is a real caveat for production deployments where breaking a business rule has actual consequences.

Finance workflows are the hardest to automate: across all models, agents complete around one third of Finance objectives, roughly half the rate of Support and Operations (~60%). Grok 4.5 leads in Finance, but even its 71% objective completion rate in that domain still means nearly a third of Finance objectives go unmet.

What's available and what it costs

Grok 4.5 is available in Grok Build, in Cursor on all plans, and from the SpaceXAI console, though Grok 4.5 is not yet available in the EU. Grok 4.5 is priced at $2 per million input tokens and $6 per million output tokens. Claude's Opus 4.8, by comparison, is priced at $5 per million input tokens and $25 per million output tokens.

Grok 4.5 is SpaceXAI's smartest model built for coding, agentic tasks, and knowledge work. It was trained alongside Cursor. It ranks #1 on Harvey's Legal Agent Benchmark and is the default model in Grok Build. Beyond coding, Grok Build is capable of building complex Excel models that involve research from the web, multi-sheet formula use, and even leaves stickies or notes behind for future reference. In PowerPoint and Word, Grok 4.5 is similarly meticulous, capable of using native PowerPoint shapes to build complex diagrams, designing intuitive slide content, and writing clear prose in Word.

The bigger picture

The 51% score is a milestone, but it also underscores how far agentic AI still has to go. Even the best model today fully completes fewer than 1 in 4 tasks without breaking a single business rule. SpaceXAI has released Grok 4.5 as the first major model since the company went public. SpaceXAI characterized its new release as a workhorse that can tackle all of the typical tasks that the AI industry has sought to automate: coding and app-building, office and clerical work, research, writing, and other forms of routine knowledge work.

This model was trained using the same compute capacity SpaceXAI is leasing to its competitors Anthropic and Google. As its own compute needs grow, SpaceXAI may have to choose between using capacity for its own models or leasing it to others as a revenue stream. That tension is worth watching as the company scales.

For teams building agentic pipelines on top of SaaS tools, Grok 4.5 is now the most credible option to evaluate first -- not because it's flawless, but because it delivers the best results per dollar by a significant margin. The guardrail violations mean you'll still want human review or hard programmatic checks on sensitive operations, but the cost curve makes it compelling for high-volume automation work.