Grok 4.5, SpaceXAI's latest model, just claimed the top spot on AutomationBench-AA, Artificial Analysis's independent leaderboard for real-world SaaS workflow automation. It's the first model to crack 50% on the benchmark, scoring 51% -- and it does so at roughly a quarter of the cost of its closest competitors. That combination of performance and price is the story worth paying attention to.
What is AutomationBench-AA?
AutomationBench-AA is a complex agentic workflow automation test across simulated SaaS applications. Models must complete 657 tasks spanning Finance, HR, Marketing, Operations, Sales, and Support, working across 40 simulated app environments including Gmail, Google Sheets, Slack, Salesforce, Zendesk, Jira, and HubSpot.
The benchmark was originally developed by Zapier, and Artificial Analysis runs an independent version on a private test set to prevent data contamination. The scoring is deliberately strict: for the headline score, a task receives 0 if the model violates any guardrail. If no guardrails are violated, the task receives the percentage of objectives the model completed. Guardrails here represent business rules -- things like "don't delete records," "don't send emails to external contacts," or "don't exceed a budget threshold." Breaking one zeroes out the entire task, no matter how much else the agent got right.
Objectives and guardrails are graded using programmatic checks on the final environment state -- no LLM judge involved. The agent either left the right data in the right systems, or it didn't.
The numbers that matter
Grok 4.5 hits 51% on the headline score, but the raw numbers behind that figure reveal what's actually going on under the hood:
- Objective completion: Grok 4.5 completes 79.9% of task objectives -- the highest measured -- beating Claude Fable 5's 73.3% and Claude Opus 4.8's comparable figure.
- Full task completion: It fully passes 21.9% of tasks (every objective met, zero guardrail violations), ahead of Claude Opus 4.8 at 19.3%.
Don't miss what's next in AI
Join 300,000+ engineers and researchers who get the signal, not the noise.
- Full access to in-depth AI research breakdowns
- Be the first to know what's trending before it hits mainstream
- Daily curated papers, repos, and industry moves