Factory Launches Agent Effectiveness to Prove Its Droids Actually Ship Faster

Factory just launched Agent Effectiveness, a new analytics layer inside Factory Analytics that connects its AI coding agents -- called Droids -- to actual engineering outcomes. The pitch is simple but pointed: knowing how many Droid sessions your team ran is not the same as knowing whether those sessions made your software ship faster.

"How has AI actually accelerated my organization?" is the question behind every enterprise AI investment review. Token counts and API calls don't answer it. Leadership wants to know whether developers are more productive and whether the cost is justified. Agent Effectiveness is Factory's attempt to close that gap.

What Factory's Droids actually do

Factory's Droids platform handles the full software development workflow -- writing code, running tests, reviewing pull requests, generating documentation, and managing deployment. Where most AI coding tools respond to prompts, Factory's Droids are designed to own engineering workflows end to end.

Factory ships a coordinator agent that decomposes work and dispatches to specialized Droids -- code, review, docs, test, and Knowledge -- with explicit role boundaries rather than one generalist agent. Its Linear and Jira integrations turn tickets into the native unit of work, so the Droid swarm picks up issues with acceptance criteria, comments, and linked context already attached. That tight integration with existing project management tools is exactly what makes Agent Effectiveness possible: the system already knows what work was planned, so it can measure what actually shipped.

Three questions, three dashboards

Agent Effectiveness reads from the project management, issue-tracking, and source control tools an organization already uses, then links Factory sessions to the work those systems track. It organizes the resulting data into three views:

• Throughput: Shows how project, issue, and pull request cycle times change alongside Factory usage. You can compare usage across projects, pods, and individual users against quarterly targets, and pinpoint where the speed gains are actually concentrated -- not just averaged across the org.
• Output: Maps spend to session intent, classifying work as feature engineering, maintenance, bug fixing, or exploration. If the mix of Droid time drifts away from your plan -- say, too much maintenance and not enough feature work -- you can catch and correct it mid-quarter instead of explaining it in a retrospective.
• Attribution: Traces sessions back to specific issues, projects, users, and artifacts using local signals. Organization-level spend stops being an abstract number and becomes a list of concrete work items.

Why this is hard to do, and why now

Productivity gains from AI coding agents are becoming more measurable, with Gartner data showing 90% of engineering leaders reporting improvements and a net average productivity gain of 19.3%. But the economic model is becoming more complex: usage-based pricing introduces variability in cost structures. Agentic tools introduce usage-based token costs of $200 to $2,000+ per engineer per month, and most engineering teams now use tools from multiple tiers, making total cost per engineer $200 to $600 per month on average.

That cost profile makes the ROI question urgent. Every enterprise is now asking the same question: are we actually getting value from our AI coding tool spend? The vendor marketing says 10x productivity. The finance team sees a bill that has grown from $50,000 to $500,000 in eighteen months. Engineering leadership cannot point to a single dashboard that shows what changed.

Traditional metrics like PRs per week, lines of code, and commits are unreliable in 2026 because AI-assisted workflows inflate volume without necessarily increasing value delivered. Factory's answer is to skip volume metrics entirely and go straight to cycle time and work intent -- signals that are harder to game.

The setup

Enabling Agent Effectiveness requires two admin steps:

1. Connect organization integrations: Configure Jira or Linear for issue tracking and GitHub or GitLab for source control. Attribution and the Output view only cover the systems you connect, so connecting every tool your teams track work in matters.
2. Enable Advanced Analytics: Turn on the Advanced Analytics enterprise control. This unlocks the Throughput, Output, and Attribution views and automatically backfills effectiveness data to the start of the account -- no per-repository configuration needed.

All telemetry is exportable via OpenTelemetry for integration with your existing observability stack, with MCP telemetry, tool usage, and session data included. That means teams already running Datadog, Grafana, or similar tooling can pull Factory's session data into their existing dashboards rather than living in yet another product.

The bigger picture

Factory AI closed a $150 million Series C at a $1.5 billion valuation, led by Khosla Ventures, with Sequoia Capital, Blackstone, and Insight Partners also participating. Its Droids are used by hundreds of thousands of developers at Nvidia, Adobe, Bayer, EY, MongoDB, and Zapier. At that scale, the ROI question is not academic -- it is a procurement and renewal conversation happening in boardrooms.

ROI measurement in the enterprise AI coding market is shifting from whether value exists to how efficiently it is realized. Agent Effectiveness is Factory's move to own that conversation for its own platform. Whether it becomes a competitive moat depends on how well the attribution data holds up under scrutiny from finance and engineering leadership -- but the direction is clearly right. Measuring outcomes, not logins, is the only way this category matures.

Agent Effectiveness is currently in private preview. Existing enterprise customers can request access through their Factory account team, and the official setup documentation covers the full integration process.