How to Add a Decision Model to Your Agentic Coding Workflow

Decision models, which pick one answer from a list you write and score every option, went from one product to a crowded field in under a month.

After TypeSafe launched Jev in September, OpenAI put a Decisions API into limited preview on September 29 (public beta since October 6), and Cloudflare and Perplexity released their own within days.

We even got Ollaya, which is Ollama but for open decision models!

In my opinion, the name oversells the idea, since these models do what classifiers have always done: return a label with a probability. The useful part is where they sit, at the points in an AI workflow where the system has to pick from a list instead of writing text.

I wanted to see one of those points up close, so I put one inside a Claude Code loop, right after the tests pass. This is part 1 of a two-part series: it covers where the call goes and how to wire it in, and part 2 covers how to test a decision model before you trust it.

For my experiment, an AI coding agent built a small document API and seeded 12 bugs into it. Claude Code then fixed each one, and all 12 fixes passed the 9 required tests, the ones that run on every change.

The rename fix still sent back the wrong document title in the API response. If a client renamed a document to " Renamed " (with leading and trailing spaces), the backend trimmed it and correctly stored "Renamed" in the database.

But the API's JSON response echoed back the raw, padded string, and since none of the 9 required tests sent a padded title, the build stayed green. So which one extra test should run after a green build: an edge-case suite, an integration suite, or an access-control suite?

That's exactly the closed question these Jev-like decision models answer: you list the options, and they pick one, with a probability for each.

I would call the decision model after the required tests pass, as one closed question that can only add a check. Then I'd keep the model in shadow mode, recording its predictions to a log file without actually triggering any test runs, until it beats a static rule you would have otherwise applied.

In the experiment, Jev and winnow:e4b, a local decision model, picked the expected extra check on every fix. Claude, asked the same question in a plain message, picked it on 9 of 12, and a simple filename-based rule on only 3.

All four still caught the rename bug, the one bug that survived a fix, and part 2 shows how I tested them and where the expected checks came from.

What follows:

• The three points where I'd pause a Claude Code loop for a decision
• My test of the middle pause on 12 Claude Code fixes, and next_check.py, the 203-line wrapper that reproduces it
• The five kinds of decision models, and which one fits your setup

Also, I built a full Decision Model Field Guide with a model chooser, code for four providers, and all 144 answers from my test. And the project's code is live on my GitHub, from the 203-line wrapper to the buggy API I tested it on.

What Jev and other decision models do

If you skipped our previous article by Ben Dickson, Jev takes three things: the current state (any text or JSON), a question, and the allowed answers, each with a one-line meaning.

Then it returns one of those answers, never one outside your list, plus a probability for every option and a confidence score (for the Choice and Score primitives). TypeSafe charges $0.042 per million input tokens and doesn't bill output, and its launch post quotes 70 to 500 ms end to end.

Those probabilities are the model's belief split across your options, so they always add up to 1. Confidence is the top one rescaled by TypeSafe's formula, so an even split scores 0 and certainty scores 1.

What changed from a classic classifier is the interface: you describe the options in plain English on every call, with no training step. That advantage disappears when your labels never change and you already have labeled data, so there I'd still train a classifier and save decision models for options that are still moving.