SemIf Turns Model Logits Into Typed Decisions

Many agent workflows use language models for bounded decisions: routing a ticket, retrying a tool, labeling content, or judging whether evidence supports a claim. SemIf, initially released as OpenJev, handles those operations by reading option scores directly from a model’s logits. The application receives a typed probability distribution without generating and parsing a JSON response.

Released under the MIT license, the independent project appeared within a day of TypeSafe launching Jev, its closed decision-model service. SemIf implements the same interface pattern with open checkpoints. TypeSafe’s model, training data, and calibration methods remain proprietary and are absent from the project. The repository has attracted roughly 4,000 GitHub stars and supports an RTX 3090, Apple Silicon, and CPU inference through llama.cpp.

A Decision API Built From Token Scores

For each decision, SemIf accepts a state, a question, and a finite set of typed options with descriptions. One model pass produces logits, the unnormalized scores assigned before token selection. SemIf extracts the scores associated with the declared option tokens, normalizes them into probabilities, and returns the distribution.

Direct scoring removes the token-by-token decoding loop, generated answer text, and JSON repair logic. Applications can select the highest-probability option, apply a confidence threshold, or pass the full distribution into later routing code.

Skip Decoding, Cut Latency

SemIf’s replay benchmark uses the same frozen Qwen3.5-4B checkpoint, input state, and 21 binary criteria on a single RTX 3090. The reported results compare direct option scoring with an autoregressive JSON array.