Reef connects agent feedback to live model updates

Reef is a new Apache-2.0 open-source project that coordinates inference, feedback collection, training, evaluation, and deployment. It sits between an agent and its model infrastructure, records each interaction, accepts scores or written critiques, and publishes accepted model or harness updates to the live server. The repository passed 1,000 GitHub stars within days of launch.

Each proxied request receives a durable record ID that links the model output to later feedback. By preserving that connection, Reef can turn production interactions into eligible training records without a separate trace-matching pipeline.

Four modules close the loop

1. Serve. Accept agent requests, return model responses, and record the interactions.
2. Observe. Match incoming scores or critiques to the recorded requests.
3. Grow. Use the configured recipe to produce updated weights or an updated agent harness.
4. Commit. Evaluate the candidate, publish accepted changes, and add the result to version history.

The Observe stage uses receipt IDs rather than trying to infer which response a score belongs to. Recipes can then select eligible records, wait for enough feedback, and determine how an update should be built and evaluated.

A familiar API with durable receipts

The inference response includes an x-reef-agent-record-id header. Clients retain that value and submit it in the references array when reporting an outcome.

The client must preserve the response header, grade the result, and send the grade back to Reef:

import os

import httpx

reef = httpx.Client(
    base_url="http://localhost:8900",
    headers={
        "Authorization": f"Bearer {os.environ['REEF_TOKEN']}",
    },
    timeout=60,
)

model_path = os.environ["MODEL_PATH"]

response = reef.post(
    "/v1/chat/completions",
    json={
        "model": model_path,
        "messages": [
            {
                "role": "user",
                "content": "Return exactly: reef is ready",
            }
        ],
    },
)
response.raise_for_status()

receipt = response.headers["x-reef-agent-record-id"]
answer = response.json()["choices"][0]["message"]["content"]

report = reef.post(
    "/reef/report",
    json={
        "score": float(answer.strip() == "reef is ready"),
        "feedback": "matched",
        "references": [receipt],
    },
)
report.raise_for_status()