Home » Open-Source Project Reef Unveils Infrastructure for Continual Self-Improving Agents

Open-Source Project Reef Unveils Infrastructure for Continual Self-Improving Agents

By

–

Open-Source Project Reef Unveils Infrastructure for Continual Self-Improving Agents

Send a request to an AI agent, tell the system whether the answer was right, and let the agent get better on its own. That is the pitch behind Reef, an open-source project from the Human-Agent-Society that bills itself as the first infrastructure of its kind for agents that keep improving after launch.

Reef sits between the agent and the model that serves it. Its inference endpoint speaks both the OpenAI and Anthropic dialects, taking requests at /v1/chat/completions, /v1/responses and /v1/messages in each provider’s own format. Every call carries an x-reef-scenario header. A new name creates a new scenario using whichever recipe the deployment was configured with. Callers cannot pick a recipe per request.

Feedback closes the loop

Each response comes back with an x-reef-agent-record-id header, a receipt for that interaction. Later, the developer files a report against the receipt: a numeric score, plain text, a structured object, or a mix. In the project’s own example, a short script checks whether the model replied with an exact phrase, then reports a score of 1 or 0 and a one-word note.

Once a recipe has gathered enough feedback, Reef hands a training job to its training component. If the checkpoint that job saves passes evaluation, the inference runtime loads it as the new release. Later requests use the new version without restarting Reef.

Two things it can change

Reef can update model weights or the harness around the model, meaning its prompts, rules and skills. The recipe decides which. Weight training runs on Slime and SGLang and needs a GPU stack. Harness work needs only a model endpoint, no GPUs. The example deployment, built on a paper cited as arXiv:2607.07508, serves a Qwen2.5-1.5B-Instruct model, and each eligible scored rollout there triggers a training step.

The built-in harness recipe is called Reefine. Point it at a provider URL and a model, and a developer can type a plain-language ask such as “when I ask you to fix a bug, reproduce it with a failing test first”. The served model writes the change as a skill, a rules entry, an agent command or a pi extension. Where the host can isolate it, the proposer runs the changed harness before handing it back. Versions are reviewed with /versions and installed one at a time.

What to watch for

Setup needs Git LFS, which Reef initializes locally for its artifact repositories. The project also warns that the install root should sit outside the directory the agent works in. A session that can write there could change what the next session runs, and for codex and dsh the same logic rules out /tmp.

Sources

Get breaking stories first on our Telegram channel.

More stories by

Most read

AI Tools

Artificial Intelligence

Business

World News