Send a request to an AI agent, tell the system whether the answer was right, and let the agent get better on its own. That is the pitch behind Reef, an open-source project from the Human-Agent-Society that bills itself as the first infrastructure of its kind for agents that keep improving after launch.
Reef sits between the agent and the model that serves it. Its inference endpoint speaks both the OpenAI and Anthropic dialects, taking requests at /v1/chat/completions, /v1/responses and /v1/messages in each provider’s own format. Every call carries an x-reef-scenario header. A new name creates a new scenario using whichever recipe the deployment was configured with. Callers cannot pick a recipe per request.
Feedback closes the loop
Each response comes back with an x-reef-agent-record-id header, a receipt for that interaction. Later, the developer files a report against the receipt: a numeric score, plain text, a structured object, or a mix. In the project’s own example, a short script checks whether the model replied with an exact phrase, then reports a score of 1 or 0 and a one-word note.
Once a recipe has gathered enough feedback, Reef hands a training job to its training component. If the checkpoint that job saves passes evaluation, the inference runtime loads it as the new release. Later requests use the new version without restarting Reef.
Two things it can change
Reef can update model weights or the harness around the model, meaning its prompts, rules and skills. The recipe decides which. Weight training runs on Slime and SGLang and needs a GPU stack. Harness work needs only a model endpoint, no GPUs. The example deployment, built on a paper cited as arXiv:2607.07508, serves a Qwen2.5-1.5B-Instruct model, and each eligible scored rollout there triggers a training step.
The built-in harness recipe is called Reefine. Point it at a provider URL and a model, and a developer can type a plain-language ask such as “when I ask you to fix a bug, reproduce it with a failing test first”. The served model writes the change as a skill, a rules entry, an agent command or a pi extension. Where the host can isolate it, the proposer runs the changed harness before handing it back. Versions are reviewed with /versions and installed one at a time.
What to watch for
Setup needs Git LFS, which Reef initializes locally for its artifact repositories. The project also warns that the install root should sit outside the directory the agent works in. A session that can write there could change what the next session runs, and for codex and dsh the same logic rules out /tmp.
Sources
- GitHub – Human-Agent-Society/reef: Infrastructure for continually self‑improving agents
- Reef — Continual Self-Improvement for AI agents





