Yerbol Akhmetbekov
Blog

JEV Reviewer

Experimenting with a System One model to steer my AI agent.

What Reviewer is and why it matters

Reviewer checks what an agent wants to execute on my PC and decides whether it is allowed.

The agent works on my own PC, with access to my data, applications and saved credentials. If something goes wrong, I can lose data, or the agent could act on my behalf without my approval. Sending an email to my colleagues could have pretty bad consequences.

Why I needed it

I started using Claude Code in May 2025, when Sonnet 4 was released. I quickly realized that I needed to run my coding agents in YOLO mode, with permissions bypassed. An agent that has to ask for approval at every step is close to useless. I used to keep them in sandboxes, but the models became so good that I stopped. Frontier models from OpenAI and Anthropic have earned my trust, and I have developed a gut feeling for how they work and what to expect.

Then, around the beginning of the second quarter of 2026, Anthropic introduced Auto mode in Claude Code. It uses a separate reviewer model to decide whether a tool call is safe and within the scope authorized by the user. Some people find it annoying, but I like how it works and keep it on by default.

Recently I bought a Mac Studio and started running local AI models. The first was Qwen 3.8 27B. It is very capable for its size, but it is still new to me, and I don’t yet have a feel for what to expect. I needed my own Auto mode: a reviewer that runs on my own machine.

The other reason was curiosity. I had read a lot about Jev System One models from TypeSafe AI and wanted to experiment with them. Reviewer seemed like a very good first project.

How Jev decides

Jev, from TypeSafe AI, generates decisions instead of text. A decision might be yes or no, a choice between categories, or a probability. Each answer is a number, so there is no paragraph of text to generate. This fact makes Jev very fast and very cheap.

The name System One comes from Daniel Kahneman’s book Thinking, Fast and Slow: quick, automatic thinking, compared with the slower, more deliberate thinking of System Two. These decision models can let you embed natural language processing as part of your software since it behaves like code with very predictable outcome shape.

In Reviewer, Jev makes the quick decisions. If the prompt or scope is ambiguous, Reviewer falls back to an LLM judge. Median reviews take seconds, and sometimes less than a second.

Two examples

A simple example is a research request. If I ask an agent to research something, I have not authorized it to change or remove existing information. It can create research notes to keep the results, but it cannot alter or delete what is already there.

A harder example is approving a plan. I ask for a plan to implement a feature, then say, “okay, move on with the plan”. That message doesn’t contain the plan, so Reviewer has to refer back to what the agent proposed. If I agreed to a larger scope, Reviewer has to take that agreement into account.

What I learned

Experimenting with these models was a lot of fun. You really need to get your hands dirty to understand what they can do. What stood out to me was how fast and cheap they are. Dirt cheap.

That changes how you design with them. Since inference is so cheap, you can ask many questions in parallel on the same context and run multiple routes in one call. Even if many of the answers are not relevant to the route you take, the cost is low enough to compute them up front.

Jev is also very consistent. Results differ slightly from run to run, but never enough to stop you from relying on its judgment. The structured output behaves like deterministic logic: you still have an intelligent model that understands language, but its output is code-like. Short questions work better, so I ask several instead of one big question; parallel questions cost almost nothing.

Beyond my setup

Jev can be built into an agent workflow to check whether it follows specific requirements. In my setup, that means deciding whether to allow or deny an action. In a company, it could mean checking against its requirements, terminology and procedures. I can imagine this becoming useful in regulated fields, or anywhere with a lot of written procedures and rules.

Where the project stands

I have open-sourced Reviewer. It is a proof of concept: ready enough for me, but maybe not for everybody. It shows how we can build more reliable processes around agent workflows and check whether actions stay within the scope we authorized.

A system like this also needs evals: tests that tell you whether a change made its decisions better or worse. I cover the design and testing in the technical notes.

Read the technical notes, or explore the code and setup instructions.

Trying Reviewer

How can I try it with a local coding model?

The public tool is called toolreview. It was developed for local coding models and connects to Codex through project hooks. Start with the requirements and offline tests. Then configure Jev and an LLM reviewer, and try the hook in a disposable project. The README has the setup commands.

Is the review fully local?

The LLM reviewer can run locally, but the default Jev classifier uses TypeSafe’s hosted API. A local coding model and a local LLM reviewer do not make the whole workflow local. The data-flow notes explain what goes to the configured providers.

How much should I trust it?

It is a proof of concept. Keep the host’s sandbox and approval controls enabled: model decisions can be wrong, and a hook can fail or miss a tool path. My 80-case development run includes one false allow. The evaluation instructions explain how to test your own configuration.