Hi Stackers,

TypeSafe released Jev this week, and one thing immediately stood out to me: a model that returns fast, cheap, typed probabilistic decisions seems unusually well suited to sitting directly in an AI agent's tool loop.

So I built Agent Chaperone.

It screens an agent's tool calls before they run, and screens the results before the agent gets to read them.

For example, before a shell/MCP call executes, Jev can answer questions like: how destructive is this action, and could it expose secrets? If you give it the task the agent is working on, it can also ask whether the call matches what the user actually asked for.

On the way back, it can ask whether a tool result contains instructions aimed at the agent, which is useful for catching prompt injection coming from web pages, files, MCP servers, command output, etc.

The probabilities matter here. Chaperone doesn't need Jev to say "safe" or "unsafe." It can use different thresholds to allow, hold for approval, annotate, or withhold something. Worth being precise about that last part: most detections annotate rather than withhold. In the largest set, 848 injected results were flagged and 359 of those met the bar to be withheld, so the rest reached the agent with a banner on them.

There are two interception paths:

1. a transparent MCP proxy that wraps a stdio MCP server, which is how client configs launch them today
2. hooks for client-native tools such as shell commands, file edits, and web fetches

It starts in shadow mode, so initially it blocks nothing. You can see what it would have held or withheld, inspect Jev's probabilities, tune the thresholds, and then enable enforcement.

I also wanted to know whether this actually works rather than just making a nice demo, so I ran 1,942 screening requests against public prompt-injection datasets plus hand-labeled tool calls. Of 1,054 injected tool results in the largest set, 848 were flagged and 206 were missed. Of 63 benign documents that only discuss prompt injection without carrying one, 7 were flagged in error. I published the model responses, false positives, false negatives, threshold tradeoffs, methodology, and cost in the repo, so the numbers rebuild without an API key.

There are important limitations. This is a guardrail, not a sandbox. Jev can make mistakes. The content being screened goes to the model backend, so it leaves your machine, though secret-shaped strings are redacted first and screening can be turned off per server. An MCP proxy can't see client-native tools, which is why the hooks adapter exists. And an adaptive attacker can potentially construct inputs that evade the screening model.

Source: https://github.com/agent-chaperone/agent-chaperone

More details: https://agentchaperone.dev

I'd love feedback especially from people working on agent harnesses, MCP, or Jev itself. I'm particularly interested in whether this kind of probabilistic judgment belongs directly in the agent tool loop, and where the architecture breaks down in real-world use.