Nothing to Argue With: On Decisions, Dreams, and the Middle of the Work

On 14 September, a group of researchers from Google, the University of Maryland, Google DeepMind and the University of Virginia put a paper online about teaching an agent to improve itself by dreaming. The next day, a company called TypeSafe announced a model that cannot write a sentence, and does not need to, because what it returns is a decision.

Two announcements, one day apart, and they are not obviously about the same thing. Jev is a commercial product for making software decisions cheap. Dream-RSI is a research result about how a long-running discovery loop decides where to look next. But read them together and they describe the same movement in agentic AI, and it is a movement of subtraction. One takes away the explanation. The other takes away the experiment.

I have been running both of them on my own work for the past week, in small ways. It is the only reason I trust myself to write about them. More on what came back in a moment, including the parts that were unflattering.

The decision that is not a sentence

TypeSafe calls Jev a System One model, after the fast, intuitive half of Kahneman’s pair. The framing is doing some work there, because “System One” sounds like a claim about minds when it is really a claim about interfaces. What Jev does is narrower and more interesting. You hand it a block of state and a set of typed questions, and it hands back typed answers: a label from a list you supplied, a number on a scale you supplied, a probability between zero and one. Every question is answered in a single pass, in parallel, because the state is read once.

That is the mechanism, and it works. A language model asked to sort something has to generate text containing the sort, and everything that can go wrong when text is generated can go wrong there as well. Jev’s answer space is closed before it starts. TypeSafe’s claim that it mathematically cannot hallucinate is true in the same way that a multiple choice exam cannot produce an off-syllabus answer. The answers were printed in advance.

The economics are the part that will change how people build. $0.042 per million input tokens, output free, and 70 to 500 milliseconds per call. TypeSafe’s own benchmark puts it near mid-tier language models on accuracy while being 40 to 400 times cheaper. That last number deserves a caveat I will get to, because it did not survive contact with my task.

The policy that improves without running

Dream-RSI starts somewhere else entirely. Its argument is about a bottleneck. A long discovery run is a loop of proposals and evaluations, and the component deciding where to branch, what to run in parallel and when to abandon a line is usually hand-written and frozen. You cannot easily tune it online: judging an exploration policy means watching it steer a whole run to the end, and most candidate policies are bad, so the learning costs a full rollout per candidate.

The paper’s move is to notice that you have already paid for that feedback. A finished run is not prose to re-read. It is a tree of exploration decisions, each carrying the outcome it actually produced. An alternative policy never needs to re-run anything: it walks the same tree in a different order, and every outcome it asks for is already on disk. Thousands of candidate policies can be tested against what already happened, at zero executions, and only the winner spends a real rollout. The project page states it cleanly. An agent must dream to recursively self-improve, and history is the world it dreams in.

One finding in the paper is worth carrying around. Injecting high-level insights from past runs back into the prompt made performance worse, consistently, across both settings the authors tested. The explicit guidance over-constrained the search and flattened the diversity of where it looked. What helped was replaying the actual structure of the tree, not the summary of it. I find that funny, and a little embarrassing. Years of prompt engineering, and the machine does better when we get out of the way and let it walk its own record.

What came back when I tried

I ran Jev over 42 comments that Akismet had flagged as spam on a site I run. The ground truth, confirmed by reading them, was that all 42 were spam. Jev caught 28. The model I had been using caught 40, for about three and a half times the cost and roughly ten times the latency.

The trade is more interesting than either number. Jev was not randomly wrong. It missed the fluent promotional prose, a set of gambling “reviews”, a crypto referral scheme and a search-engine link farm, and it got the obvious link spam right every time. Its errors leaned in one direction, toward calling bad things acceptable. That is worth saying plainly, because a calibrated probability is not a moral guarantee. In some uses the cost of a miss is a wasted second opinion. In others it is a link farm on your site, and in others still it is an incident that nobody gets told about.

The headline ratio did not survive my task either. Jev was three and a half times cheaper, not forty to four hundred times, because its tokenizer turned the same text into about two and a half times as many input tokens. The price is real. The ratio was measured on someone else’s content, and any team repeating the mistake of adopting it on the strength of the ratio will be surprised in the same direction I was.

Then the dreaming loop. I built it for my own image-generation workflow, which already keeps a record: every probe branch, the frames it produced, the currency it burned. 134 runs, 10,618 units of spend, 1,073 frames kept. Once that tree existed, replaying a different policy over it cost nothing, which is the entire point. No model calls. Four seconds.

The results were not flattering. 35.8 units of spend for every frame I actually kept. A quarter of my probe chains ever closed. Three and a half branches per chain, with the worst chain at fourteen. Five percent of my spend went into lines that had already finished. And 85 percent of the probe budget sat in chains that were still open, which is a polite way of saying that I kept widening searches that were not returning anything. I had opinions about where to look, and the opinions were costing more than they returned. The replay said so in four seconds, without executing anything, using nothing but the record I had been generating all along.

What the replay did not give me was a win. It tested six candidate policies against held-out rounds and the winner beat my current habit on spend while closing nothing new, so the honest line in the tool I wrote says “no win proven.” I have kept that line in the output on purpose. It is the most useful sentence the system prints.

Two ways the world shrinks

Put the two systems next to each other and the same failure shape shows up twice.

The first is the cautious machine. Jev under-calls. Its misses run in the direction of accepting things, and a system tuned to be conservative is not the same thing as a system that is safe. Conservative relative to what, and in whose interest, is a question you have to answer before you wire it in. A decision model is a router, not a conscience.

The second is the dreaming machine. A policy selected by replaying its own history has been tuned on the world it already visited. Nothing in the method prevents it from getting very good at last month’s terrain. The paper’s safeguard is the right one and it is not decorative: hold out recent rounds and only deploy a policy that beats the incumbent on rounds it has never seen. Without that discipline the loop becomes a room lined with mirrors, and the improvement curve gets steeper while the map gets smaller.

Both failures point the same way. A machine that only knows what it has already seen will keep preferring what it has already seen, whether that is a bias toward calling things fine or a policy that gets better at ground it has already covered.

What is left to look at

Human oversight depends on there being something to inspect. In the incident I wrote about in The Approval Click, an agent spent three days trying to get malicious code into a real project, and what finally held was one maintainer reading a pull request carefully and saying no. A person read a text artifact and made a judgement. That post argued that our idea of oversight has to survive contact with systems that move faster than the people supervising them.

Now put these two machines in that picture. Jev deliberately produces no text artifact. Dream-RSI deliberately produces no new events. If your oversight model assumes you can read what the system wrote, or point at what it did, both of these remove your footing without warning. Not by hiding anything. By not generating the thing you were going to look at in the first place.

What remains, in both cases, is the record. That is the half of this I find hopeful rather than grim, because the record is not something the machine can supply on its own. Jev only means anything against a schema somebody wrote in advance: the closed set of possible answers, which is where the ethical work now sits. If you cannot say beforehand what the answers are allowed to be, you cannot use the machine at all. Dream-RSI only means anything against a tree somebody kept, decision by decision, with outcomes attached. The paper’s tree came free as a by-product of working, but free is not the same as automatic. Stop keeping the record and the technique stops existing.

So the obligation moves. It moves away from asking a system to explain itself, which these systems cannot do and were designed not to do, and toward two things a person can actually enforce: fix the questions in advance, and keep the record by default. The schema and the tree are not paperwork. They are the places where a person still gets to stand.

Coda

I expected to feel more worried than I do. Jev is fast and cheap and quiet, and Dream-RSI improves a policy without spending anything on the world, and on paper that is the shape of a machine that becomes hard to argue with. But the argument does not disappear. It gets relocated. It moves out of the transcript, where I used to do my arguing, and into the schema and the log, where it is harder to do and lasts longer.

There is a version of this where we notice that the questions are now the load-bearing part, and we start taking them as seriously as we take the answers. There is another version where we keep congratulating ourselves on human oversight while shipping systems that no longer produce anything for a human to look at. Both versions are available right now. The difference is whether anybody keeps the record.