Company / September 16, 2026
Agents improve while they work
Most agents fail the same way every day, and a prompt edit is a guess. While. catches the failure in live traffic, trains the model on it, and proves the gain on a held-out test before anything ships.
Every company we talk to that has shipped an agent has run into the same three problems, usually within the first month.
The same mistakes, every day. The agent fails one particular kind of request. Maybe it is a refund with two line items, or a customer whose phone number is not on file. It failed that way on Monday and it will fail that way on Tuesday, and you find out from a support ticket rather than a dashboard.
The fix is a guess. Someone edits the prompt. Maybe it helps. Maybe it quietly breaks something that used to work. There is no way to know until it ships.
There is no proof before it ships. Nobody can show that the agent got better on the thing that was broken, or that it did not get worse on the things that were fine.
A prompt edit does not learn from the failure. A model can, if you train it on the failure. That takes a loop, and most teams have a dashboard instead.
The loop
While. runs next to the agent you already have, and it does four things.
- Watch. Live traffic becomes a map of what the agent handles well and where it struggles.
- Simulate. The weak spots turn into thousands of practice runs, with tools that break on purpose and customers who do not cooperate.
- Train. Every run gets a score, and the ones that pass train the next version of the model.
- Prove. You get a before-and-after score per kind of task, on conversations that were kept out of training.
Then it starts again, because the next day's traffic is the next training set.
What changes
A failure gets caught in the traffic the same day instead of arriving as a support ticket a week later.
The model trains on thousands of runs of the failure instead of getting a prompt edit and a hope.
You see a before-and-after score before anything ships, not after.
And you end up owning a model that learns your business, instead of renting one that never will.
Where we are
The loop is live now. It runs on your own compute with your own keys, and hosted training is optional.
Gray is before training, green is after. Each number links from the home page to its post or its recipe.
| Item | Value |
|---|---|
| Support agent, before | 5% |
| Support agent, after | 30% |
| Refund agent, before | 18% |
| Refund agent, after | 85% |
| Text-to-SQL agent, before | 53% |
| Text-to-SQL agent, after | 74% |
The method behind the first result is in Simulated data for support agents, and the third is in RL for text to SQL.
If you have an agent in production and a failure you are tired of seeing, ask for access.