Building an Agent That Gets Better With Every Model

AI models are going to keep getting better.

For companies building on them, that creates an architectural question: how do you make each new generation of models improve your product without rebuilding the application around it?

A pattern is becoming clear—the model needs a real working environment—files, a shell, persistent state, and controlled access to the systems where the work happens.

That gives the product leverage on model progress. As models get better at reasoning, coding, tool use, and long-context work, more of that capability can flow into the product directly.

This is the architecture we’re using to build FurtherAI’s agent.

Where request-based LLM architectures hit their limit

Our first assistant used a conventional LLM application architecture: a backend orchestrator owned the chat loop, streamed responses, dispatched tools, and persisted artifacts.

That was the right abstraction for short chat turns. It was the wrong unit of abstraction for agentic work.

A prompt like Summarize this paragraph fits comfortably inside a single request-response cycle. But prompts like Reconcile all submissions from today against carrier guidelines and generate a report flagging discrepancies or What percentage of extracted fields were edited by a human, per field per day, for June and July, for the Claims Intake workflow? do not.

Those tasks can take tens of minutes. They require the assistant to inspect documents, query structured data, run intermediate steps, and carry work through to completion. In a request-scoped architecture, that kind of work looks like a hung request. Hard-coded timeouts can terminate the turn before completion and discard work the assistant has already done.

Documents were still mediated through retrieval and custom tools. We started with a vector embedding pipeline that chunked documents and served relevant passages to the model. We also experimented with giving the agent tools to browse and inspect files more directly.

That helped, but as frontier model context window sizes grew, our internal experiments suggested that retrieval was becoming less effective than simply  letting frontier models directly interact with input documents.

The agent had no runtime for executing code. We gave the assistant a tool it could use to delegate code generation to a different agent. However, since the assistant didn’t have access to the sandbox environment, it couldn’t iterate when the output wasn’t as expected.

This put the burden of verification on the user. They had to inspect the result and iterate on the answer in a back-and-forth with the assistant. But real work is multi-step: parse the documents, join against the reference sheet, compute the deltas, inspect the output, fix the edge cases, then format the result.

A generate-only code tool forced the model to get that whole pipeline right in one shot. The problem was not that the model could not write useful code; it was that the application gave it no way to run the code, inspect the result, and iterate.

Drawn out, the system looked like this:

The shape of the problem was simple: the model could only interact with the world through interfaces we had predefined. It saw retrieved chunks of documents instead of full files, and every action depended on a tool or orchestration path we had built ahead of time.

Moving the model into a working environment

The important shift was moving the model into an environment where new capabilities could compose without us predefining every path.

For agentic work, we think it is more useful to treat frontier models as something we host, not just something we call. Each assistant session would now run as an agent: a frontier model hosted inside a real working environment, with a filesystem, shell, persistent state, and enough time to carry multi-step work through to completion.

Two affordances mattered immediately.

A filesystem is something both sides can see.

A filesystem gives the agent the same natural affordances people use when working with documents: opening and closing files, understanding folder structure, moving between related files, and browsing pages in context. Instead of reducing everything to retrieved chunks, the agent can navigate documents as a whole, with nothing lost to chunking.

A shell composes capabilities we cannot predefine. In the old system, every useful action required a tool we had thought to build. Once the agent has a shell and an appropriate toolchain, it can combine primitives in ways we didn't enumerate beforehand: write code, run it, inspect the result, transform files, and iterate.

Moving the model into a working environment gave us two important capabilities: direct access to files, and a shell where capabilities could compose. It also introduced a new class of infrastructure problems

The infrastructure we built

Each assistant session now runs as an agent inside cloud sandbox environments with filesystem access, a shell, and agent harnesses from the frontier labs.

The backend no longer owns the agent’s tool-call loop. Instead, it coordinates events between the browser and the sandbox, persists turn state, and acts as the trust boundary for access to customer systems.

The sandbox runtime

Every sandbox needs a machine image, which means deciding what a capable agent should wake up with.

Each session now runs in an isolated E2B microVM with a filesystem, shell, document tooling, and provider-specific runners behind a common contract. The agent can inspect raw documents, run code, and generate Office/PDF artifacts.

This is where the vertical application layer matters. A general-purpose agent can work with files and code, but an insurance agent needs governed access to submissions, extracted fields, carrier guidelines, vehicle schedules, and customer-specific data. The model can keep getting better, but the product has to provide the domain context and trust boundaries that make it useful in an enterprise workflow.

For example, the agent can run governed SQL-style analysis over customer submissions, inspect the fields our extraction workflows pulled from each file, and decode VINs through NHTSA when working on trucking submissions. These are small domain primitives, but they they let a general-purpose agent operate inside an insurance workflow.

Giving an autonomous agent capabilities without giving it the keys

The sandbox is powerful by design. That makes the trust boundary more important, not less.

We cannot naively trust either input documents from customers or model output. When it comes to inputs, the documents themselves are the attack surface, since by definition, these documents, including broker emails, attachments from unknown senders, and PDFs forwarded through multiple inboxes, arrived from outside our company. Prompt injection techniques could allow attackers to do anything the sandbox can do.

Even without the presence of malicious attackers, model-written code is untrusted by definition. Model-written code can loop forever, use up all its memory, delete files in the sandbox, or hammer an internal API over and over.

The agent holds no credentials for our database. No API keys, no customer passwords, and no internal config. When it needs customer data, it calls back through our backend over MCP using a short-lived JWT minted for exactly one turn.

That token carries an organizationID the agent cannot influence, a sandboxID that pins the token to one sandbox, and an explicit allow-list of tools. Before serving a call, the backend verifies the token and checks that the turn is still live. When the turn ends, the token becomes inert.

This gives us an important separation: the agent has a computer, but authority over customer systems remains in infrastructure we control.

Sessions now outlive requests

The agent harness owns the loop, but progress does not live inside a single request.

Autonomous sessions outlive requests. Nobody builds resume infrastructure for a 30-second chat turn; for a 10-minute agent turn you have to. The turn now runs in the sandbox whether or not anyone is watching, events land in a Redis buffer with a sequence number, and a client that comes back to an assistant session resumes instead of restarting.

This sounds like ordinary distributed-systems engineering because it is. A model deciding what to do next doesn't remove the old reliability problems; it adds new failure modes to them.

How agent infrastructure fails differently

Shipping the architecture was the easy part. The more interesting lessons came from watching it fail.

Never throw away completed work

Having the agent complete the work and having the assistant deliver it to the user are separate problems.

The worst example was turn-budget exhaustion: the agent could hit its limit on model and tool-call iterations after already writing the files a user asked for, but before producing its final response. The work existed in the filesystem, yet the user saw a failed turn.

We added a finalizer that delivers completed artifacts when a run hits its max-turn or context-window limit, while making clear that the run was cut short.

If useful work has already happened, an infrastructure failure should not erase it.

Infrastructure failures can look like model failures

We also saw cases where an agent started without access to the tools it needed. Nothing crashed: the model responded normally and correctly concluded that it couldn't complete the task. To the user, it just looked like a bad AI answer.

The more autonomy an agent has, the more infrastructure failures look like the model having a bad day.

That means tools need to report failures clearly - auth errors, timeouts, missing data - so the agent knows whether to retry, find a workaround, or stop.

This also changes observability: when an answer looks bad, we need to know whether the model failed or whether the environment around it did.

Those are now two questions we ask before shipping any new agent surface:

• What happens to useful work already in progress when this fails?
• How will we distinguish infrastructure failure from a bad model answer?

The last mile of performance

The problem we're actively attacking now is speed.

For a typical homepage assistant session, users wait about 7 seconds before the first token appears. Document-heavy turns can be much slower to include OCR parsing times.

We initially thought sandbox startup was a meaningful part of that delay, so we experimented with warming sandboxes as soon as a user focused on the assistant. It worked, but less than we expected: warming saved about 2.1 seconds on the turns where it applied, and less than a second when averaged across all turns because most conversations were already reusing an open sandbox.

That pushed us to instrument the seven seconds of silence more carefully.

Only about 1.9 seconds comes from our own stack starting up. Most of the remaining time is the model producing its first token.

That changes the optimization problem. The next gains are less about booting infrastructure faster and more about what we put in front of the model: how much context does an autonomous agent actually need up front, and how do we give it enough to act correctly without paying the latency and attention cost of telling it everything first?

What we're building toward

For users, the result is straightforward: the assistant can work directly with their documents, create the outputs they ask for, and carry longer tasks through without losing progress.

We’re seeing that translate into usage: between March and July 2026, users started 45% more assistant conversations, while total user messages grew 27%.

The goal is not to rebuild the assistant around every new model release. The goal is to make the architecture general enough that better reasoning, better coding, better tool use, and better context handling show up as a better product.

That is what it means, in practice, to build an agent that gets better with every model.

The work ends up being a strange mix of distributed systems, security engineering, product infrastructure, and figuring out what an agent will do once you hand it a shell.

If that kind of work sounds fun to you, we're hiring.

‍