From tool-calling to true autonomy: our sandbox rebuild

We used to run our AI assistant like most LLM applications: an orchestrator in our backend made model calls, dispatched tools, and streamed text back to the browser.

That worked until frontier models became capable of doing sustained, autonomous work.

The useful unit of work moved from one good completion to one good autonomous session that assumes the agent has a computer.

So we gave ours one.

The architecture that stopped being enough

Our first assistant ran inside our backend: one orchestrator drove the chat loop, streamed responses, dispatched tools, and persisted artifacts. That worked for short chat turns. It broke down when customers started asking for work that took tens of minutes, required full-document context, or needed code execution.

Longer tasks didn't fit our request-scoped architecture. A prompt like  Summarize this paragraph fits comfortably inside a single request-response cycle, but trying to fit prompts like  Reconcile all submissions from today against carrier guidelines and generate a report flagging discrepancies or What percentage of extracted fields were edited by a human, per field per day, for June and July, for the Claims Intake workflow? into that same framework ran into conflicts with our existing background-task timeout policies.

Our users’ most complex prompts started to involve the assistant working for tens of minutes at a time to generate an answer. But our orchestrator expected LLM APIs to return quickly, so these long periods of work looked like hung requests. Hard-coded timeouts could terminate a turn before completion and discard work the assistant had already done.

Documents were still mediated through retrieval and custom tools. We started with a vector embedding pipeline that chunked documents and served relevant passages to the model. We also experimented with giving the agent tools to browse and inspect files more directly.

That helped, but as frontier model context window sizes grew, our internal experiments suggested that retrieval was becoming less effective than simply letting frontier models directly interact with input documents.

The agent had no runtime for executing code. We gave the assistant a tool it could use to delegate code generation to a different agent. However, since the assistant didn’t have access to the sandbox environment, it couldn’t iterate when the output wasn’t as expected.

This put the burden of verification on the user. They had to inspect the result and iterate on the answer in a back-and-forth with the assistant. Also, real work is multi-step: parse the documents, join against the reference sheet, compute the deltas, then format the output. A generate-only code tool forces the whole pipeline to be right in one shot, which is still beyond the capabilities of frontier models.

Drawn out, the system looked like this:

The shape of the problem was simple: the model could only interact with the world through interfaces we had predefined. It saw retrieved chunks of documents instead of full files, and every action depended on a tool or orchestration path we had built ahead of time.

Therefore, we gave the agent a computer

As agents became more capable at doing longer and longer tasks, we stopped thinking about frontier models as entities we call and started thinking about them as entities we host. Each assistant session would now run as an agent: a frontier model hosted inside a real working environment, with a filesystem, shell, persistent state, and enough time to carry multi-step work through to completion.

Two affordances mattered immediately.

A filesystem is something both sides can see.

A filesystem gives the agent the same natural affordances people use when working with documents: opening and closing files, understanding folder structure, moving between related files, and browsing pages in context. Instead of reducing everything to retrieved chunks, the agent can navigate documents as a whole, with nothing lost to chunking.

A shell composes capabilities we cannot predefine. In the old system, every useful action required a tool we had thought to build. Once the agent has a shell and an appropriate toolchain, it can combine primitives in ways we didn't enumerate beforehand: write code, run it, inspect the result, transform files, and iterate.

Giving the agent a computer bought us two things and cost us two things. The rest of this post is those four.

The infrastructure we built

Each assistant session now runs as an agent inside cloud sandbox environments with filesystem access, a shell, and agent harnesses from the frontier labs.

The backend no longer owns the agent’s tool-call loop. Instead, it coordinates events between the browser and the sandbox, persists turn state, and acts as the trust boundary for access to customer systems.

The sandbox runtime

Every sandbox needs a machine image, which means deciding what a capable agent should wake up with.

Each session now runs in an isolated E2B microVM with a filesystem, shell, document tooling, and provider-specific runners behind a common contract. The agent can inspect raw documents, run code, and generate Office/PDF artifacts.

Since our end users are in the insurance industry, we have worked to provide the agent with custom tools to help it answer questions in that domain. For example, one tool lets the agent perform governed SQL-style analytics over a customer’s submissions and extracted rows while another provides structured access to what our extraction workflows already pulled from each submission. NHTSA VIN decoding is available as a tool as well, because trucking submissions live and die on vehicle schedules.

Giving an autonomous agent capabilities without giving it the keys

The sandbox is powerful by design. That makes the trust boundary more important, not less.

We cannot naively trust either input documents from customers or model output. When it comes to inputs, the documents themselves are the attack surface, since by definition, these documents, including broker emails, attachments from unknown senders, and PDFs forwarded through multiple inboxes, arrived from outside our company. Prompt injection techniques could allow attackers to do anything the sandbox can do.

Even without the presence of malicious attackers, model-written code is untrusted by definition. Model-written code can loop forever, use up all its memory, delete files in the sandbox, or hammer an internal API over and over.

The agent holds no credentials for our database. No API keys, no customer passwords, and no internal config. When it needs customer data, it calls back through our backend over MCP using a short-lived JWT minted for exactly one turn.

That token carries an organizationID the agent cannot influence, a sandboxID that pins the token to one sandbox, and an explicit allow-list of tools. Before serving a call, the backend verifies the token and checks that the turn is still live. When the turn ends, the token becomes inert.

This gives us an important separation: the agent has a computer, but authority over customer systems remains in infrastructure we control.

Sessions now outlive requests

The agent harness owns the loop, but progress does not live inside a single request.

Autonomous sessions outlive requests. Nobody builds resume infrastructure for a 30-second chat turn; for a 10-minute agent turn you have to. The turn now runs in the sandbox whether or not anyone is watching, events land in a Redis buffer with a sequence number, and a client that comes back to an assistant session resumes instead of restarting.

This sounds like ordinary distributed-systems engineering because it is. A model deciding what to do next doesn't remove the old reliability problems; it adds new failure modes to them.

How agent infrastructure fails differently

Shipping the architecture was the easy part. The more interesting lessons came from watching it fail.

Never throw away completed work

Having the agent complete the work and having the assistant deliver it to the user are separate problems.

The worst example was turn-budget exhaustion: the agent could hit its limit on model and tool-call iterations after already writing the files a user asked for, but before producing its final response. The work existed in the filesystem, yet the user saw a failed turn.

We added a finalizer that delivers completed artifacts when a run hits its max-turn or context-window limit, while making clear that the run was cut short.

If useful work has already happened, an infrastructure failure should not erase it.

Infrastructure failures can look like model failures

We also saw cases where an agent started without access to the tools it needed. Nothing crashed: the model responded normally and correctly concluded that it couldn't complete the task. To the user, it just looked like a bad AI answer.

The more autonomy an agent has, the more infrastructure failures look like the model having a bad day.

That means tools need to report failures clearly - auth errors, timeouts, missing data - so the agent knows whether to retry, find a workaround, or stop.

This also changes observability: when an answer looks bad, we need to know whether the model failed or whether the environment around it did.

Those are now two questions we ask before shipping any new agent surface:

• What happens to useful work already in progress when this fails?
• How will we distinguish infrastructure failure from a bad model answer?

The last mile of performance

The problem we're actively attacking now is speed.

For a typical homepage assistant session, users wait about 7 seconds before the first token appears. Document-heavy turns can be much slower to include OCR parsing times.

We initially thought sandbox startup was a meaningful part of that delay, so we experimented with warming sandboxes as soon as a user focused on the assistant. It worked, but less than we expected: warming saved about 2.1 seconds on the turns where it applied, and less than a second when averaged across all turns because most conversations were already reusing an open sandbox.

That pushed us to instrument the seven seconds of silence more carefully.

Only about 1.9 seconds comes from our own stack starting up. Most of the remaining time is the model producing its first token.

That changes the optimization problem. The next gains are less about booting infrastructure faster and more about what we put in front of the model: how much context does an autonomous agent actually need up front, and how do we give it enough to act correctly without paying the latency and attention cost of telling it everything first?

What we're building toward

For users, the result is straightforward: the assistant can work directly with their documents, create the outputs they ask for, and carry longer tasks through without losing progress.

We’re seeing that translate into usage: between March and July 2026, users started 45% more assistant conversations, while total user messages grew 27%.

We’re now focused on making those runs faster, more capable, and more reliable.

It’s a strange mix of distributed systems, security engineering, and figuring out what an agent will do once you hand it a shell.

If that kind of work sounds fun to you, we're hiring.

‍