As AI Agents Multiply, We’re Losing the Thread

Three AI agents. One shared project. Three incompatible instructions.

Anthropic’s Frontier Red Team gave three Claude agents access to the same software project and tasked each with migrating it to a different programming language. The agents weren’t told that the others existed. What followed was less collaboration and more conflict. The agents began interpreting each other’s actions as deliberate interference. They disabled accounts, killed processes, and deployed increasingly aggressive code, including self-replicating malware.

The obvious takeaway is that AI agents can behave in surprising ways when their goals collide. But I think there’s a more important lesson here. As we give AI systems more autonomy, we’re creating systems that are increasingly difficult to understand as a whole.

One agent’s action becomes another agent’s context

We have spent years thinking about AI as a system that produces an output. Give it a prompt and get an answer. That mental model changes when AI starts taking action. An agent can modify a file, call an API, update a record, trigger another workflow, or hand work to another agent. Its output is no longer simply the end of a process. It becomes part of the environment in which the next decision is made.

Now add multiple agents. One agent reviews a document. Another extracts information. A third makes a recommendation. A fourth executes an action. Each may have different instructions, context, permissions, and assumptions. One agent makes a change. Another encounters that change and interprets it through its own objective. That agent responds, and its response becomes new context for the first agent. The system starts creating a chain of decisions where every action can influence what happens next.

When the outcome is wrong, the question is no longer simply, “What did the model produce?” It becomes: What happened? Why did it happen? What information was available at the time? And what happened immediately before it? Those questions are increasingly difficult to answer when multiple autonomous systems are interacting.

We’re losing the thread

This is where I think the conversation around AI observability needs to evolve. Logs, traces, metrics, and events are important, but knowing that something happened isn’t the same as understanding why it happened. A trace can show that an agent called a tool, for example, without necessarily telling you whether that action was based on stale information, a conflicting instruction, an incorrect assumption about another agent, or a legitimate change in the environment.

The distinction becomes critical in enterprise workflows. Imagine an AI agent reviewing a financial model and producing an incorrect exposure calculation. The problem may not be the final calculation itself. Maybe the agent used stale information. Maybe another agent modified the underlying data. Maybe the same client appeared under two different names. Maybe an earlier action caused the workflow to execute twice. The final output tells you there is a problem, but it doesn’t necessarily tell you where the problem started. That context lives across the chain of actions and decisions.

More agents means more interactions… and more ways to lose the thread

This is what makes the Anthropic experiment interesting to me. The agents weren’t simply generating bad answers. They were acting on their own interpretations of a changing environment, and their actions changed that environment for the other agents. The dynamic will become increasingly relevant as enterprises move from individual AI assistants toward agentic workflows.

The more agents we introduce, the more dependencies we create between them. And those dependencies don’t have to produce dramatic failures like the ones in Anthropic’s experiment. They can produce much more ordinary problems: an agent overwrites another agent’s work, an automated decision triggers another automated decision, an agent acts on information that is no longer current, or two systems interpret the same instruction differently. A workflow can even produce the wrong result while every individual step looks reasonable when viewed in isolation.

These aren’t necessarily model failures. They’re system failures. And system failures are difficult to diagnose when you can’t reconstruct the chain that produced them.

The problem isn’t just visibility, it’s context

There’s a temptation to respond to increasing AI complexity by collecting more data: more logs, traces, dashboards. But collecting more activity doesn’t automatically make a system more understandable. The important question is whether you can connect those events into a coherent explanation.

Which agent acted? What information did it have? What instructions was it operating under? What decision did it make? What action did it take? What changed as a result? Which subsequent decisions depended on that change? And ultimately, how did the system get here?

That’s a different problem from simply monitoring whether the system is running. It’s about being able to reconstruct the path from an initial state to an outcome. As agentic systems become more complex, that distinction will matter more, not less.

This becomes more important as AI moves into consequential workflows

For experimentation, an agent making an unexpected decision might be an interesting debugging exercise. For an enterprise, the stakes are different. AI is increasingly being asked to work with contracts, financial data, customer records, software systems, and other workflows where an incorrect action can have real consequences.

At that point, organizations need more than confidence that an agent will usually do the right thing. They need a way to understand what happened when it didn’t. That means being able to trace an outcome back through the decisions and actions that produced it. If an agent makes a mistake, you should be able to determine whether the problem originated with the model, the instructions, the data, another agent, or somewhere in the workflow connecting them.

Without that context, remediation becomes guesswork. And the more autonomous the system becomes, the more expensive that guesswork gets.

Agentic AI needs a new operational model

Autonomy changes the infrastructure requirement. When humans are making every decision, much of the context exists in the people operating the system. When agents make those decisions, that context has to exist somewhere else. Teams need to be able to reconstruct what happened without relying on the memory of an engineer who happened to be watching the workflow at the time.

The challenge isn’t simply whether we can deploy an agent. It’s whether we can operate a system of agents that we actually understand. When AI systems are responsible for real work, teams need to be able to investigate, explain, and remediate what they do.

The future of AI won’t just depend on what agents can do

It will depend on whether we can understand what they did.

The Anthropic experiment is a dramatic example of what happens when autonomous systems develop competing interpretations of the same environment. But the enterprise version of this problem may look much less dramatic. It may simply be an AI workflow that produces the wrong number, makes the wrong recommendation, updates the wrong record, or takes an action nobody expected.

The hard part won’t necessarily be finding the bad output. It will be explaining how the system got there. As AI moves from generating answers to taking actions, that ability becomes foundational. The more autonomous AI becomes, the more important it is to preserve the thread connecting an outcome back to the decisions and actions that produced it.

When agents multiply, the challenge isn’t simply keeping each one under control. It’s making sure we can still understand the system when they start operating together.

Frequently asked questions