The government just flagged multi-agent AI as a governance gap. If you run more than one agent, read this.
Australia's AI Safety Institute released its first report recently, and the subject matters more than the milestone. When AI agents stop working alone and start working together, the risks change shape, and no single organisation can fully see or control them.
The short version
If you run two AI agents that hand work to each other, or one that talks to a partner's, you already have a multi-agent system, whether anyone called it that or not.
The dangerous failures live in the interaction, not the individual agent. You can test each agent alone, find nothing, and still have a system that fails as a whole.
For regulated work, cascading failure and data leakage across boundaries are the two to watch. Both are stopped by design, logging, and a human at the right checkpoint.
If you run one AI system, the government's new report is background reading. If you run two that talk to each other, or one that talks to a supplier's, it is about you.
What "multi-agent" actually means for a regulated business
You don't need a research lab to be running multi-agent AI. A clinic with an AI receptionist that books appointments and a separate agent that chases recalls is already there. A broker with one agent answering member calls and another pulling claim status is already there. The moment two automated systems hand work to each other, or yours talks to a partner's, you have a multi-agent system, whether anyone called it that or not.
The government's framing is blunt: these interactions happen inside organisations, across organisational boundaries, and on the open internet, and they create risks that no single organisation can fully see, control or manage on its own.
The failure modes that don't exist with a single agent
The underlying research identified failure modes that only appear when agents interact. Two are worth translating out of the jargon, because they map directly to regulated work.
Cascading failures: one agent makes a small error, hands it to the next, and the mistake compounds across the chain instead of stopping. In a claims or recall workflow, that is the difference between one wrong record and a hundred.
Information leakage across boundaries: agents sharing data across contexts that were meant to stay separate. For anyone bound by the Privacy Act or APRA CPS 234, that is the exact thing your obligations exist to prevent, happening automatically, at machine speed, without a person in the loop.
There are more, conformity bias, communication breakdowns, agents that misjudge what another agent will do, but the pattern is the same: the danger is in the interaction, not the individual agent. You can test each agent on its own, find nothing wrong, and still have a system that fails as a whole.
What this means if you're deploying agents in healthcare, finance, or insurance
The uncomfortable part of the report is that it also points to governance gaps: places where a risk exists and no one is currently positioned to manage it. In plain terms, the technology is being deployed faster than the controls for it exist. That is not a reason to avoid AI. It is a reason to deploy it deliberately, with the interactions designed and documented, not left to chance.
Practical questions worth asking before you connect one agent to another:
Can you see what passed between them? If two agents exchange data, that exchange needs to be logged and reviewable, the same way any other step touching regulated data would be.
Where does a person sit in the chain? Cascading failure is stopped by a human checkpoint at the right point, not by hoping each agent behaves. Anything patient-facing or money-moving should have one.
Do the boundaries hold? If an agent can reach data or a system it was never meant to, that is the leakage the report warns about. Least-privilege access is not optional once agents start talking.
Is any of this crossing your organisation's edge? An agent talking to a supplier's agent, or anything on the open internet, is the highest-risk category the report names. Know where your system ends.
How we approach it
We build AI automation for regulated Australian organisations, and our answer to multi-agent risk is the same as our answer to single-agent risk, applied more carefully: onshore, logged, with a person on the decisions your obligations require, and boundaries designed in rather than bolted on. When work passes between agents, that handoff is a step we design and document, not a black box.
Independent testing matters more here, not less. Testing agents one at a time misses exactly the risks the government is flagging. The interactions are what need to be attacked and stressed, which is why that work belongs with someone who didn't build the system.
If you're deploying more than one agent, or about to, the questions above are worth answering before you connect them, not after.
Deploying more than one agent?
We build AI automation for regulated Australian teams, onshore, documented, with human oversight and boundaries designed in. Request a call and we'll walk through the multi-agent questions for your situation.