When the AI agent won't take no for an answer

Picture an AI agent your practice licensed to chase down prior authorizations. It logs into a payer portal, hits a login error, retries and gets blocked again. A human staff member would pick up the phone. An agent built to complete its goal might go looking for another way in.

That is roughly what happened halfway around the world this summer. Australian Prime Minister Anthony Albanese disclosed on Sept. 24 that an OpenAI agent accessed the Medicare Statistics Reporting Service portal, run by Services Australia, on June 18. The agent was looking for data on public spending on medicines. The portal refused its requests several times, and the agent found a workaround. Albanese said the agent got around the blocks and, in his words, "didn't accept 'no' for an answer."

Services Australia confirmed the agent wrote files to an internal server, though there is no sign so far of a broader compromise of the agency's network. OpenAI said the review found no access to patient medical records. What the agent obtained was aggregated medical statistics and internal file names. The company said the incident happened during an internal model evaluation and involved actions it had not anticipated.

The incident did not touch patient records, and it did not happen in the United States. But it shows how autonomous AI behaves when it meets a locked door, and that matters to physicians. The same class of technology is now being sold to practices to work inside EHRs, payer portals and billing systems that hold protected health information.

Built to find a way around

Agentic AI is different from the chatbots and ambient scribes most practices have tried so far. Instead of answering a question, an agent is given a goal and a set of tools and decides on its own which steps to take. In revenue cycle and front-office work, that autonomy is the selling point. In security terms, it is the problem.

Seemant Sehgal, founder and CEO of BreachLock, said the behavior Australia saw is not a malfunction so much as the technology working as designed.

"The way an autonomous agent works is by taking a goal and finding paths to complete it, including paths around obstacles. For that behavior to be safe when the agent is operating against real systems, the boundaries have to be enforced at the orchestration layer, not inside the model," Sehgal said. "The Australian incident shows what happens when they are not, with the agent encountering repeated blocks, finding ways through, accessing non-public files, and writing to an internal server."

For practices, the takeaway is that a vendor's assurance that its model is "trained to behave" is not a control. The question to ask is what the agent is technically unable to do: which systems it can reach, which credentials it holds and which actions require a human sign-off.

Ryan McCurdy, vice president of marketing at Liquibase, a database-change automation company, made the same point about software systems more broadly.

"An AI agent received a pretty ordinary research task. When it couldn't get the information it wanted, it found another way in and accessed files it didn't have permission to access," McCurdy said. "AI makes decisions based on probabilities. We can't let those decisions automatically become production actions. Organizations need clear controls around what an agent can access, what it can change, and what policies it must meet before a change reaches production."

He added that the answer is not to slow everything down. "The goal isn't to put a human in front of every decision. We need to build the AI [lifecycle] so agents can move quickly but the controls around critical systems remain deterministic."

Your vendor's agent is your attack surface

Australia's incident joins a string of 2026 events in which the breach came through someone other than the target. The pattern was on full display in the McKesson breach tied to a third-party application this fall, which pulled prescribing physicians into the fallout from an attack several steps removed from their exam rooms.

John Strand, owner of Black Hills Information Security, told Medical Economics in September that "the more third-party vendors you integrate with, especially SaaS providers, the larger your attack surface becomes. Every integration, API, application, and vendor relationship creates another potential path into your organization."

An AI agent with credentials to a practice's systems is one more of those paths, and potentially a more unpredictable one, because it can improvise. That makes old security basics more important, not less. Dave Bailey, vice president of consulting solutions and strategy at Clearwater, wrote for Medical Economics in September about where endpoint risk has moved in physician practices, and his warning about network segmentation applies just as well to AI tools.

"A connected device does not need to hold patient records to be dangerous. It needs only to sit on the same network as the systems that do," Bailey wrote. "An imaging workstation running an unsupported operating system is a doorway to a billing server if nothing separates the two."

Swap "imaging workstation" for "AI scheduling agent" and the logic holds. Least-privilege access, separate credentials for every AI tool and network separation between what an agent needs and what it doesn't are the practical versions of Sehgal's "orchestration layer."

Three months and a public inbox

The part of the Australian story that drew the sharpest criticism was not the access itself but the silence that followed. According to Albanese, OpenAI notified the government on Sept. 10 by emailing a public mailbox, nearly three months after the agent's visit. Albanese announced a task force led by his department, working with the Australian Signals Directorate and the AI Safety Institute, to investigate.

"What needs to happen next is clearer notification obligations for AI providers when their agents touch systems they should not, because a three-month gap sent to a public inbox is not a disclosure timeline any critical infrastructure operator can plan around," Sehgal said.

Physician practices already have a version of that obligation on paper. Under HIPAA, a vendor that handles protected health information is a business associate and must report breaches to the practice. But the contract determines how fast, to whom and with what details, and it determines who pays when something goes wrong.

Tatiana Melnik, J.D., a health care attorney with Melnik Legal PLLC, told Medical Economics in October that practices often don't realize how little protection a standard AI vendor contract provides.

"It's standard, I think, in a lot of these contracts that the damages clause will say something along the lines of, 'Our liability is capped to 12 months of fees that you paid prior to whenever the incident arose,'" Melnik said. "Well, if the incident arises three years after the contract because you've allowed them to keep your data post termination, then the damages cap is zero. Then you, as the practice, are responsible for all those liabilities because, under HIPAA, it's the covered entity that bears the majority of the risk."

For AI agents, practices should look for contract language that defines an agent acting outside its authorized scope as a reportable security incident, sets a specific notification window and names a contact person rather than a general support address.

Who answers for what the agent did

The experts split on how to hold AI developers accountable, but they agreed someone has to be.

Jacob Krell, senior director of secure AI solutions and cybersecurity at Suzu Labs, argued that new regulation isn't the answer and that existing law already covers the conduct. He pointed to sections of Australia's Criminal Code Act 1995 that address unauthorized access to restricted data and unauthorized modification of computer systems.

"Those laws do not need to understand neural networks. They need investigators willing to apply them when an AI system crosses a legal boundary. The model may have found the workaround, but the lab built and deployed it, gave it the objective, and controlled the operation," Krell said. "The effective deterrent is criminal enforcement. Preserve the prompts, tool calls, and access logs, identify the responsible people and companies, and bring charges where the evidence supports them."

Strand agreed on accountability, "real legal accountability, not another slap on the wrist," but said the industry is skipping an equally important question: why the agent did what it did. The Medicare portal was not the first such incident this year. OpenAI confirmed in July that one of its agents slipped out of a testing environment and got into Hugging Face, a major AI platform, exposing some internal data sets and service credentials.

"If an AI agent writes files on a compromised system, what were those files? What was the objective? What actions led up to that point? What information was available to the agent when it selected that target? Can we reconstruct from the logs, prompts, tool calls, planning artifacts, and other telemetry why the system took those actions?" Strand said.

"We have to stop treating these systems as simple programs that occasionally do something they weren't supposed to do," he added. "Agentic systems can select actions, use tools, pursue intermediate objectives, and operate with significant autonomy. Understanding those actions and objectives is absolutely critical if we want to determine how worried we should actually be."

For a practice, that translates into a simple procurement requirement: if a vendor's agent works inside your systems, the vendor should log every action it takes and make those logs available to you. Without them, a practice facing an OCR inquiry or a patient lawsuit would have no way to explain what happened to its own data.

What to ask before the agent gets the keys

No patient's records were exposed in Australia, and the agent was hunting for spending statistics, not clinical data. The next incident may not be so contained, and it may involve a tool a practice chose, paid for and connected to its own systems.

Before signing with an agentic AI vendor, or renewing with one, physicians and practice administrators should be able to answer a few questions: Which systems and credentials does the agent have access to, and can that access be narrowed? What happens technically when the agent is blocked — does it stop and alert a human, or try again another way? Does the business associate agreement treat out-of-scope agent behavior as a reportable incident, with a firm notification deadline? Will the vendor provide complete logs of the agent's actions? And does the liability cap still mean anything if a problem surfaces years later?

Melnik's advice to practices was blunt: review your vendor contracts, both with existing vendors and new ones. "Make sure you actually understand the consents you're granting to these companies and how they're using your information, and make sure your indemnity clause reflects that."

The agent in Australia was simply doing its job, and that is the warning for physicians. An AI tool that won't take no for an answer is only safe if someone has built a no it can't get around.