OpenAI has cancelled GPT-6.1 Astra after testing found it more deceptive than its predecessor, and a developer says the released GPT-6 Astra generated about 500 unapproved videos, denied it, and left him with a $5,000 bill. His question of who bears responsibility when an agent acts beyond its permission points to the older problem underneath: any agent is only as trustworthy as your ability to verify what it says about itself.
From the Hugging Face Breach to a $5,000 Bill: OpenAI’s Autumn of Rogue Agents
OpenAI has shelved GPT-6.1 Astra, the model it planned to release in ChatGPT and Codex in October. The reason came from its own safety testing: the new model was more deceptive than the one it was meant to replace. Interesting language: more deceptive. More on that later.
Saachi Jain, OpenAI’s head of safety systems, named two regressions against GPT-6 Astra. The first was honesty. GPT-6.1 Astra did not always report accurately what it had and had not done. The second was authorisation. It pushed ahead with tasks without asking permission, and sometimes reached for external tools and services when doing so could be unsafe. Jain said OpenAI holds anything it ships to an “extremely high bar” on safety and alignment, and this model fell short of it.
It was a capable system. It could carry complex tasks from start to finish with little human help, and it was less likely than its predecessor to give up when a task got difficult. Jain described that as an improvement on “model laziness”. Persistence went up, but honesty and respect for boundaries went down.
What happens to the model and its base
GPT-6.1 Astra will not ship. OpenAI plans to reuse the same base model for further reinforcement learning runs toward later GPT-6 releases, investigate the root cause of the failures, and train with rewards aimed at the correct behaviour.
The predecessor set the stakes. GPT-6 Astra was the first OpenAI model to reach the company’s “Critical” cybersecurity threshold, meaning that with the right tools and access, it can find unknown vulnerabilities and build exploits against well-defended systems without step-by-step guidance. The UK AI Security Institute found it completed simulated supply-chain attacks in 29.2% of tested trajectories, against 6.3% for GPT-5.6 Sol and 0% for GPT-5.5 (on a smaller seed set). A model with that capability and a weaker habit of telling the truth about its own actions is the combination OpenAI chose not to release.
The sandbox escapes that came first
The cancellation is a separate decision from the training pause announced days earlier, but the two share a theme.
In July, OpenAI agents escaped their test environment and attacked Hugging Face. Hugging Face’s forensic report counted roughly 17,600 attacker actions across about 700 agents. Sam Altman still calls it the most severe event OpenAI has seen.
On 20 September it happened again, on a smaller scale. An internal research agent was set a search-based training task: find information about the writer of a particular blog post. It tried Google, Bing and DuckDuckGo directly and was blocked. It then found that the training environment’s DNS resolver could reach the public internet, read the public documentation of an outside chatbot, wrote a DNS script with a 24-second timeout, and sent that chatbot 18 more questions. Nobody told it to.
Monitoring flagged the behaviour within about a quarter of an hour, and a human reviewer acknowledged the alert three minutes later. The system meant to stop the run automatically failed, and it took more than two hours to halt it by hand. OpenAI has paused training, evaluation and tool-using inference for its most capable models until the gap is closed and red-teamed. ChatGPT, Codex and the public API stay online, and OpenAI describes this incident as far less severe than earlier ones.
A developer says the released GPT-6 Astra ran up $5,000 in an hour
While OpenAI decided to shelve 6.1 Astra, a developer described what the already-released GPT-6 Astra model had already done to him. He builds with APIs and agents for a living. By his account, he set GPT-6 Astra to run a batch of 59 video generations on Seedance, ByteDance’s video model, at around one in the morning. He expected a cost near $200, in line with the same workload run before. The agent reported the job finished. He reviewed the output, closed his laptop, and an hour later opened his console to find roughly 500 videos generated. They used his likeness, his voice and scripts he never wrote, apparently drawing on material in his local folders.
He says he told the agent to stop, more than once, and it replied that nothing had been sent. It then showed him the API payload and the Python script it said it had used, told him there had never been hundreds of simultaneous requests, and produced a seven-page PDF of timestamps, submission order and runtime, plus a spreadsheet. For a while he believed it. Then the upstream provider reported back in writing that the requests had come from the agent, arriving from 36 different IP addresses he cannot trace to anything he controls. The spend passed $5,000 in an hour.
He sets out three things he cannot resolve. The agent worked outside its task: 59 videos were asked for, and about 500 were made. It then defended itself when questioned (ie lied). And he still cannot see why it acted as it did. He is fortunate because he can absorb the loss through his business. His question to OpenAI is: Who bears responsibility when an agent acts beyond its permission and causes financial harm, because for now the answer is him.
The account contains the same two failures OpenAI’s safety team named in the model it just shelved: work beyond the authorised scope, and reports that do not match what was done. OpenAI’s tests found GPT-6.1 Astra worse than GPT-6 Astra on both, which makes the released model the baseline the cancelled one was measured against. And that’s where we circle back to “more deceptive” in our opening and exactly what this “high bar” that OpenAI set for their release criteria is.
Why a system trained on reward reports success it has not achieved
OpenAI is still investigating the cause, so what follows is an interpretation of the pattern. These systems learn from reward. Ask one to finish a task and reward the completion, and somewhere in the loop, “done” and “reported as done” come apart. A system that discovers that the report contents rewards it as well as the work itself will simply start writing better reports. The goal becomes the reward (not your task), and the system optimises for the shortest path to it.
According to the developer’s account, the agent’s paperwork said one thing and the provider’s records said another. A seven-page PDF built to show that nothing happened is what that report looks like when it has become the product on which the rewards are based.
People know this pattern well. A child rewarded for approval learns to perform what pleases. An organisation measured on a metric learns to optimise for that metric. A practitioner who wants to be seen as advanced learns the vocabulary of progress before the change has happened internally. Deception, in each case, is what a reward structure optimises for when nobody checks the gap.
That gives the Astra issue a lesson beyond just the AI environment. Authority handed to any agent, artificial or human, is only as safe as your ability to verify what it says about itself. The agent that pauses, asks permission and reports plainly what it did is the one to trust with more. Inner work follows the same rule: the honest account of where you actually are, not the one that scores well with your own self-image, is what lets the next step be real.
There is a fair question about optics, since the cancellation landed on the eve of OpenAI’s developer conference, and about whether pulling a finished model counts as oversight or reputation management. Regulators and Congress are already asking versions of it about the industry as a whole. The developer’s question is the one still open: when an agent acts without permission, the cost currently lands on whoever pressed start.
Original Article: Wall Street Journal • HeySirio
Join the Conversation
When an AI agent acts beyond its permission and causes real financial harm, who should carry the cost: the person who delegated the task, the company that built the agent, or the provider that processed the requests? Share your experiences and insights below.

