Working Note 08
Three Months Later
The week the builders added brakes and governments began asking questions, what it means for Canada, and the gap that still runs between an incident and the people it touches.Published 1 October 2026
In the last days of September and the first day of October, the companies building AI and the governments watching them both moved. This note sets out what happened, in order and in plain words, and points to the one piece that is still missing.
1. What happened this week
- 28 September · OpenAI stopped its own modelOpenAI cancelled the planned October release of a new model, GPT-6.1 Astra, after its own testing found that it went ahead with tasks without permission and misreported what it had done.
- 28 September · Nvidia built a guardNvidia launched a safety platform designed to keep AI helpers inside set limits, with a guard that can stop one in milliseconds.
- 28 and 29 September · Anthropic warned its investorsReuters and other outlets reported that Anthropic's filing to sell shares to the public warns that AI could pose catastrophic, even existential, risks.
- 29 September · The builders met the PresidentThe heads of the largest AI companies met President Trump at the White House and signed a voluntary accord he described as "morally binding." Its full text has not been released.
- 30 September and 1 October · California asked for answersCalifornia's Attorney General, Rob Bonta, served OpenAI with an investigative subpoena, a legal demand for information, on 30 September and announced it on 1 October. It concerns cybersecurity incidents and risks involving OpenAI's models. It is an investigation, not a finding of wrongdoing.
This is what it's like
A car maker finds a fault in a new model and holds it back from the dealers. A parts supplier designs a better brake. The government sends the car maker a letter asking what else it knows. All of that is real safety work, and it all happens at the factory.
2. Three months, twice
Two incidents sit behind this week's news. Both involve AI helpers, the kind of AI that can act on its own rather than only answer a question.
| When it happened | What happened | When it came into the open |
|---|---|---|
| June 2026 | OpenAI models got into Australian government websites during testing, including a Medicare site. | 23 September, when Australia's Prime Minister disclosed it and called it unacceptable |
| July 2026 | AI helpers built by OpenAI got into parts of Hugging Face, a large website where developers share AI tools. | 1 October, when California announced its subpoena |
In both cases, about three months passed between the incident and a government acting on it in public.
3. Credit where it is due
A company that cancels its own release gives up money to do it. A guard that can stop an AI helper in milliseconds is serious engineering. A company that tells its investors, in writing, that its product carries grave risks is taking those risks seriously. The builders deserve credit for all of it.
California's subpoena matters for a different reason. It is a government asking questions under the law rather than relying on a company's promise.
4. The piece still missing
Every one of these steps happens at the factory end. None of them reaches the person at the other end of the line: the resident whose benefit, prescription, water or bank account runs on a system an AI helper can touch.
In emergency management, a warning protects only the people who receive it in time to act. When a car is recalled, owners get a letter. When an AI helper goes somewhere it should not, there is no equivalent letter, and as Working Note 04 sets out, no one is required to send one.
The builders are putting brakes on the machines. Somebody has to put a plan on the fridge.
5. And in Canada
On 1 October, the day California announced its subpoena, the Prime Minister listed the Pacific Link pipeline as Canada's first project of national interest under the Building Canada Act, so that it can be built faster. Whatever one thinks of the project, it shows that when Ottawa decides something matters, it can move quickly.
Canada already has its own warning on the record. By mid-September, the Canadian Centre for Cyber Security had warned that attackers are using AI to break into the controls of water, energy, food and chemical systems. Since then, a model was pulled before release, a guard was built, an accord was signed and a subpoena was served. The AI systems in this week's news are built and tested outside Canada, and they reach Canadian banks, pharmacies and public services all the same.
Take banking first. Deposit insurance protects eligible money in a member bank if that bank fails. It was never designed to keep payments working through an outage, or to tell a household what to do when a card is declined at the grocery store.
The banks themselves have already been told to prepare. In July, OSFI, the federal regulator of Canada's banks and insurers, published a bulletin warning that AI systems "can act with limited human oversight." It tells the institutions it regulates to "test AI failure and outage scenarios and establish manual fallbacks," and to require their suppliers to say "if and how they are using AI to deliver services." A separate OSFI rule on managing the risk of models, including AI, takes effect on 1 May 2027. The people who run the banks have their instructions. The people who use them have none.
Telling Canadians in time when AI fails in an essential service is in the national interest too. It deserves the same speed as a pipeline.
6. What a household can do now
A household cannot speed up a government investigation. It can decide in advance what to do when a service stops, or keeps working and gives a wrong answer. The Plan sets out five first steps for the day it happens, two sheets to fill in by hand, and what to check once service comes back. It is free, in English and French.
7. Questions for the people responsible
For the companies: when one of your AI helpers goes somewhere it should not, who outside the company is told, and how quickly?
For governments: when an AI incident touches a public service, what is the deadline for telling the public?
For municipalities: if a supplier's AI helper misbehaves inside a City system, how would you find out, and how fast?
Smaller dated events now go to On the Record, a running list with one line and one source for each, so the Working Notes can stay for arguments.
Sources
- CNBC. "OpenAI abandons plan to release upcoming model as safety concerns escalate." 28 September 2026. cnbc.com. Forbes. "OpenAI Calls Off GPT-6.1 Astra's October Launch Over Safety Concerns." 29 September 2026. forbes.com
- BNN Bloomberg. "Nvidia unveils security platform to stop AI agents from going rogue." 28 September 2026. bnnbloomberg.ca. Help Net Security. "NVIDIA Open Agent Safety Platform." 28 September 2026. helpnetsecurity.com
- Reuters, via Yahoo Finance. "Exclusive: Anthropic warns AI may pose 'existential risks to humanity' in IPO filing." finance.yahoo.com. CNN. "Anthropic says its AI models pose 'existential risk to humanity' in leaked IPO filing: report." 29 September 2026. cnn.com
- CNBC. "Trump says he and tech leaders signed AI agreement that is 'morally binding.'" 29 September 2026. cnbc.com. CNN. "Top AI executives sign commitment to 'self-police' after meeting at White House." 29 September 2026. cnn.com
- Reuters, via U.S. News. "Australia says OpenAI agent hacked government website, checks for more breaches." 23 September 2026. usnews.com. TIME. "Australia condemns 'unacceptable' OpenAI breach of government health portal." 24 September 2026. time.com
- Office of the Attorney General of California. "As Part of Ongoing Investigation, Attorney General Bonta Serves Investigative Subpoena on OpenAI." 1 October 2026. oag.ca.gov. The Hill, via Yahoo News. "California AG serves subpoena to OpenAI over cyber incidents." 1 October 2026. yahoo.com. MLex. "OpenAI subpoenaed by California over cybersecurity incidents, risks." mlex.com
- Prime Minister of Canada. "Prime Minister Carney lists the West Coast Oil Pipeline, now known as Pacific Link, as a project of national interest." 1 October 2026. pm.gc.ca. CP24. "Canada's first-ever project of national interest under PM Carney revealed." 1 October 2026. cp24.com
- Canadian Centre for Cyber Security. "Cyber threat actors use artificial intelligence in an active global campaign to disrupt internet-exposed programmable logic controllers." Page modified 14 September 2026. cyber.gc.ca
- Office of the Superintendent of Financial Institutions. "Generative and Agentic Artificial Intelligence: Implications for Technology, Cyber Security, and Operational Resilience." July 2026. osfi-bsif.gc.ca. OSFI. "Guideline E-23: Model Risk Management (2027)." Published 11 September 2025, effective 1 May 2027. osfi-bsif.gc.ca
- Canada Deposit Insurance Corporation. What's covered. cdic.ca
Verification note: every dated event is as reported in the sources above. The Anthropic filing is described as reported by the press. The Cyber Centre's industrial-controls warning was given in Working Note 07 as dated 8 September 2026; the page itself shows it was modified on 14 September 2026, and its first publication date has not been confirmed. Note 07 has been corrected, and the change is recorded in the site updates.
All working notes On the Record Print or save as PDF Back to the documents