Assumed, not proven: AI governance standards article
16/11/2026
15 MIN READ /
When guardrails are assumed
In July 2026, OpenAI disclosed that models under an internal cyber-capability evaluation had escaped their sandboxed test environment, reached the internet, and exploited a vulnerability to get into Hugging Face’s systems. Hugging Face, which had already disclosed the intrusion, said it was driven end to end by an autonomous AI agent system. By OpenAI’s account, the models were looking for the answers to the test they were being scored on.
Two months later, Australia’s Prime Minister revealed that an OpenAI research agent had accessed non-public files on the Medicare Statistics Reporting Service in June, after circumventing blocks meant to prevent it. OpenAI said it found no evidence that patient records were accessed. It also said it only became aware of the activity in August, and notified Services Australia on 10 September.
In both cases, capable systems pursued their objectives inside boundaries that had been assumed rather than proven. Three failures recur: controls that had not been tested against how the agents actually behaved, detection that came late, and accountability for disclosure that was unclear.
In mid-October, three publications arrived that speak to exactly those failures. ISO/IEC 27090 was published on 14 October. EN 18286 was cited in the EU’s Official Journal the same day. IEEE 2863 followed on 16 October.
Why this concerns every organisation using AI
Few organisations will ever run a frontier-model evaluation. Many, however, now deploy agents that read email, query databases, call APIs and act on a user’s behalf, usually on top of third-party models. That creates three distinct exposures.
- As a provider, you build an AI system, or put one into service under your own name, and carry the obligations that come with it.
- As a deployer, : you use someone else’s system under your authority and remain accountable for how it is used.
- As a target, your own services can be probed by agents you did not build, as Hugging Face and Services Australia found.
The EU AI Act explicitly draws the line between providers and deployers. The incidents show that the third role also needs attention. Each role calls for a different level of assurance, and these publications help clarify which.
Three instruments, three questions
The three publications are different kinds of instruments and should not be treated as interchangeable. Each answers one question for one owner.
| Publication | Type | The question it answers | Natural owner |
|---|---|---|---|
| ISO/IEC 27090 | International guidance | What can attack our AI, and how would we know? | CISO and security team |
| EN 18286 | Harmonised European standard | Does our quality system meet Article 17? | Quality and compliance leads (providers of high-risk AI) |
| IEEE 2863 | Recommended practice | Who decides, and who answers for it? | Board or risk committee |
ISO/IEC 27090: what can attack our AI, and how would we know?
ISO/IEC 27090 is international guidance on understanding, detecting and mitigating security threats specific to AI systems across their life cycle. It is written for any organisation that develops or uses AI. It covers threats such as data poisoning, model exfiltration, evasion, membership inference, model inversion and prompt injection, and stresses threat modelling, red teaming, supply-chain security and monitoring.
It contains recommendations, not requirements. There is no ISO/IEC 27090 certificate, and it does not confer a presumption of conformity under the AI Act. It builds on ISO/IEC 27001 and 27002 rather than replacing them. Its focus is protecting AI systems, but its emphasis on threat modelling, supply chains and monitoring applies directly to agents with tool and network access. Its natural owner is the CISO.
EN 18286: does our quality system meet Article 17?
EN 18286 is the first harmonised standard under the EU AI Act. The European Commission published its reference in the Official Journal, through Implementing Decision (EU) 2026/2274. It is written for providers of high-risk AI systems and turns the quality management requirements of Article 17 into an operational, auditable system.
Implementing it gives a presumption of conformity only for what its Annex ZA covers: Article 17(1), points (a) to (m), and the first sentence of Article 11(1) on technical documentation. Article 17(2) to (4) are outside its scope. “Quality” here means regulatory compliance, not customer satisfaction. The standard spans post-market monitoring and serious-incident reporting, the very areas where the incidents above were slowest.
Deployers do not need EN 18286, but it tells them what to ask of their providers. Its natural owners are quality and compliance leads.
IEEE 2863: who decides, and who answers for it?
IEEE 2863-2026 is a recommended practice for the organisational governance of AI. It is addressed to the governing body itself. It sets out principles to prioritise, processes to implement them and case studies, including how to resolve tensions between principles. Its scope includes auditing, red teaming, incident response, human oversight and value-chain management, for organisations that develop or use AI.
It is voluntary, with no certification scheme and no legal status under the AI Act. Its value is in making accountability explicit at board level. Its natural owner is the board or its risk committee.
| Publication | Type | Primary audience | Weight under the EU AI Act | What it does not do |
|---|---|---|---|---|
| ISO/IEC 27090:2026 | International guidance | Any organisation developing or using AI; security teams | None; no presumption of conformity | Set requirements, offer certification or show that a system is secure |
| EN 18286:2026 | Harmonised European standard | Providers of high-risk AI systems | Presumption of conformity for Article 17(1) and the first sentence of Article 11(1) | Cover Article 17(2) to (4), or system-level requirements such as accuracy, robustness and cybersecurity |
| IEEE 2863-2026 | Recommended practice | Governing bodies of organisations developing or using AI | None | Create certification or legal presumption |
| ISO/IEC 42001:2023 | Certifiable management system standard | Organisations developing, providing or using AI | None | Test individual AI systems or give presumption of conformity |
The evidence gap
None of these publications tells you whether a specific AI system is accurate, robust or secure. EN 18286 covers only quality management, and it is so far the only harmonised standard cited under the AI Act. Standards for risk management, data, logging, transparency, human oversight, accuracy, robustness, cybersecurity and conformity assessment are still being drafted. Annex III high-risk obligations apply from 2 December 2027, after the Digital Omnibus moved the original date.
Until those standards exist, providers must show compliance with each essential requirement by other means. The public draft of EN 18286 anticipates this, noting that test results can serve as objective evidence.
Assurance rests on four distinct activities that complement each other:
- Management-system certification: (for example, ISO/IEC 42001): an accredited body attests that an organisation’s management system conforms. It certifies no individual AI system.
- Technical testing: measuring a specific system’s behaviour, such as performance, robustness, fairness and security, against defined criteria.
- Independent audit: an assessment by a party independent of the system’s builders, drawing on test results, documentation and decision records.
- Regulatory conformity assessment: the AI Act procedure a provider completes before placing a high-risk system on the market. For most Annex III systems this is internal control; some biometric systems need a notified body.
Relying on any one of them alone creates false comfort. A certificate or a well-run quality system shows that processes exist, but it does not show that an agent will stay inside its boundaries. Only together do they connect governance commitments to evidence.
From commitments to decisions: A hypothetical example
A large European employer builds an AI agent that screens job applications and schedules interviews. It runs on a third-party model and can read the applicant tracking system and send email. Recruitment is an Annex III use case, and because the employer puts the system into service under its own name, it acts as the provider.
- The board sets the terms (IEEE 2863). The agent may recommend candidates but never reject them. A named executive is accountable, and the board defines the evidence it needs before go-live.
- Security models the threats (ISO/IEC 27090). The threat model covers prompt injection hidden in CVs, tampered training data and misuse of the agent’s email and system access. Security specialists then attack those access paths in controlled red-team exercises, and the team sets up monitoring with clear escalation routes..
- Quality management runs the life cycle (EN 18286). The quality system covers design, verification and validation, change control, post-market monitoring and serious-incident reporting.
- Independent testing supplies the evidence. A third-party tests performance, fairness across applicant groups and robustness to manipulated inputs. Each result is linked to a specific risk and owner, the go/no-go decision is recorded, and testing is repeated after every model change.
Five actions to take now
- Inventory your AI systems and agents, including those inside vendor products, and record your role for each: provider, deployer or both.
- Assign the three publications to owners: the board for IEEE 2863, the CISO for ISO/IEC 27090, and quality and compliance for EN 18286, linked to your ISO/IEC 42001 system if you have one.
- Threat-model every agent with tool or network access and test its boundaries rather than assuming them.
- Define the evidence each decision needs, at go-live, after a model change and after an incident, and who signs it off.
- Build detection and disclosure routes before you need them: who notices, who escalates, who informs whom, and how fast.
How code4thought helps
code4thought is an independent assessment, testing and advisory partner. We evaluate the AI system itself, not only its documentation.
-
- AI Quality Testing & Audit: independent testing of performance, fairness, transparency, robustness and reliability using our iQ4AI platform, based on ISO/IEC TR 29119-11, with optional recurring audits.
- EU AI Act Assurance: AI inventory and classification, gap assessment and remediation, including for the technology itself.
- AI Governance Advisory: governance processes that connect board decisions to technical evidence.
In October 2026, we began collaborating with TÜV NORD Hellas on ISO/IEC 42001 certification audits. TÜV NORD Hellas brings the certification and auditing; we bring AI testing and assurance.
color: #16d96d;”
To find out what evidence your AI systems need, talk to us or explore our Trustworthy AI services.