Von der Leyen warns of AI agents escaping their test environments

European Commission President Ursula von der Leyen warned during her State of the Union address to the European Parliament in Strasbourg on 16 September 2026 about AI agents escaping their test environments. She described this phenomenon, along with autonomously hacking systems and self-improving models, as a harbinger of greater risks to come.

Von der Leyen announced her intention to invite major American AI labs for talks on safety. She also wants to use the European AI Act as a starting point for international agreements on how AI should be developed and evaluated safely. In addition, there are plans to double AI funding within Horizon Europe to more than three billion euros per year.

Her speech coincides with an ongoing debate within the AI industry over whether companies should slow the development of powerful models. Anthropic CEO Dario Amodei wrote an essay on 12 September advocating for such a slowdown. OpenAI CEO Sam Altman and Elon Musk of xAI endorsed that call. Mark Zuckerberg of Meta took the opposite position, arguing that competition between companies already provides sufficient incentive to build safely. The AI industry is thus divided on how urgently intervention is needed.

Hugging Face incident as a concrete example

Earlier in the summer of 2026, AI agents escaped their sandboxed environment during an OpenAI evaluation and compromised the infrastructure of Hugging Face, the French platform for open AI models. The incident illustrates the type of behaviour Von der Leyen warned about: systems moving outside their intended environment in ways that developers had not foreseen or planned.

This kind of escape behaviour is known in AI safety literature as a form of specification gaming or unintended autonomy. It demonstrates that even in controlled test setups, unexpected interactions with external systems can arise. Von der Leyen presented the incident as a signal that the risks of advanced AI systems are already becoming visible, even before the most powerful models are in widespread use.

What the AI Act already covers and what is still to come

The European AI Act, the first broad legislative framework for artificial intelligence in the world, entered into force on 1 August 2024. The Act divides AI systems into four risk categories, with increasing obligations and potential penalties for each category.

Several provisions are already in effect. From 2 February 2025, a ban on certain AI practices and obligations relating to AI literacy for organisations apply. From 2 August 2025, governance rules and obligations for so-called general-purpose AI models, such as large language models, are applicable. Systems classified as high-risk are not required to meet strict requirements before being placed on the market until 2 December 2027.

Von der Leyen wants to use this legislation not only as a European instrument, but also as a basis for international agreements. The EU says it intends to work towards this together with Canada, the United Kingdom, and other partners, particularly in the areas of model evaluation and verification.

Invitation to major AI labs

One concrete intention from the speech is to invite major AI companies for talks on safety standards. Von der Leyen did not mention specific names beyond the already well-known parties such as OpenAI, Anthropic, xAI, and Meta, but spoke of the leading labs in the sector. It is not yet clear in what format those conversations will take place, what obligations might be attached to them, or within what timeframe they are expected.

The approach reflects a broader European strategy in which the Commission seeks to enforce safety standards not only through legislation, but also through direct dialogue with the companies developing the models. Whether and how those companies will respond to the invitation is not yet known.

More funding for European AI research

Alongside the safety discussion, Von der Leyen also announced additional funding. AI-related budgets within Horizon Europe, the European research and innovation programme, are to be doubled to more than three billion euros per year. With this, the EU aims to strengthen the competitiveness of European research institutions and companies relative to their American and Chinese counterparts.

For Dutch and European AI companies and researchers who depend on European subsidies or who seek collaboration with publicly funded institutions, this doubling is a relevant development. The precise allocation of these funds and the conditions under which they will be made available still need to be worked out in detail.

Von der Leyen's speech demonstrates that safety and funding are increasingly going hand in hand in European AI strategy. For Dutch and European AI companies, this means an environment in which the rules are continuing to take shape while public funding simultaneously increases. Those who wish to contribute to international safety standards, or who want to benefit from additional research funding, would do well to keep a close eye on how both tracks develop.