AI code is embraced internally, but governance lags far behind

AI coding tools have become virtually standard at large companies. The State of AI-Powered Software Development report by software security firm Black Duck shows that adoption among enterprise organisations has now reached 97 percent. Developers use the tools on a daily basis, and usage is expected to grow further in the coming years.

Yet governance of those tools is lagging considerably. Only one third of development teams have implemented governance policies for the use of AI-generated code. This means that the vast majority of organisations do deploy AI in software development, but without established frameworks covering security, quality control or liability.

Notably, the teams that do have their governance in order report better outcomes. They record higher efficiency and fewer code-quality incidents, according to the Black Duck report.

Rapid adoption, slow policy formation

The gap between use and policy is not new in the technology world, but with AI coding tools that gap is particularly wide. Whereas organisations normally take years to roll out new development environments, AI assistants such as GitHub Copilot, Cursor and similar tools have penetrated the daily working practices of development teams in a short space of time.

That speed makes it difficult for IT departments and legal teams to keep up. Questions about the intellectual property of AI-generated code, the origin of training data and liability for errors remain unanswered in many organisations. Black Duck points out that this represents not only a legal risk but also an operational one: without clear guidelines, developers do not know which tools are permitted, under what conditions, and how they should handle the output.

What governance looks like in practice

Governance for AI-generated code typically involves a combination of technical and organisational measures, such as:

• An approved list of AI coding tools within the organisation
• Mandatory code reviews in which AI-generated output is explicitly flagged and verified
• Policies on the use of proprietary or sensitive data as input for AI tools
• Agreements on licence risks for code that may have been derived from copyright-protected sources
• Logging and audit capabilities to enable retrospective tracing of which code was AI-generated

Organisations that have introduced these kinds of measures appear to operate not only more securely but also more efficiently, according to the report. The latter may seem counter-intuitive, but the explanation is that clear frameworks actually give developers more freedom to use tools without constant uncertainty about what is permissible.

Risks of unmanaged AI code

The absence of governance brings concrete risks. AI models generate code based on large volumes of training data, including public codebases. As a result, generated code may unintentionally contain fragments subject to a licence that is incompatible with the intended application, such as a GPL licence in commercial software.

There are also security concerns. AI models sometimes reproduce known vulnerabilities from their training data without the developer being aware of this. Without structured review processes, such vulnerabilities can make it into production undetected.

Black Duck, whose business focuses on software composition analysis and security, has a direct interest in this topic. That does not change the fact that the identified gap between adoption and governance is recognised by a broader group of researchers and industry organisations.

Relevance for the Dutch market

This issue is also relevant for Dutch companies and startups. The GDPR places requirements on how organisations handle personal data, including when it is used as input for AI tools. A developer who feeds customer data into an external AI assistant without a clear policy risks acting in violation of data protection rules.

The EU AI Act is also drawing closer. Although the legislation primarily targets the use of AI systems in high-risk applications, the broader focus on AI accountability will compel organisations to document and justify their internal development practices as well.

For founders and technical directors, the report is a prompt to take stock of which AI tools their teams are using and whether policies exist for them. Black Duck's figures suggest that in two out of three cases, the answer to that last question is no.