In the spring of 2026, a chatbot on a cosmetic clinic’s website awarded the clinic’s managing directors medical specialist titles they had never earned. It was all made up. Before the Hamm Higher Regional Court, the excuse “that was an unforeseeable hallucination” didn’t hold up. The court viewed the chatbot as an integral part of the business organization and held the operator fully liable. Anyone who uses AI and publishes its output is held accountable for it. That is precisely why AI governance will become mandatory in 2026. This article shows how liability is actually distributed today and how you can protect your company.
TL;DR
36 percent of German companies now use AI, up from 20 percent the previous year. 53 percent cite legal uncertainty as the biggest obstacle (Bitkom, 2025) . Two German court rulings from the spring of 2026 settled the matter: The operator is liable for the output of its AI just as it would be for its own statement. At the same time, the next phase of the EU AI Act takes effect in August 2026. This article explains the legal landscape, contextualizes the two rulings, and outlines the governance components a company needs now.
AI governance is the system of rules, roles, and controls that an organization uses to manage, document, and account for its use of AI – from the selection of systems to human oversight to liability for the generated output.
What Is AI Governance?
Many companies still treat AI tools as if they were software purchases. A team tests a tool, finds it useful, and rolls it out. Legal responsibility is tacitly transferred along with it, without anyone consciously assuming it.
Let’s put it a little more bluntly: The moment an AI system communicates externally – in a customer chat, in a generated quote, or in an automatically generated response – its output becomes a statement on behalf of the company. Governance is the mechanism that keeps these statements under control.
It answers three questions: Who is authorized to use which AI and for what purpose? Who reviews the output before it is released? And who bears responsibility if something goes wrong?
AI governance goes beyond traditional IT governance. A conventional IT system does what it’s told and can be tested. A generative AI system produces new, not entirely predictable output with every query. It is precisely this uncertainty that is the reason why control must be applied both to the individual result and to the system that produces it. Governance ensures that people within the company know when they can trust an AI result and when verification is necessary.
Who is responsible if the AI produces nonsense?
Until recently, this was a question for specialized attorneys. Since the spring of 2026, there have been two specific rulings that every company executive should be aware of.
Munich Regional Court I: Google’s AI Summary as Original Content
On May 28, 2026, the Munich Regional Court I ruled on Google’s AI summary (Case No. 26 O 869/26). A Munich-based publishing house had filed a lawsuit because Google’s AI-generated summary claimed that the company was known for unscrupulous business practices and lured customers into subscription traps. The statements were entirely fabricated and did not appear in any of the underlying sources.
Google argued that it merely displayed third-party search results. The court saw it differently. It ruled that the AI summary constituted content in its own right, attributable to Google, and that Google was liable for it as a direct tortfeasor (LTO, 2026). Responsibility thus shifts from the individual AI output to the system behind it: specifically, its design, deployment, and control.
Hamm Higher Regional Court: The Chatbot That Invented Medical Specialty Titles
The case mentioned at the beginning of this article was heard by the Hamm Higher Regional Court (Case No. 4 UKl 3/25, decision of May 12, 2026). A beauty clinic’s chatbot had falsely attributed fictitious medical specialty titles to the clinic’s managing directors. The court classified the chatbot’s output as a business act and rejected the defense’s argument regarding “unpredictable hallucinations.” The court held that the chatbot is part of the operator’s business organization, and the operator bears full responsibility for its output (Datenschutzticker, 2026).
For companies, this is the more practically relevant decision. The case here involves a completely ordinary operator who placed a chatbot on their website, not a tech conglomerate with a global AI product. The courts’ message is the same in both cases: A hallucination is not a force majeure event. It is an operational risk inherent in the use of AI.
Both rulings share the same logic. The courts do not ask whether the company could have foreseen the specific error. They ask who put the system into operation and benefited from it. This shifts responsibility to where the decision to use AI was made: within the organization itself. In practice, this means that a disclaimer in the fine print is of little value if the output is presented to the outside world as binding information.
The 2026 Regulatory Framework
These rulings come at a time when the regulatory framework is also shifting. Five factors are important for companies.
1. EU AI Act: What obligations apply, and when do they take effect?
The EU AI Act (Regulation (EU) 2024/1689) has been in force since August 1, 2024, and is being phased in gradually. As of February 2, 2025, prohibitions on certain AI practices and a requirement for AI competence within companies have been in effect. As of August 2, 2025, the rules for general-purpose AI models, including governance and sanction structures, have been in effect. Starting August 2, 2026, the Act will apply generally, including the obligations for high-risk systems as specified in Annex III (EU AI Act Implementation Timeline, 2024/2026).
The AI Act uses risk categories. Prohibited systems are entirely banned. High-risk systems—such as those used in human resources or credit decisions—require risk management, data governance, documentation, and human oversight. Systems with limited risk, such as chatbots, are subject to transparency requirements. For many companies, this means they must know exactly which AI systems they are operating and which category these systems fall into.
2. AI Competence: The Unspoken Requirement Since February 2025
One component of the AI Act is often overlooked in the debate. As of February 2, 2025, the regulation requires providers and operators to ensure their staff have sufficient AI competence. This refers to the ability to use AI systems knowledgeably, assess their limitations, and identify risks (EU AI Act Implementation Timeline, 2024/2026).
This obligation links governance and empowerment. A ban without training leads to teams using AI covertly. Training without rules leads to everyone applying their own standards. Combining the two turns a legal requirement into a real advantage, because employees then use AI safely and productively.
3. Copyright: What the Code Requires for General-Purpose AI
A separate component concerns copyright. Under the EU AI Act, providers of general-purpose AI models must implement a copyright compliance policy and publish a sufficiently detailed summary of the content used to train their model (Article 53). The European Commission’s GPAI Code of Conduct from July 2025 specifies these obligations and organizes them into the chapters on transparency, copyright, and security (European Commission, 2025). This is relevant for companies that use such models in their own AI applications because generated text may contain protected passages. Anyone using AI to generate content should therefore be aware of the rights associated with training data and outputs.
4. AI Liability: Why the AI Liability Directive Was Scrapped and What Applies Instead
For a long time, a dedicated AI liability directive—the AI Liability Directive—was planned. In February 2025, the European Commission announced that it was withdrawing the proposal because no consensus was in sight (IAPP, 2025).
This does not create a legal gap. It has been replaced by the revised Product Liability Directive (EU) 2024/2853, which has been in force since December 2024 and must be transposed into national law by December 9, 2026. It explicitly defines software and AI as products and establishes strict liability for manufacturers, which also covers errors resulting from a system’s continuous learning (IHK Hannover, 2024). For companies offering AI-supported products or services, liability thus becomes more tangible and harder to avoid.
5. GDPR and AI Data
The third pillar concerns data protection. In its Opinion 28/2024 of December 2024, the European Data Protection Board (EDPB) clarified that AI models are not automatically considered anonymous. Whether personal data is contained in the model must be assessed on a case-by-case basis. Anyone relying on a legitimate interest as a legal basis must pass a three-step test (EDPB, 2024). In practice, this means that the use of an AI tool is always a data protection decision that must be documented.
AI Governance Frameworks: Guidelines for Companies
No one has to reinvent AI governance from scratch. External standards provide benchmarks for developing internal rules and metrics. Two frameworks have gained international acceptance.
The key international standard for AI management systems is ISO/IEC 42001. It describes how an organization establishes, operates, and continuously improves a management system for the use and development of artificial intelligence, similar to well-known standards for quality or information security management. Companies can seek certification under this standard to demonstrate that they manage AI systematically.
The NIST AI Risk Management Framework (AI RMF 1.0) from the U.S. serves a similar purpose on a voluntary basis. It identifies the characteristics of trustworthy AI and provides a process for managing risks throughout the entire lifecycle. Since 2024, a separate profile for generative AI has supplemented the framework to address the specific risks posed by systems such as large language models (NIST, 2024). Neither framework replaces legal obligations, but they do help translate the EU’s AI Regulation into concrete processes.
How to Tell If Your AI Governance Is Working
Governance that cannot be measured ultimately remains nothing more than a statement of intent. That’s why metrics are essential. KPIs for AI governance require systematic planning and monitoring, and they work best when they combine leading and lagging indicators. A governance dashboard consolidates these metrics at the operational, tactical, and strategic levels, allowing executives to assess the state of their AI landscape at a glance.
Key AI Governance KPIs:
Quality and Accuracy
Model accuracy measures a system’s correct predictions relative to a baseline model. Additionally, the number of misclassifications indicates how often an AI system produces unexpected or incorrect results. These metrics reveal whether a system is reliable enough for its intended purpose.
Fairness and Explainability
Fairness metrics analyze differences in prediction results across demographic groups, thereby uncovering discriminatory patterns. An explainability score measures the proportion of AI decisions that are comprehensible. Both metrics contribute to the characteristics described by the NIST framework as attributes of trustworthy AI, including fairness and explainability.
Controls and Audits
Regular human evaluation reviews AI results against defined thresholds and ensures effective human oversight. Automated audits, in turn, use AI-supported systems to detect compliance violations. The higher the risk of an application, the more rigorous the monitoring: A risk-based classification requires more frequent KPI checks for high-risk AI than for a simple assistance function. The completeness of documentation can also be tracked as a metric.
These concepts are standard practice and can be aligned with the trustworthiness characteristics of the NIST AI RMF. The specific metrics a company needs depend on its applications. Therefore, a consultation on AI governance usually begins with the question of which in-house processes are critical in the first place.
Get regular access to exclusive insights from thought leaders and practical tools on topics such as digital evolution, cultural transformation, future-readiness, resilience, mindfulness, and the design of new work environments.
From the Legal Landscape to a Leadership Task
The legal landscape is clearer than many companies realize. The difficult part is implementation. After all, AI governance cannot be delegated to the legal department and left to be resolved there. It is a leadership task because it determines roles, authorities, and culture.
In our training sessions, we regularly see the same pattern. Employees have long been using AI tools, often with personal accounts and without authorization. Leadership is the last to find out. This “shadow use” is the real governance problem because it creates liability risks that no one is managing. The first step, therefore, is not a set of rules, but honesty about where AI is already in use within the company.
Based on the legal landscape, we can identify several building blocks that virtually every company needs:
- An inventory of all AI systems in use, including those used unofficially.
- An AI policy that clarifies which tools are permitted for which purposes.
- Human oversight in areas where AI output affects external parties or informs decision-making.
- Documentation of the intended uses, the data basis, and the verification steps.
- A clear assignment of responsibility all the way up to the executive level.
These building blocks are not an end in themselves. They translate what courts and the AI Act require anyway into everyday work practices.
Conclusion: Responsibility & AI Governance Need a Name
The most common mistake in practice is governance without clear accountability. A policy that belongs to no one is not followed. That is why every critical AI application needs a designated person in charge and a process for reporting issues. Many companies consolidate this task into a single role that coordinates AI deployment, maintains the inventory, and serves as the point of contact for legal, IT, and business units.
A tiered model has proven effective: Business units are responsible for specific deployments, a central office sets standards and conducts reviews, and internal audit or an external party provides independent oversight. This interplay prevents control and use from being concentrated in the same hands.
Senior management remains accountable. The EU AI Act and current case law address the organization as a whole, and organizational liability falls back on the management level. A documented chain of responsibility therefore provides double protection. It fulfills regulatory requirements and, in the event of a serious incident, makes it clear that the company acted with due care. It is precisely this traceability that courts and regulatory authorities want to see.
Where the use of AI involves personal data, a data protection impact assessment is also required. It documents which data a system processes and what risks are associated with it. Combined with the AI inventory, this creates a robust picture of where responsibility lies within the company.
A real-world example illustrates this clearly. A sales team uses an AI tool to create quotes more quickly. Without governance, the following happens: The tool pulls data from an outdated price list, includes a delivery time commitment that no one has verified, and sends the quote to the customer. If this results in a loss, the company is held liable. With governance, the same process looks different: The tool is listed in the inventory, approved for the quoting process, accesses a well-maintained database, and a person approves the result before it reaches the customer. The time savings remain, while the risk decreases.
The Bitkom figure from the beginning of the year fits perfectly here. If 53 percent of companies cite legal uncertainty as the biggest obstacle, then governance is the way to remove that obstacle. Clear rules take the fear out of implementation and replace it with confidence in taking action.
Where does your company stand? The triangility AI Readiness Check shows in just a few minutes how mature your approach to AI is—from governance to team expertise. It serves as a starting point for a robust AI strategy.
Governance does not prevent the use of AI. It makes its use accountable. Companies that understand this early on benefit in two ways: they reduce their liability risk and create the conditions for teams to use AI confidently and productively. Those who put off addressing governance, on the other hand, do not shift responsibility—they merely obscure it.
Frequently Asked Questions About AI Governance
Who is liable for the output of an AI system within a company?
According to German court rulings from 2026, the operator is liable. Both the Munich I Regional Court and the Hamm Higher Regional Court consider AI-generated output to be content in its own right that is attributable to the company. Anyone who uses an AI system and publishes its output is held accountable for it just as they would be for their own statement.
What is AI governance?
AI governance is the system of rules, roles, and controls through which a company manages, documents, and takes responsibility for the use of AI. It ranges from the selection of systems to human oversight to clarifying liability for the generated output.
Does the argument “the AI hallucinated” protect against liability?
No. In May 2026, the Higher Regional Court of Hamm explicitly rejected the defense based on “unpredictable hallucinations.” Such an AI hallucination is considered an operational risk associated with AI use and does not exonerate the operator.
What does the EU AI Act require of companies starting in 2026?
Starting August 2, 2026, the obligations for high-risk systems under Annex III will apply. Affected companies must implement, among other things, risk management, data governance, documentation, and human oversight. Systems such as chatbots are subject to transparency requirements.
Is there a specific AI liability rule in the EU?
The proposed AI Liability Directive was withdrawn in February 2025. Instead, the revised Product Liability Directive (EU) 2024/2853 applies, which treats software and AI as products and must be transposed into national law by December 2026.
Is management personally liable for AI errors?
AI governance is a management responsibility. If AI systems are used without oversight, documentation, and clear accountability, this constitutes organizational negligence that falls back on management. A clear assignment of responsibility therefore also protects management.
Does my company need an AI policy?
In most cases, yes. As soon as employees use AI tools, liability and data protection risks arise. An AI policy specifies which tools are permitted for which purposes and makes the tacit use of AI manageable.
How does a company get started with AI governance?
The first step is to take inventory of all AI systems actually in use, including unofficial ones. This is followed by an AI policy, rules for human oversight, documentation requirements, and a clear assignment of responsibilities. An AI readiness check helps determine the level of maturity.
Which KPIs are suitable for AI governance?
Common metrics include model accuracy, misclassification rate, fairness metrics, an explainability score, and the degree of human control. Effective KPIs combine leading and lagging indicators and can be aligned with the trustworthiness characteristics of the NIST AI Risk Management Framework.
Which standards support AI governance?
Leading international standards include ISO/IEC 42001, the standard for AI management systems, and the NIST AI Risk Management Framework. Both provide benchmarks for internal rules and metrics and help translate the obligations of the EU AI Regulation into concrete processes.
Is my company liable for copyright infringements caused by AI?
Anyone who uses AI to generate text or images bears the risk that the output may contain protected content. Under the EU AI Act, providers of general-purpose AI models must maintain a copyright policy and document their training data. Before publication, verify the rights associated with AI-generated content.
Sources
- LG München I, Urteil vom 28.05.2026, Az. 26 O 869/26 –
- OLG Hamm, Entscheidung vom 12.05.2026, Az. 4 UKl 3/25 –
- EU AI Act, Verordnung (EU) 2024/1689 –
- Rückzug der AI Liability Directive – IAPP, 2025
- Produkthaftungsrichtlinie (EU) 2024/2853 –
- EDPB, Stellungnahme 28/2024 zu KI-Modellen –
- KI-Nutzung und Hemmnisse in deutschen Unternehmen –
- ISO/IEC 42001, Standard für KI-Managementsysteme –
- NIST AI Risk Management Framework (AI RMF 1.0) inkl. Generative AI Profile, 2024 –
- EU AI Act, GPAI Code of Practice, 2025 –
Develop AI leadership skills: The only learning journey for leaders that combines artificial intelligence, new leadership, and AI transformation.
Join our New Leadership Community:
We send you our monthly newsletter on leadership, culture, organization and technology. With exciting, curated inspiration for the new world of work.
Get in Touch
Contact Verena for personalized information on how to become more mindful as a leader.
You are currently viewing a placeholder content from Zoho Forms. To access the actual content, click the button below. Please note that doing so will share data with third-party providers.
More Information
