Minimum must-have – Where does enthusiasm for AI end and responsibility begin within a company?

Many companies are implementing artificial intelligence faster than they can determine who is responsible for it internally. Whilst they do follow the regulations closely, there is a lack of internal rules, processes and clearly defined responsibilities regarding the use of AI within the company… And that is exactly how real problems arise. We have therefore summarised the essential minimum of what to bear in mind when it comes to AI governance within a company.

The most costly failures associated with the deployment of artificial intelligence in recent years have not stemmed from unmanaged technology, but rather from a lack of internal controls over its use.

Most discussions on artificial intelligence and the law focus on the European AI Act. However, the Act itself will not answer the questions your company faces on a daily basis: which tools your HR department is permitted to use and how; whether you can upload a record of a meeting with a client to a public chatbot; or who decides whether an AI system is permitted to process your orders independently? Each organisation must set these rules for itself.

Companies are generally well aware of which regulations apply to them. What is often lacking, however, are precisely the internal structures and safeguards that would put regulation into practice: binding rules, approval processes and specific individuals who bear responsibility. According to ISACA, an international professional association specialising in information technology governance, auditing, risk management and cyber security, only four out of ten organisations have a formal and comprehensive AI policy. Thirty per cent have only a limited policy in place, and a quarter have no active policy at all.

Indeed, the use of artificial intelligence does not transfer responsibility to the technology and/or its provider. It remains with your company, and management is also responsible. The duty of care entails acting in an informed manner and with due diligence; failing to take risks into account may also result in personal liability for members of the management bodies. Penalties under the AI Act can amount to up to EUR 35 million or 7 per cent of global turnover. The main risk, however, is the loss of trust among customers and staff, and the devaluation of investments in projects that need to be halted.

Lessons learnt from mistakes

The most costly failures associated with the deployment of artificial intelligence in recent years have not stemmed from unmanaged technology, but rather from a lack of internal controls over its use. Amazon halted development of its recruitment tool after it was found that it was systematically putting female job applicants at a disadvantage. Air Canada lost a dispute with a customer to whom the company’s chatbot had provided incorrect refund terms. The court ruled unequivocally: the company is liable for all information on its website, regardless of whether it was generated by a human or an algorithm.

Penalties for fabricated quotations from case law have also been imposed on renowned law firms and consultancy firms, despite the fact that they had in place internal rules governing the use of AI. After all, if your company’s AI controls lack clear standards, trained staff and a verifiable process, it exists only on paper.

Inventory check

The first step in internal governance is not to draw up policies, but rather to carry out an inventory check. You cannot manage a risk you are unaware of, and the actual extent of AI utilisation tends to be significantly broader than management is willing to admit.

Shadow AI, i.e. the use of publicly available tools without the employer’s knowledge, does not, as a rule, stem from an attempt to circumvent the rules.

Employees have become accustomed to using AI for summarising texts, translations and document analysis, often from a time when the company did not offer any approved alternative. The problem arises when this is done via personal accounts and outside the scope of any internal rules. Within the company, you then don’t know who is using which tool, what information they are entering into it, where the data is stored, or whether it can be used to train models. The most sensitive information includes customers’ and employees’ personal data, confidential client data, trade secret, and source code. The scope of technical oversight is, of course, limited; the company is unable to monitor access from private devices.

What to do, then? A blanket ban is not a viable solution; as a rule, it merely leads to a shift to an even less regulated environment. The way forward is to keep systematic records: identify which tools are used within the organisation, by whom, with what data, and from which supplier. The real solution, therefore, is to offer employees a secure and practical alternative – typically enterprise versions of the tools, with the appropriate contractual and security guarantees.

Internal governance need not be perfect from the outset, but it must exist and be open to further adjustments. We consider four elements to be the minimum requirements.

One key role needs to be added to this: the person responsible for internal governance, who has a mandate spanning business, IT, compliance and HR, and reports directly to senior management. Abroad, the title “Chief AI Officer” is commonly used for this role. However, it is not the title that matters, but the existence of the role and its remit. This is not a technical function, but a management one. And this is precisely where we see the greatest shortcoming in companies. Furthermore, in theory, it does not have to be held by just one person.

AI agents as new team members

Most companies today use artificial intelligence in an assistant role: the system makes suggestions, and the human makes decisions. The next generation of AI agents, however, is already capable of planning and acting independently. It retrieves information, communicates with other systems, sends a message or initiates a follow-up process.

In the context of governance, it is useful to think of an agent in the same way as a new employee. You also grant it only the access rights it needs within the company, define what it is permitted to do independently and what requires approval, and keep records of its activities. As autonomy increases, so do the requirements for oversight: if an agent is involved in sensitive decision-making, for example in the area of human resources or in credit assessment, it is subject to very strict regulatory rules. In teams that combine people and autonomous systems, it must be clear from the outset who approves the outputs and who is responsible for them.

Why it’s worth it

Internal governance is not an obstacle to innovation, but an important prerequisite for its sustainability. Companies with established processes deploy new tools more quickly, as they do not have to start from scratch on every project. Furthermore, demonstrable governance is becoming a selling point: financial institutions now routinely require it of their suppliers, and it is to be expected that other customers will gradually begin to make similar demands. Recently, effective AI governance has also emerged as a key component in controlling the costs associated with AI use and ensuring preparedness for any potential outages or changes to specific language models.

Just as cybersecurity and data protection management became standard practice years ago, so too should AI governance become standard practice. The competitive advantage will not lie in who uses AI, but in who can use it in a controlled manner. In the coming years, the most successful companies will not be those that are the first to deploy AI, but those that manage to integrate it safely into their day-to-day operations.

Key contacts