EU AI Act Essentials: What Procurement Needs to Know

Why Procurement Should Care

The EU AI Act presents unique challenges and opportunities for procurement professionals. Given the Act's stringent requirements and the heavy penalties for non-compliance – 7% of global annual revenue or €35 million, whichever is higher – procurement teams must proactively manage their AI technology providers and ensure all AI deployments meet the new standards.

This post outlines the key provisions of the Act and offers practical steps for procurement organizations to achieve compliance.

Key Tenets of the Act

• Risk-based Approach:

  The Act leverages a risk-based approach, which means that different requirements apply in accordance with the risk level of the system. Those risk levels are:

  • Unacceptable Risk: AI systems that manipulate human behavior or exploit individuals’ vulnerabilities (e.g., age or disability) to distort their behavior are considered an unacceptable risk and violation of fundamental human rights. These systems will be banned under the Act.

  • High Risk: AI systems identified as high risk include critical infrastructure. High-risk systems will be required to comply with strict requirements, including risk-mitigation systems, high-quality data sets, activity logging, detailed documentation, transparent user information, and human oversight.

  • Limited Risk: AI systems classified as limited risk, such as chatbots, are designed to ensure customers/users are informed that they are interacting with an AI system.

  • Minimal Risk: There are no restrictions to minimal-risk AI systems, such as AI-enabled video games or spam filters.

• General-purpose AI (GPAI) Models

  The Act now has additional rules for general-purpose technology. GPAI models that do not pose systemic risks will be subject to some limited requirements.

• Governance Architecture

  The Act establishes several new governing bodies, including:

  • An Artificial Intelligence Office responsible for Act enforcement

  • An advisory forum comprised of industry experts and stakeholders who will provide technical expertise to the Artificial Intelligence Board and the European Commission

  • An Artificial Intelligence Board comprised of member states’ representatives to advise and assist on the consistent and effective application of the Act

  • A scientific panel of independent experts to assist and advise on enforcement of the Act

• Integration with GDPR

  The General Data Protection Regulation (GDPR) will govern the processing of personal data in relation to the Act.

Impact on AI Providers and Deployers

The Act applies to providers/developers of AI software and systems who place them on the market for consumption, whether the provider charges a licensing fee or provides the AI technology as open source. It also applies to importers and distributors of AI technology in the EU. Finally, the Act applies to companies that deploy AI technology. These “deployers” can be legal persons or entities that, within their authority, use AI technology during their professional activities.

Specific Guidance for Procurement

Procurement and Legal teams should examine the tenets of the laws and regulations to understand current deployments and how (or if) they will impact decisions on which AI technology providers to use. Along with future deployments, companies also need to analyze current deployments. Such analysis includes:

• Inventory and Categorize Current AI Deployments

  Inventory current deployments and categorize their risk. The EU has provided a Compliance Checker for businesses to determine compliance with the Act.

• Conduct a Gap Analysis

  Use the inventory to ensure systems are traceable and monitored. Conduct a gap analysis to determine where compliance gaps may exist.

• Review and Update Practices

  Review and update data privacy and security practices as needed to conform to the Act.

• Train Employees

  Train employees on the fundamental aspects of the Act.

• Negotiate with Vendors

  Include a discussion about the Act in all current negotiations with software vendors.

Procurement and Legal should consider the insertion of clauses and terms in licensing contracts which stipulate adherence and conformity to the Act. Here is a sample structure of contractual clauses:

1. EU AI Act Compliance Representation

   Establish representations and warranties that any AI system provided under the contract complies with the EU AI Act’s requirements, including but not limited to transparency, accountability, and safety standards.

2. Indemnification and Liability

   Obtain assurances that the vendor agrees to indemnify and hold the client harmless from any claims, damages, or penalties arising from non-compliance with the EU AI Act by the vendor’s AI system. In the event of non-compliance causing harm to the client or third parties, the vendor acknowledges its liability as per the EU AI Act’s provisions.

3. Ongoing Compliance Obligations

   Ensure that the vendor commits to monitoring changes in the EU AI Act and promptly updating the client if any modifications impact the AI system’s compliance status.

4. Termination Clause:

   Ensure that if the vendor fails to rectify any EU AI Act non-compliance within a reasonable period after receiving notice from the client, the latter has the right to terminate the contract without penalty.

Conclusion