Guardrails amidst the chaos
The world of AI – perhaps especially in the contact centre and customer experience space – is clouded with exaggerated claims, disputed evidence and unqualified ‘experts’. Everyone talks about guardrails, but there’s no settled agreement as to what is and isn’t reasonable or acceptable. So legal regulations and requirements would presumably be helpful in defining some foundational guardrails? But at a federal level the US government is active opposed to any AI-specific regulation and in the UK, while the government points to its ‘sector led’ approach, there are no plans for an overarching AI law.
However, it’s very different in the EU where the AI Act has been law since 2024 and will be fully implemented from this August. The Act is often described as “the world’s first comprehensive AI legislation”.
In which case, if you’re based in the UK, North America, Africa or Asia you may well be thinking “why should I care? That doesn’t affect me”.
If so, you’d be wrong; very wrong.
Why the EU AI act matters to you
No EU presence? It doesn’t matter. If you have EU customers you will need to comply with the Act’s requirements. And even if you don’t your EU suppliers will. And even if you don’t have EU suppliers or partners, it’s quite likely that in the global regulatory vacuum the EU AI Act will become a default standard, similarly to the way GDPR did for data protection.
While the odds of your organisation being prosecuted under the Act are low, bear in mind that fines are at GDPR levels. So, for the gravest transgressions you’d be looking at €35 million or 7% of global annual turnover.
What does the Act say?
Unsurprisingly, the Act’s quite lengthy, but there are some key highlights to get your head around:
The definition of AI:
”a machine-based system designed to operate with varying levels of autonomy, capable of adapting after deployment, and generating outputs such as predictions, recommendations, content, or decisions that can influence physical or virtual environments”
What’s ok and what’s not:
- Unacceptable Risk AI: Banned totally
- Social scoring, manipulative AI, and biometric categorisation based on sensitive traits are prohibited
⚠ Watch Out
Use of black-box AI for things like fraud prevention or dynamic pricing could put you at risk.
- High-Risk AI: Strict new controls in place
- Applies to recruitment, education, healthcare, credit scoring, policing, and safety-critical infrastructure
Requirements: Detailed risk assessments, transparency, human oversight, and conformity checks before launch
⚠ Watch Out
Don’t assume you’re exempt. Even seemingly innocuous recruitment screening tools could fall within the scope of these rules.
- General-Purpose & Generative AI: New obligations
- Foundation models (like ChatGPT or image generators) must ensure transparency, appropriate labelling AI-generated content, management of systemic risks, and clarification of the use of copyrighted data
- Limited-Risk AI: Transparency required
- Chatbots and similar tools must clearly inform users they’re interacting with AI
⚠ Watch out! Many voice bot providers currently advise clients to hide the fact that customers are interacting with machines. This will need to change – even as it becomes increasingly hard for customers to tell.
- Minimal-Risk AI: Largely unaffected by the Act
- Spam filters, video game AI, and similar tools are mostly out of scope of the Act
Who carries the liability?
As you might expect, the Act differentiates between AI developers (providers) and AI users (deployers).
- Developers (providers) are liable for ensuring that AI systems comply with the Act’s requirements, including safety, transparency, traceability, and respect for fundamental rights. They are specifically liable if harm results from software defects, cybersecurity vulnerabilities, or algorithmic discrimination.
- Users (deployers) are responsible for the legal operation of AI under their control. They need to ensure proper monitoring, human oversight, and adherence to transparency obligations are in place. Users will be held liable if harm occurs due to misuse, failure to supervise, or neglecting operational safeguards.
This means there is less scope for commercial partners to attempt to contractually ‘offload’ legal obligations onto their customers or suppliers than we often see in the realm of data protection. Added to which, when so many organisation and service providers are taking the opportunities to adapt and build upon AI foundation models, they may find themselves legally regarded more as developers than users.
What’s to be done?
What’s clear is that the use of AI in customer experience and contact centres in alignment with the EU AI Act isn’t a one-team or one-time task. Organisations need to truly understand where AI is being used, to achieve what and how. This isn’t just a job for the compliance team. Tech, data, proposition, digital, risk, pricing, finance, legal and customer experience colleagues all need to be involved. And to stay involved as AI solutions innately change and develop over time.
It’s a classic cross-functional business change project, but one that is likely to spur or reflect significant changes in business rules and structures – as well as needing to become embedded into ‘business as usual’ processes.
Need Help Navigating the EU AI Act?
At Customer Contact Panel, we help organisations find and successfully implement compliant, effective AI solutions, so you can innovate with confidence and accountability.
Drop us a line and we’d be happy to have a chat.
If you’re looking for a practical breakdown of what the EU AI Act requires and how to prepare, read our detailed guide:
EU AI Act Compliance: What Every Business Needs to Know and Do
