The first year of EU AI Act transparency enforcement is taking shape, and organizations are bracing for a period of uncertainty. Article 50 of the regulation introduces transparency obligations for AI systems that interact with natural persons or generate synthetic content, with penalties reaching up to 15 million euros or three percent of worldwide turnover. But according to Edwin Weijdema, Field CTO at Veeam, the real-world impact of the first twelve months may look very different from the worst-case scenarios circulating in boardrooms.
First-year enforcement: fines vs corrective orders
Weijdema points to the enforcement patterns of other EU regulations, such as NIS2 and GDPR, as a guide. Member countries are responsible for enforcing the AI Act at the national level, and their procedures vary significantly. That makes precise predictions difficult, but the first year may function as a bedding-in period. Rather than imposing heavy financial penalties, regulators are more likely to issue corrective orders requiring organizations to change, suspend, relabel, or withdraw non-compliant AI-enabled processes.
Several factors will shape these decisions. Regulators are expected to weigh proportionality, the scale of impact, whether a breach was intentional or negligent, how quickly the organization cooperated, and whether basic governance controls existed in the first place. Organizations that demonstrate genuine effort to comply may receive more lenient treatment. Even so, the broader risk may be operational rather than financial. An order to halt an AI system until compliance is proven can be far more disruptive than a fine, especially if that system is embedded in a critical workflow or customer-facing service.
When does indirect interaction count as direct?
One of the murkiest areas of Article 50 is its application to agentic AI systems that interact with people indirectly. A ticketing queue, shared inbox, or supplier procurement portal does not automatically qualify as direct interaction, but it can. The decisive factor, Weijdema explains, is whether the AI system itself is communicating with a natural person or whether a human intermediary exercises meaningful review and control.
Under the EU AI Act, the transparency obligation applies when a person is interacting with an AI system and needs to be informed that they are dealing with AI, unless the circumstances make it obvious. If an AI drafts a response and a human reviews and sends it, the risk profile is very different from an AI agent autonomously replying to a customer, supplier, or employee. The latter can start to look like direct interaction, even if the communication occurs through a ticketing system or procurement portal rather than a chatbot window. The AI Act does not care whether the interaction happens in a chatbot window or a ticket queue. It cares whether the human is effectively dealing with the machine.
Weijdema advises companies to make deliberate choices about separating internal agents from customer-facing agents, using appropriate access barriers, privacy controls, and role-based permissions. He draws an analogy: telling an agent not to enter a room is not enough; organizations also need to put a lock on the door. The same access and privacy controls should exist across the entire organization, not just across AI agents.
Simulated phishing and cloned voices
Security teams face a particular challenge when using AI for simulated phishing and vishing exercises. These tests often rely on realistic materials, sometimes including cloned voices of executives or generated phishing emails. The problem is that labeling the material as AI-generated defeats the purpose. Weijdema warns that such exercises are not automatically exempt from the AI Act's transparency requirements, and organizations should not assume they are.
Cloning an executive's voice is especially sensitive. If AI is used to make a real person appear to say something they did not say, the situation can quickly become a deepfake scenario. A security purpose does not automatically create an exemption, and the argument that the exercise works better without disclosure is not, by itself, a compliance justification. Organizations that decide not to label AI-generated elements in security exercises should be able to demonstrate that the legal basis and risk have been carefully assessed.
Weijdema recommends involving legal and compliance departments early in the process, documenting the reasoning behind any decision to withhold disclosure. Privacy, human resources, and employee representative input may also be necessary, especially if the exercise uses a real person's voice, image, or likeness. In most cases, he advises considering alternatives such as fictional personas, synthetic voices that do not imitate real employees, prior general notice that simulations may use synthetic media, and immediate post-exercise disclosure. Documentation should cover the purpose of the exercise, scope, AI tools used, whether any real person was imitated, what disclosure was provided and when, what personal data was processed, why the approach was necessary and proportionate, what safeguards were in place, and how employees were debriefed afterward.
His advice to security teams is direct: a security objective does not magically turn an undisclosed deepfake into a compliant one. If the test requires cloning the CEO's voice, legal should be in the room before anyone presses send.
Where will the first Article 50 action originate?
The enforcement infrastructure for the EU AI Act is still incomplete. As of mid-June, only nine of the twenty-seven member states had designated both a market surveillance authority and a notifying authority. Twelve had partial designations, and six had neither. This uneven readiness creates uncertainty about where the first enforcement action will come from.
Weijdema believes the first Article 50 action is formally most likely to originate from a market surveillance authority, since that is where enforcement responsibility sits at the national level. But the practical trigger may come from elsewhere. Consumer groups are likely candidates for early challenges, especially for AI systems that affect or interact with large numbers of people. Competitors, employees, journalists, civil society organizations, or affected individuals could also file complaints that prompt regulator-led investigations.
Defamation claims are possible, particularly where synthetic audio or video damages someone's reputation, but such claims are more likely to run in parallel with enforcement than to serve as the first clean Article 50 case. Weijdema expects the first case to be regulator-led on paper but very possibly complaint-led in reality. That distinction matters for organizations trying to anticipate risk, since complaints can come from any direction and may force regulators to act faster than they otherwise would.
The accountability question no one can answer yet
Clients are increasingly asking a question that has no good answer yet: How do we prove what an AI agent did, why it did it, and who was accountable? In cybersecurity and governance, risk management, and compliance, evidence matters. Logs, approvals, identities, access controls, retention policies, and audit trails are all essential. But agentic AI can reason, retrieve data, generate content, and take actions across multiple systems, which means governance must move from policy documents into technical controls.
Weijdema advises treating AI agents like privileged digital identities. Each agent should have an owner, a defined role, least-privilege access, monitoring, approval gates, and a kill switch. Organizations that adopt this approach early will be better positioned for both compliance and resilience. The challenge is that existing accountability frameworks were designed for human actors and deterministic software, not for autonomous systems that can act across interconnected platforms.
Another persistent question involves the boundary between transparency and security testing. Security teams need realistic simulations, but the AI Act pushes organizations toward disclosure when people interact with AI or are exposed to deepfakes. The hard part is designing exercises that remain realistic without crossing legal, ethical, or employee trust boundaries. Security teams want realism, regulators want transparency, and the challenge is designing exercises that satisfy both.
Other unresolved questions remain. Who is ultimately accountable when an AI system causes harm: the vendor, the deployer, the business owner, or the executive team? How do organizations prove to regulators, customers, and the board that AI governance is working in practice, not just documented in policy? And how much business value are organizations willing to lose in order to stay compliant, transparent, and auditable when using AI at scale? These questions are not new, but the EU AI Act forces them into the open. The first year of enforcement will likely be less about dramatic fines and more about learning how to answer those questions convincingly, before a regulator or a complaint demands it.
Source: Help Net Security News