A year may seem like a long time in enterprise technology, but in artificial intelligence, the past 12 months have completely rewritten the enterprise architecture playbook. Agentic AI—systems that can pursue complex objectives by breaking them into sub-tasks and using tools autonomously—is no longer a novelty. It is forcing IT leaders to rethink infrastructure, data management, and operational costs from the ground up.

Speaking at the Dell Technologies World conference in Las Vegas, Dell’s global chief technology officer, John Roese, outlined the paradigm shift. “We have shifted our assumption that the use of AI is no longer a one-shot task like a chatbot,” he said. “It’s about handing objectives to the AI system, and that’s what agents do today.” As an example, he pointed to Google’s redesigned search engine, which uses multiple agents to search, compile, and build a results page based on a single objective.

The implication is profound: enterprises are tearing up old generative AI use cases and rebuilding them as agentic workflows. The user experience improves dramatically—humans become instructors rather than doers—but the underlying architecture must adapt. This article explores the key dimensions of that transformation, from hardware requirements to data strategies, cost management, and the human dynamics that ultimately determine success.

Busting the GPU training myth

During the initial AI boom, enterprises rushed to secure thousands of GPUs for model training. Roese argues that this is largely a myth for most businesses. “The myth out there is that enterprises need thousands of GPUs,” he said. “Our biggest workload inside of Dell only sits on 16 GPUs and supports 40,000 people. You don’t need thousands of GPUs in an enterprise, because for each workload, agent or project, you only need a handful of GPUs, sometimes half a GPU.”

The reason is that enterprise AI estates are shifting from training to inference. Agents do not require model training; they rely solely on inference—running pre-trained models to make predictions or decisions. That changes the hardware equation. When enterprises built simple chatbots, the architecture placed a very light load on CPUs. But agentic AI is different. Agents use external tools, communication protocols, knowledge graphs, and other components that do not naturally live inside a GPU.

“When you move to agentic, it’s almost balanced,” Roese explained. “The number of CPUs and GPUs are very similar—about maybe for every two GPUs you have a CPU. You don’t just build an AI infrastructure with a pile of GPUs—you build it with GPUs and traditional CPU compute.” This balance is a critical insight for IT planners who might otherwise overinvest in GPU capacity while neglecting the supporting compute that agents rely on.

Air-gapped frontier models and the edge

Another major shift is in how powerful AI models are deployed. A year ago, the most capable frontier models were only accessible through cloud APIs. Today, hyperscalers such as Google, AWS, and Microsoft allow top-tier models to run on-premise through services like Google Distributed Cloud. Roese noted that this opens up new deployment topologies. “You can consume it in a virtual private cloud or your datacentre, and you can air-gap it from everything else,” he said. “We didn’t have any of those options, except the API one, a year ago.”

At the same time, AI is moving to the edge in a structured way. Agentic frameworks like OpenClaw that run natively on devices and AI PCs have emerged. “Those have finally put some structure around running agents on devices, and that’s incredibly powerful, and not a fad,” Roese said. This means enterprises can deploy AI agents in remote locations, retail stores, manufacturing floor, or even on employee laptops, without relying on constant cloud connectivity. The edge deployment reduces latency, enhances privacy, and can lower bandwidth costs.

Re-architecting the data layer

Data strategies are evolving in tandem with agentic AI. Roese warned that bolting standard data storage systems onto AI compute clusters is no longer sufficient. Agentic AI demands high-speed access to context—vector databases, graph databases, and annotation tools—that must be deeply integrated into compute. “One of the performance bottlenecks is you can’t get data fast enough to the GPUs to do the work,” he said, adding that “the GPUs you’re paying for are sitting idle, waiting for data.”

To reduce latency, Dell’s AI data platform is now plumbed directly into Nvidia’s Cuda-X interfaces, effectively running data layer services at GPU speed. This kind of tight integration is becoming essential as agentic AI amplifies the need for real-time knowledge retrieval. For example, a customer service agent might need to pull up a knowledge graph of product specifications while simultaneously analyzing a user’s sentiment–all in sub-second time. Without a re-architected data layer, the agent stalls and the user experience suffers.

Organisations are also adopting data annotation pipelines that feed structured and unstructured data into vector databases. These annotations help agents understand context and make better decisions. The data layer is no longer a passive storage repository; it is an active participant in the inference process.

Mastering tokenomics and model routing

With more model deployment options come different pricing mechanisms. IT leaders must manage the cost of AI consumption, even as the cost per token is expected to decline. Roese insists that “there’s no path where it becomes cheaper to do AI”—meaning enterprises must treat AI workloads as an arbitrage game. Tokenomics, the study of token costs and usage, becomes a critical discipline.

Using specification-driven development as an example, Roese noted that if an agentic framework spawns dozens of coding tasks and blindly sends them all to top-tier frontier models, the bill will skyrocket. Instead, enterprises can implement model routing: complex planning tasks—such as creating software specifications—are sent to expensive frontier models, while routine coding tasks are routed to smaller, on-premise open-source models where energy is the only operational cost. “Building a piece of software and doing spec-driven development might have four or five different economic paths to ultimately get to the best overall economic efficiency,” said Roese.

Mastering model routing, he added, will be a competitive differentiator. Enterprises that can dynamically choose the most cost-effective model for each sub-task will achieve lower product development costs and faster iteration cycles. This requires a robust orchestration layer that can evaluate task complexity, model capability, and real-time pricing.

The human element

Ultimately, the hardest part of operationalising agentic AI relates to the human element. Roese described the traditional human job as a “container of work” that includes a mix of hygiene, productivity, coordination, and expert tasks. Agents cannot perform an entire job, but they are highly capable of executing specific types of work within that container. Dell audited 6,400 jobs across its own business to see how AI agents would impact its workforce.

“The first thing we realised is every single job in the company is going to change,” said Roese. “I’m taking work out of the job and removing stuff from the container. If the container is now only half full, do I need half the number of people, or do I expand that by half? Am I able to do more expert work?” These questions force organizations to think about job redesign, reskilling, and new roles.

The impact of AI on the workplace is so profound that change management has become a key remit of IT leadership. “For the last four months, I’ve spent 50% of my time dealing with human dynamics,” said Roese. “AI has ceased being a technology and an ROI discussion. It’s now very much an organisational and human dynamic discussion. You simply can’t use these things unless you fully understand how you’re going to adapt the human population around them.”

This means IT leaders must work closely with HR, business units, and employee representatives to design new workflows, offer training, and address anxiety. Agentic AI can remove tedious tasks, enabling workers to focus on higher-value analysis and creativity. But it also creates uncertainty. Successful adoption hinges on transparent communication and a clear vision of how AI will augment, not replace, human talent.

In summary, agentic AI is driving a comprehensive rethink of enterprise architecture. Hardware needs are shifting from GPU-heavy training clusters to balanced CPU-GPU inference systems. Data layers must be re-architected for speed and context. Tokenomics and model routing provide a way to manage costs. And the human element demands careful change management. Enterprises that adapt holistically will unlock the full potential of agentic AI, while those that treat it as just another technology rollout risk falling behind.

Source: ComputerWeekly.com News