Enterprises scaling agentic AI face rising operational costs and shrinking security windows, according to Brian Gracely, senior director of portfolio strategy at Red Hat. To maintain efficiency, companies are shifting from monolithic models to “right-sized” models using semantic routing and accelerating patch cycles to counter AI-driven vulnerability discovery.
The Financial Shift from Chatbots to Agentic AI
Agentic AI usage is orders of magnitude higher than during the chatbot era, moving cost management from an engineering concern to a recurring boardroom discussion. According to Gracely, who presented these findings at VentureBeat’s AI Impact event, many enterprises are realizing a dependency on a small number of model providers.
Gracely noted that top providers are already telling the market that they’re losing money and are trying to go public to make up those gaps. This creates a risk for enterprises that may eventually buy at a very high-cost level or lack of control over their infrastructure. To mitigate this, organizations are exploring alternatives to maintain control over costs and infrastructure.
Reducing Costs Through Semantic Routing and Model Right-Sizing
The fastest lever for cutting agent costs is “right-sizing,” or matching the model’s capability to the specific complexity of the task. Gracely argued that companies often overspend by defaulting to the most capable model available for simple tasks, such as resolving an insurance claim, which doesn’t require a model to know about the history of Western civilization or World Cup soccer scores.
Enterprises are implementing two primary technical solutions to solve this:
- Semantic Routing: This mechanism automatically classifies requests and sends each to a model sized for the task without requiring users to choose.
- Caching: By caching repetitive queries, companies cut how often a request needs to reach GPU compute, lowering overall token spend.
Gracely compared the current need for token discipline to the FinOps practices that took years to mature in order to take control of cloud compute spending. Just as financial teams once had to learn what an EC2 instance and an S3 bucket are, they must now understand tokens to avoid paying for “Rolls-Royce” or “caviar” capabilities when they are trying to do basic types of things.
AI-Driven Vulnerabilities and the 14-Day Patch Window
AI is accelerating the speed at which vulnerabilities are discovered, forcing enterprises to rethink how quickly they can identify, validate and deploy patches. Gracely stated that the window for companies to stay ahead is somewhere between seven and 14 days.
The threat is no longer just isolated critical flaws. AI security tools can identify combinations of seemingly minor vulnerabilities that become dangerous only when chained together. Because AI can uncover these combinations faster than human teams can traditionally patch them, the ability to rapidly manage and update software is becoming a strategic capability rather than simply an operational one.
Solving Organizational Friction for AI Adoption
The transition from AI pilots to production scale depends on the deep, sustained involvement from subject matter experts (SMEs) and compliance teams. Because agents are designed to encode the knowledge of these experts, earning their buy-in is a prerequisite.
Gracely emphasized that organizational friction often stems from fear that the technology is going to take away their job. He argued that companies must think about incentives so that SMEs cooperate with innovation. Without a strategy to incentivize people in the long run, agent adoption typically stalls among early champions and fails to spread across the wider enterprise.
| Challenge | Traditional AI Approach | Agentic AI Scaling Strategy |
|---|---|---|
| Model Selection | Default to most capable model | Semantic routing to right-sized models |
| Cost Management | Engineering concern | Boardroom-level FinOps for tokens |
| Security | Long-established patch cycles | Rapid 7-14 day response windows |
| Adoption | Afterthought buy-in | SME-led encoding and incentive alignment |
As enterprises move past the initial anxiety of “falling behind,” the focus is shifting toward the disciplined application of infrastructure and human management. The success of agentic AI will likely be determined by those who treat model efficiency and patch speed as core business metrics rather than secondary IT concerns.