Enterprise AI agents: governance, cost discipline, and security next to scale

Scaling enterprise AI agents from pilot to production requires more than model selection. According to Red Hat senior director of portfolio strategy Brian Gracely, the real barriers are cost discipline, security awareness, and organizational buy-in. Speaking at VentureBeat's AI Impact event, Gracely outlined how agent usage is already outpacing chatbot-era expectations, driving rising token costs and dependency on a few model providers.

What happened

Agentic AI usage is orders of magnitude higher than during the chatbot era, making AI costs a growing concern for enterprises. Organizations are becoming increasingly aware of their dependence on a small number of model providers. Gracely noted that the two or three top providers are losing money and may raise prices, motivating enterprises to explore alternatives for cost control. The biggest cost issue is defaulting to the most capable model regardless of task complexity.

Semantic routing and caching are mechanisms many companies use to automatically classify requests and send each to a model sized for the task, reducing GPU compute needs. Gracely compared the financial discipline needed for token spend to FinOps practices that matured for cloud compute. He emphasized educating financial teams about tokens and model selection.

On security, AI-powered vulnerability discovery is forcing faster patch cycles, with a window of about 7 to 14 days to stay ahead. AI tools can identify not just individual flaws but also dangerous vulnerability chains. Finally, organizational adoption requires sustained involvement from subject matter experts whose knowledge the agent encodes, with incentives to ensure long-term cooperation.

Why AI builders should care

For AI builders and product teams, the cost and security practices around agent deployment determine whether a project leaves pilot status. Investing in governance, model-agnostic infrastructure, and explainable token spend helps avoid overpaying for tasks that do not need the most capable model. Organizational buy-in from domain SMEs is critical to embedding institutional knowledge into agents and ensuring adoption beyond pilots.

Practical implications

Implement semantic routing to send requests to appropriately sized models for the task, reducing unnecessary GPU usage. Cache repetitive queries to cut compute load and speed up response times. Educate financial and technical teams about tokens and model costs to prevent defaulting to the most expensive model. Adapt patch management processes to faster AI-driven vulnerability discovery, aiming for shorter embargo windows while maintaining safety.

Caveats

All analysis reflects a VentureBeat article summary and may not cover every enterprise scenario or all vendor claims. Specific deployment outcomes, timelines, or pricing details are not provided in the source excerpts. The article discusses governance and production risk concepts rather than a universal framework applicable to all enterprise contexts.

FAQs

What is the role of FinOps in enterprise AI agents?

FinOps-like governance helps teams manage token spend and cloud compute costs in AI agent deployments. It requires educating financial teams about tokens and model costs, similar to how cloud FinOps taught them about EC2 instances and S3 buckets. This enables task-appropriate model sizing rather than defaulting to the most capable model, reducing unnecessary spending. Red Hat's Brian Gracely emphasized this comparison at VentureBeat's AI Impact event.

How can enterprises control token spend when using AI agents?

Enterprises can control token spend by implementing semantic routing to classify requests and route them to model sizes appropriate for the task, and by using caching to reduce repeated GPU compute. Educating finance and product teams to understand token spend and avoid always selecting the top-tier model is also essential. Gracely noted that defaulting to the most capable model regardless of task complexity is the biggest cost issue.

What governance practices reduce security and production risk in AI agents?

Governance practices that reduce security and production risk include integrating production risk management with SME involvement to encode institutional knowledge into agents, and ensuring organizational buy-in through incentives for domain experts. AI-powered vulnerability discovery also necessitates faster patching cycles, with a window of about 7 to 14 days to stay ahead of attackers. Gracely highlighted that AI tools can identify both individual flaws and dangerous vulnerability chains.

Why is organization buy-in important for scaling AI agents beyond pilots?

Sustained adoption requires incentives and buy-in from domain experts who know the institutional context. Without SME participation, agent deployments risk stagnation after pilot phases. Gracely emphasized that you have to think about incentives so people don't feel threatened and cooperate with innovation long-term.