Enterprise AI agents outrun legacy infrastructure: how three giants fixed the bottleneck

At VB Transform 2026, infrastructure leaders from LinkedIn, Walmart, and Zendesk shared a common finding: legacy infrastructure, not model limitations, is the primary bottleneck slowing enterprise AI agents in production. Each company hit a different wall, but all three concluded that infrastructure built for human workflows cannot keep up with agents that think in milliseconds.

What happened

LinkedIn's first bottleneck was Kubernetes. Containers spinning up on demand took seconds, too slow for agent workloads. The fix was moving from on-demand provisioning to pre-provisioned pools of containers that swap agentic workloads in real time. A second problem surfaced when agents controlled their own orchestration: an LLM evaluating another LLM's output shared the same failure mode. LinkedIn responded by building its own harness, pushing LLMs to the leaf of the workflow. Roughly 80% of the workflow is now scripted, deterministic code, with LLMs used only where reasoning is required.

LinkedIn also built two subsystems for provider independence. The first is an AI gateway, a single interface that every outbound model call runs through regardless of provider. The second is a memory subsystem that holds context independent of any model provider, allowing quick switching between models.

Walmart's bottleneck came from success. An agent harness put into employees' hands went viral internally, and what the company calls citizen developers began building their own agents. The upside was innovation, but the downside was duplication: dozens of overlapping agents with no coordination. The fix was building governance to spot duplication, promote the best version of an agent, and get it into production without engineering becoming a chokepoint. Walmart also built its own internal gateway to stay vendor agnostic across three workload types: fully deterministic workflows, planner-and-reasoner workflows, and a hybrid of the two.

Zendesk hit its bottleneck from the data side. Sami Ghoche, who joined through Zendesk's acquisition of Forethought, described sitting on roughly 20 billion customer conversations. The instinct is to hand that history to a large language model with a big context window, but Ghoche said that doesn't work. Instead, Zendesk invested heavily in underlying data pipelines and data infrastructure to provide reliable context without overloading the model.

Why AI builders should care

The panel's shared conclusion is that the bottlenecks enterprise teams face are infrastructure-centric, not model-centric. For AI builders, this means that investing in governance, evaluation, and provider-agnostic integration layers will yield more impact than chasing the latest frontier model. The choice between a frontier model and an open-weight model should come down to workload effectiveness, not a fixed policy.

Practical implications

Three pieces of advice emerged directly from the panel:

Invest in evals before anything else. Ghoche called evals the thing common to every use case, internal or customer facing. "It'll force you to break the problem down, and once you have a robust set of evals, you can move a lot faster," he said.

Own your agent harness from day one. Gosby's advice was to put the AI harness directly in employees' hands early, paired with the infrastructure to monitor what it produces. "It will unlock a huge amount of innovation," she said.

Build for model and context independence. Singh emphasized building for independence so that enterprise context can be reused when switching models or harnesses. "Keep that context within your enterprise so that you can reuse it when you ship the model or the harness tomorrow," he said.

Caveats

The claims are based on a three-company panel at VB Transform 2026. Integration experiences may vary by organization size, data sensitivity, and existing tech debt. The panelists represented large enterprises with significant engineering resources; smaller teams may face different trade-offs. The advice around governance and evaluation is broadly applicable, but specific infrastructure choices (pre-provisioned containers, custom gateways) may not be necessary for all production AI agent deployments.