EnvHarness: Google's open-source framework for adaptive AI training environments

Google Research has released EnvHarness, an open-source framework that wraps static AI agent training environments with a programmable layer, letting them adapt to each agent's weaknesses without modifying the underlying simulator. For teams building coding, web navigation, or office automation agents, this means getting more training value from a smaller set of high-quality environments instead of continuously building new ones from scratch.

What EnvHarness actually does

EnvHarness sits between the agent and the environment as a lightweight plugin. It can change where an agent starts, what it sees, which actions it can take, and how long a task lasts, while leaving the original environment and its verifier intact. The framework provides three components: a Stage alters the starting state (e.g., placing a mug inside a drawer instead of on a desk), a Contract modifies the interaction (e.g., filtering actions or hiding details), and a Chain joins tasks together to create longer trajectories. All modifications preserve the trusted verifier that determines task success.

EnvRigger: automated adaptation

The other key piece is EnvRigger, which automates the process of deciding how to modify the environment. It follows an Observe, Diagnose, Write, Validate loop. First it runs the agent several times and examines trajectories for recurring failure patterns. Then it composes EnvHarness components designed to expose or correct those failures, and runs fresh trajectories to check that the changes create a useful and solvable training example. For instance, if a coding agent submits a patch without running tests, EnvRigger can create a Contract that intercepts premature submissions and forces the agent to run the test suite first.

Benchmark results

The researchers tested EnvHarness on ALFWorld, WebArena, SWE-bench Verified, OfficeQA, and SpreadsheetBench. Agents trained with EnvHarness environments improved by up to 9 points on held-out tasks compared to training on the unchanged environments. On SWE-bench Verified, the average trajectory shortened from 55.01 to 49.61 steps. EnvHarness also outperformed environment generation systems like SWE-smith and GenEnv on the same benchmarks. The scaling experiments showed that as the EnvHarness training pool grew to 300 environments, the agent continued improving, while learning from the same number of original environments plateaued earlier.

Integration and costs

To use EnvHarness, teams need to write a Bridge that exposes their environment through the framework's common ActionableEnv interface. For containerized workflows, Wang says EnvHarness can attach as a lightweight outer plugin on top of existing test runners, keeping the container image and unit tests intact. The main cost is compute: the EnvRigger loop requires multiple agent rollouts to diagnose weaknesses and validate modifications. This makes EnvHarness best suited for digital sandboxes where rollouts are cheap and state can be restored quickly, such as coding environments, web automation test systems, and tool-use simulations. It should not be run directly against live production databases or physical robots.

What it means for builders

EnvHarness does not train the agent itself. It creates experiences that another learning mechanism has to consume, such as skill extraction, fine-tuning, or reinforcement learning. The framework is complementary to agent-side optimization tools like Self-Harness or DarwinX. By dynamically reshaping the environment around the agent's current weaknesses, EnvHarness offers a practical path to reduce development overhead and get more coverage from each environment. The code is available on GitHub under Apache 2.0.