OpenEnv

// Overview

OpenEnv is an end-to-end framework for creating, deploying and using isolated execution environments for agentic reinforcement-learning training. It sits between the harness, the environment and the trainer as an interoperability layer: environments are packaged as Docker containers behind a small server, and clients talk to them over HTTP and WebSocket using the Gymnasium-style calls RL practitioners already know — `reset()`, `step()` and `state()`. MCP is a first-class transport, and the project is deliberately neutral about rewards, leaving those to whichever library you already train with.

The value is on both sides of that interface. Trainer authors get one API instead of one integration per environment, and can drive an environment running anywhere — locally in Docker or as a Hugging Face Space. Environment authors get tooling that handles isolation, packaging and deployment, plus an `openenv` CLI that scaffolds a new environment and pushes it to Spaces. Clients come in both async and sync flavours, so the same environment fits an asyncio training loop or a plain script.

The project is BSD-3-Clause licensed, installs from PyPI as `openenv`, and its README carries an explicit early-development warning: expect bugs, incomplete features and APIs that may change. Direction is set through public RFCs on the repository — baseline API specs, tool discoverability for agents, MCP support, delayed rewards for trajectory-based scoring, agentic-harness integration — and in June 2026 governance moved from a Meta and Hugging Face project to a multi-organisation steering committee, with the PyTorch Foundation, vLLM, SkyRL, Lightning AI and Stanford's Scaling Intelligence Lab among the supporting organisations.

// What it does

• Gymnasium-style API — reset(), step(), state() — over HTTP and WebSocket, so a trainer needs one integration rather than one per environment
• Environments run as isolated Docker containers behind a FastAPI server; deploy locally or as a Hugging Face Space
• Async and sync clients from the same class, via an async context manager or the `.sync()` wrapper
• `openenv` CLI scaffolds a new environment and deploys it to Hugging Face Spaces
• MCP support as a first-class citizen, plus RFCs covering tool discoverability and delayed trajectory-level rewards
• Reward-agnostic — define rewards in whatever RL library you already use, including torchforge, vLLM-based stacks and SkyRL

// Getting started

Install the core package, add an environment client, then drive the environment with reset/step.

pip install openenv
pip install git+https://huggingface.co/spaces/openenv/echo_env

import asyncio
from echo_env import CallToolAction, EchoEnv

async def main():
    async with EchoEnv(base_url="https://openenv-echo-env.hf.space") as client:
        result = await client.reset()
        print(result.observation.echoed_message)

        result = await client.step(
            CallToolAction(
                tool_name="echo_message",
                arguments={"message": "Hello, World!"},
            )
        )
        print(result.observation.result)
        print(result.reward)

asyncio.run(main())

from echo_env import CallToolAction, EchoEnv

with EchoEnv(base_url="https://openenv-echo-env.hf.space").sync() as client:
    result = client.reset()
    result = client.step(
        CallToolAction(
            tool_name="echo_message",
            arguments={"message": "Hello, World!"},
        )
    )
    print(result.observation.result)

openenv --help

git clone https://github.com/huggingface/OpenEnv
cd OpenEnv/examples/grpo_blackjack

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Train an agent with RL against a task environment without writing a bespoke integration for each trainer
• Publish an environment your team or the community can run, isolated in Docker and hosted on a Hugging Face Space
• Evaluate the same policy across several environments through one client API
• Expose environment tools to an agent over MCP rather than a custom protocol

// How OpenEnv compares

OpenEnv alongside other open-source rlhf & alignment tools AI/TLDR tracks, ranked by GitHub stars.