All videos
0:00 / 0:00
research

Training Agents 4: From reward functions to environments.

Hugging Face2 September 2026Watch on YouTube

Part of series

Ep. 5 · RL voor AI-Agenten

Praktische workshops over reinforcement learning en beloningsfuncties voor het trainen van agentic AI-systemen met open-source tools.

View the series

What you'll learn

  • You learn why verifiers stop working once an agent takes many actions, and why the reward has to live where the actions happen
  • The reset()/step() contract is explained, from CartPole to a coding agent with a shell, files, and tests
  • You see the anatomy of an RL environment with task, tools, backend, reward, and episode, plus the OpenEnv, Verifiers, OpenReward and Harbor ecosystem
  • A live demo shows how Qwen3-1.7B improves on MBPP and how a real coding agent (OpenCode) progresses on DeepCoder problems
  • You discover forms of reward hacking inside the environment and how to limit them

Frequently asked questions

What is the difference between a reward function and a reward environment?
In session 3 the model answered once and a Python function scored that answer. In this session the model runs a loop, such as opening a file, running a command, reading the error, and trying again. The training signal comes from what happens after acting, so the reward lives where the actions happen.
Which tools are used in this session to train agents?
The session demonstrates OpenEnv for building environments, TRL's GRPOTrainer and AsyncGRPOTrainer with environment_factory for training, and Harbor for a coding agent in simulated environments on Hugging Face sandboxes.
What does the OpenEnv CLI contract involve for environments?
The OpenEnv CLI contract includes commands such as init, push, pull, and fork for building and serving an environment. It lets you set up an environment with OpenEnv, push it to the Hub, and later pull or fork it again.
What results does a real coding agent achieve on DeepCoder problems?
In demo 2 a real coding agent (OpenCode) is trained on DeepCoder problems in HF sandboxes, with a reward increase from 0.27 to 0.71 in ten steps. In demo 1 the held-out pass rate for Qwen3-1.7B on MBPP rises from 0.49 to 0.59.

Topics

reinforcement-learningagent-trainingopenenvtrlharbor

Read next

Description from the channel

Training Agents, Session 4: from reward functions to environments. In this live session, the reward stops being a function and becomes a place the agent acts in. In Session 3 the model answered once and a Python function scored it. Today the model runs a loop: open a file, run a command, read the error, try again. The training signal comes from what happened after acting. We'll walk the reset()/step() contract from Gym to LLM agents, build an environment with OpenEnv, push it to the Hub, and plug it into TRL's GRPOTrainer with environment_factory. Then we go one step further and train a real coding agent harness through Harbor via OpenEnv, with AsyncGRPOTrainer on Hugging Face sandboxes. What we'll cover: - Why verifiers stop working once an agent takes many actions, and why the reward has to live where the actions happen - The reset()/step() contract: from CartPole to a coding agent with a shell, files, and tests Anatomy of an RL environment: task, tools, backend, reward, episode - The ecosystem: OpenEnv, Verifiers, OpenReward, Harbor, and how OpenEnv imports the others - Building and serving an env with the openenv CLI: init, push, pull, fork - Training with TRL: environment_factory and get_reward, no reward_funcs - Demo 1: Qwen3-1.7B on MBPP inside a live Python session, held-out pass rate 0.49 → 0.59 - Demo 2: a real coding agent (OpenCode) on DeepCoder problems in HF sandboxes, reward 0.27 → 0.71 in 10 steps - Reward hacking moves into the env: the try/except that never failed, the CVE fix read from .git history, and how to lock the box down