One contract for every RL environment.
The Python SDK and CLI behind Nenyax. Load any format, drive it with any model or policy, judge it your way, and improve it with any learner.
RL environments are scattered. Your code shouldn't be.
Many SDKs
Gymnasium, OpenEnv, Verifiers, NeMo Gym, Harbor, reasoning-gym: each with its own API, and each trainer supports only a few.
One shape
Manifest, Capabilities, Trajectory and Judgment look the same whatever the source format.
One call
env.rollout(policy) works whether the environment steps, runs its own loop, or is a graded task.
Who drives the loop decides the mode.
Nenyax doesn't replace these frameworks or define another format. It routes each native format into one contract, and every format maps to exactly one mode.
- stepYou drive it with session() → act(). Gymnasium, OpenEnv, reasoning-gym, NeMo Gym, ORS, HF datasets.
- rolloutIts own evaluation loop drives your model. Verifiers, lm-evaluation-harness.
- taskA harness runs and grades it. Harbor, Karotte, AgentEnv, HUD.
Every format, one of three modesenv.rollout(policy) works in all three
Load, define, judge, train.
The same Endpoint drives every environment and returns the same Trajectory. Every model call is recorded, with token ids and logprobs when the server returns them.
Measured, not claimed.
nenyax check runs an environment and records the highest tier it actually reaches. A check that can't run is skipped, and a skip blocks its tier.
- ConnectableIt loads, the manifest is valid, and an episode can start.
- EvaluableA full episode completes with a finite judgment.
- TrainableA seed or stable task id recreates the situation, and the model path is recorded.
- AuditedThe verifier discriminates: a reference policy beats a null policy on every sampled task.
nenyax checkreasoning_gym:basic_arithmetic
Run end to end, not assumed.
Every row was run by the integration suite, a real-model experiment, or both.
| Mode | Highest tier | Notes | |
|---|---|---|---|
| reasoning_gymreasoning-gym 0.1.25 | step | audited | 106 procedural datasets, native verifiers |
| gymnasiumGymnasium 1.3 | step | trainable | no oracle exists, so no audit |
| openenvOpenEnv 0.6 | step | evaluable / trainable | evaluable on public Spaces, trainable on seeded servers |
| nemo_gymNeMo Gym 0.6 | step | audited | mcqa and reasoning_gym resources servers, over HTTP |
| verifiersVerifiers 0.3.1 | rollout | audited | legacy and v1 tasksets; Environments Hub gsm8k |
| harborHarbor 0.23 | task | audited | terminus-2 wraps any chat model; oracle/nop audit the tests |
Improve with any learner.
A learner is anything with policy() and update(groups). The built-ins cover in-context learning for closed APIs and weight updates for open models.
- incontextCurates worked examples from the model's own best episodes. No weight changes, so it works with closed APIs.
- grpo, dapo, rloo, gspo, rft, dpo and moreWeight updates for local open models on CUDA, MPS or CPU. Verified for correctness, still experimental at scale.
- tinker, or yoursHosted LoRA on Thinking Machines Tinker, or your own class by entry point.
Each round kept only if it improvesGRPO · held-out
Round 4 was worse, so it was reverted0.71
Take it anywhere.
Serve anything as OpenEnv. Any step-mode environment becomes a standard OpenEnv server (/ws, /reset, /step, /schema), so OpenEnv clients and the trainers built on them use it unchanged. The episode's judgment rides along in the final observation's metadata.
Export what you record. Trajectories export as JSONL, Parquet or Hugging Face datasets, as raw, SFT or DPO rows, and traces go out over OpenTelemetry's GenAI conventions to Langfuse, Phoenix, MLflow or Weights & Biases.
Seven kinds, one interface each.
Scaffold a plugin that passes its conformance test on day one, then use it by name in any config.
- Scaffoldnenyax new-plugin sandbox acme
- Checknenyax sandbox-check acme proves a backend delivers what it claims.
- UsePick it by name: use = "acme".
Plugin kindsone interface each
nenyax new-plugin sandbox acme
Everything from the terminal.
From install to a checked environment.
Install
Install the Nenyax CLI from your workspace.
Check
See what an environment really does.
nenyax check <uri>Try
One episode with your model, readable replay.
nenyax try <uri>Train
Pick a learner and a provider.
nenyax train <uri>Good to know.
Install the SDK.
Load your first environment, check its tier, and run it with your model.