Any model on any benchmark.
A benchmark is a fixed set of tasks that measures every model the same way. Pick one from any hub, pick the models you want to compare, and get scores measured by the benchmark's own verifier on your own runner.
Built so a number means something.
Indexed from every hublistings
Prime Intellect1,777
Hugging Face452
OpenAI Evals432
lm-eval-harness208
reasoning-gym106Indexed from every hublistings
Prime Intellect1,777 Hugging Face452 OpenAI Evals432 lm-eval-harness208 reasoning-gym106Search every hub at once.
Filter by domain, hub and how often a benchmark has been measured. Each one shows its copies side by side, the one to use, and the best score so far.
- DomainsMath, reasoning, code, games, writing, tool use, web and browser, agentic terminal, science.
- Merged copiesWordle lives in 26 places, IFEval in 25, GSM8K in 17. You pick one benchmark, not one copy.
- Use it in one clickEvery benchmark has an Evaluate button; your runner does the rest.
open-thought
basic_arithmetic
Procedurally generated arithmetic with a built-in verifier.
Audited
harbor-framework
terminal-bench
Agents solving tasks in real terminal environments.
Not checked
primeintellect
wordle
Wordle game environment.
Not checked
Farama Foundation
CartPole-v1
The classic cart-pole control problem.
Not checked
More than one number.
Every evaluation keeps what produced the score, so you can trust it, compare it and come back to it.
GSM8K200 questions · seed 0
Qwen2.5-1.5B14.0% SmolLM3-3B9.5% Qwen2.5-0.5B5.0%Per model, on the same tasks
Model, revision, number of questions and seed are recorded with every score, so two runs really are comparable.
run #36 DriverUnavailablenot scored never a 0
Never scored as zero
A run that errors isn't scored, so it can't pass for a model that got everything wrong.
Advisoriesreviewed
Known problems, next to the score
Reward hacks, verifier bugs and contamination, reported and reviewed beside the benchmark.
Trajectory · model_calls[0]score 1.00
Every model call kept
Each run keeps its trajectory: every model call with token ids and logprobs when the upstream returns them, so you can see why a score is what it is.
What you'd have to stitch together.
| With Nenyax | By hand | |
|---|---|---|
| Find the right copy of a benchmarkcopies merged across hubs | ||
| Same harness for every formatVerifiers, Harbor, OpenEnv, NeMo Gym … | ||
| Errored runs kept out of the score | ||
| Every model call recordedtoken ids and logprobs when available | ||
| Runs on your own machines | ||
| Results compared and ranked |
Two commands to a score.
An evaluation project: one benchmark, one model, your runners.
Good to know.
Put a score on the board.
Connect a runner, pick a benchmark and a model, measure a baseline.
nenyax project run my-eval baseline --watchscored by its own verifier