See exactly which cases each model nails.
A green ✓ / red ✗ matrix across every case in your suite, with per-model accuracy and per-case pass rates — so regressions and blind spots jump out at a glance.
Hosted dashboard for EvalCore
Turn your EvalCore runs into a living leaderboard.
eval-forge ingests your EvalCore run reports the moment they finish — then turns them into a ranked leaderboard, a model × case heatmap, full agent transcripts, and trend sparklines. Share any of it with one public link.
| # | Model | Backend | Accuracy | Pass/Total | p50 ms | p95 ms | Mean tokens | #runs |
|---|---|---|---|---|---|---|---|---|
| 1 | gpt-4o-mini | remote | 24/25 | 410 | 980 | 142 | 12 | |
| 2 | claude-haiku | remote | 23/25 | 530 | 1180 | 168 | 9 | |
| 3 | qwen2.5-7b | local | 21/25 | 220 | 640 | 205 | 18 | |
| 4 | llama-3.1-8b | local | 18/25 | 280 | 760 | 233 | 14 | |
| 5 | mistral-7b | local | 15/25 | 195 | 520 | 256 | 7 |
Compare any model or backend, side by side.
qwen2.5-7bgpt-4o-minillama-3.1-8bclaude-haikumistral-7blocalremote
Latest · Best · Allleaderboard views
p50 + p95latency percentiles
Fullagent transcripts
One linkpublic sharing
Model × case heatmap
| Model | arith_addition_trivial | arith_subtraction_negative | lang_factual_contains | lang_yes_no_regex | tool_search_called | tool_email_with_args | Accuracy |
|---|---|---|---|---|---|---|---|
| gpt-4o-mini | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 100% |
| claude-haiku | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | 83% |
| qwen2.5-7b | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | 67% |
| llama-3.1-8b | ✓ | ✗ | ✓ | ✓ | ✗ | ✗ | 50% |
| mistral-7b | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | 33% |
| Pass rate | 100% | 60% | 100% | 60% | 60% | 20% | 67% |
Trends
Track accuracy & latency over time.
Every persisted run is a point on the curve. Inline sparklines show how a model's accuracy climbs and its p50 latency drops as you iterate on prompts and tools.
accuracy88%
p50 latency280ms
Transcripts
Replay exactly what the agent did.
Open any case to read the full role-tagged transcript — user, assistant, and tool turns, including the exact tool calls and arguments — next to a PASS / FAIL verdict.
PASStool_email_with_args
user
Email Dana the Q2 numbers and CC finance@acme.io.
assistant
Sending the Q2 summary now.
send_email({"to":"dana@acme.io","cc":"finance@acme.io","subject":"Q2 numbers"}) tool
{"status":"sent","id":"msg_8f21"} assistant
Done — the Q2 numbers are on their way to Dana, finance CC’d.
Built for the loop you already run.
Zero-config upload
Flip on the EvalCore upload feature and every run lands here automatically.
Public share links
Publish a read-only dashboard with one URL — no account required to view.
Latest / Best / All
Switch the leaderboard between the newest, best, and every run per model.
p50 & p95 latency
Nearest-rank percentiles per run, heat-colored so slow models stand out.
Token usage tracking
Mean tokens per case alongside accuracy — see cost and quality together.
Filter & sort runs
Sort any column and filter by model to focus on what you care about.
Live on eval-forge
Explore all →Recent public projects
Get started
Add three lines to your EvalCore run.
Enable the upload feature, drop your forge token in once via the environment, and call .upload_to_forge() on your run metadata. The next run POSTs its RunRecord straight to your dashboard — no extra plumbing.
# Cargo.toml eval-core = { version = "0.2", features = ["upload"] } // drop in your API key once via env // EVAL_FORGE_TOKEN=sk-eval-… // EVAL_FORGE_PROJECT=eval_… let meta = RunMeta::builder() .persist_to("results", "qwen2.5-7b") .upload_to_forge() // POSTs the RunRecord JSON .build(); run_eval_with_meta(&agent, &cases, meta).await?;
Ready to see your runs in one place?
Point EvalCore at eval-forge and your next run shows up as a live, shareable dashboard.