Author your own benchmarks
Questions, prompt, sampling, scorer: versioned and re-runnable.
Benchmark any OpenAI-compatible model on your own tasks. Rackor shows you where a smaller, cheaper model holds up, with full transcripts and scores to back it.
Questions, prompt, sampling, scorer: versioned and re-runnable.
Every prompt, response, and score. See exactly why it passed or failed.
Models side by side, question by question. Find where cheap holds up.
Point at a URL and key, local or hosted. Nothing to rewrite.
No SDK, no data export. Connect, benchmark, decide.
Add any OpenAI-compatible URL and key. Test it in one click.
Run a published version, or write and version your own.
Benchmark one or many. Track pass rates and compare head-to-head.
Billed in credits: 1 completed run = 1 credit; an AI-generated question = 3.
Start evaluating models, no card required.
A monthly credit pool for steady, solo evaluation.
Private benchmarks and runs, for teams.
Negotiated volume and terms.
Anything that speaks the OpenAI chat-completions API: llama.cpp, vLLM, Ollama, OpenRouter, OpenAI, and most hosted providers. You add a base URL and a key, and Rackor runs against it. No SDK or code changes.
Start free, no card required. Connect an endpoint and run your first benchmark in minutes.