You can now benchmark different AI coding agents against each other to see which performs best for your codebase and coding style. This means less wasted time on poor AI implementations and a faster path to high-quality code.
You can enable Benchmarking for any project from the Project Settings page.
To create a benchmark ticket, add the URLs of several representative merged GitHub pull requests (Ground Truth PRs). Include a variety of PR types to teach Superconductor what good code looks like for your project. Superconductor infers the spec from the ground-truth merges, then automatically launches implementations using the agents you've selected (Claude Code, Codex, Gemini, etc.).
Agents’ work is automatically evaluated by a reviewer agent for correctness, code quality, and completeness. Runtime and cost are tracked as well. Results roll up into a Quality-vs-Cost scatterplot and a per-ticket table with per-profile means, along with links to individual implementations and evaluations.
Build your own SWE-Bench by
- Creating a project on superconductor.com with your repo or repos
- Pinging us at help@superconductor.com to enable the beta feature
- Selecting your agents and gold standard PRs!
