Agent Experience (AX) platform
Ship products
agents love
to use.
AI agents are the new power users of the web. See your product through their eyes, and ship with confidence.
›_Session timelineRUNNING · RUN 0128
> Prompt: "Deploy my app using the CLI"
- Research · Read the CLI docs·
- Setup · Install the CLI·
- Auth · Log in from the CLILogin wall · 3 retries·
- Deploy · Run the first deploy·
Overall score
Running…
0/100
⚠Top friction: Login flow
01 · The platform
A better experience.
For a new kind of user.
Agents are your next power users. Give them a product that just works.
01
Failure modes
See where agents get stuck and how to fix it.
02
Eval scenarios
Test the flows that matter in any setup.
03
Benchmark
Compare models and scenarios, and benchmark against competitors.
04
Regression alerts
Get alerted when a critical workflow stops working for agents.
Trusted by
02 · How it works
A self-improving loop.
Evaluate critical workflows, watch how agents complete them, find the failure modes and ship fixes. Then track every release and repeat.
1Start
2See
3Improve
03 · Benchmarks
Can an agent sign up for your product on its own?
We run the same tasks across devtools, in public. See who agents get running first, and who they give up on.
Agent Arena
#1#2#3#4#5#6
You
How easy is it for AI agents to get started with devtools?
Explore the Arena01What is Agent Experience (AX)?+
AX is user experience for AI agents: how easily an agent can use your product and get real work done with it, without a human stepping in.
02What can I evaluate with Firegoat?+
Any workflow a coding agent can attempt: getting started, auth, your CLI, MCP server or SDK, and multi-step tasks. Run it on 25+ coding agent and model combinations, from Claude Code and Codex to OpenCode and Cursor.
03We tried building our own evals. How is this different?+
Evals are hard to get right. We focus on consistency: the same workflows, run the same way on every agent and model, so you get the full picture of your AX over time. You don't design, build or maintain any of it. We send you fixes and alert you to regressions. Your time goes into building what agents and humans want, not the measurement system behind it.
04How do you keep results honest?+
The benchmark is independent. Every product gets the same task and the same agents, scored the same way. Read our methodology
05How much of my team's time does it take?+
It's up to you. We set up your first evals with you, then they run on their own. Every finding comes with a suggested fix your coding agent can apply, so your team can be as hands-off as merging the PR. Teams that want to go deeper dive into the agent traces, adjust the judging criteria and add scenarios for sharper results.
06Does it work with GitHub?+
Yes. Connect the GitHub app and run an AX eval on any PR you choose, against its preview deployment. Results land as a comment on the PR: what worked, what broke, and where agents got stuck.
07How do I get started?+
Request access and we'll set up your first evaluation with you.
Agents are already using your product.
Get agent-ready.
We'll run your key workflows through Claude Code and Codex and walk you through what to fix first.