ax agent

These commands require authentication and an active organization. All accept --org <ORG_ID> to target another organization for one command. agent list, agent test, and agent version accept --json for machine-readable output. Arguments marked required must be supplied.

The ax agent validate command was removed. Run ax experiment validate <path> instead; it applies the same agent-version and compatibility checks as submit.

Compatibility status

Compatibility is tracked for an exact agent version and canonical model combination. The status field has three values:

A combination is supported only when its status is verified and the model is on the agent's model menu, the catalog models that agent offers. Listing, validation, and submission all apply this one rule: agent list and ax model list list only supported combinations, ax experiment validate and submission run only supported combinations and skip every other one with a warning, and a definition with nothing left to run is rejected. An untested combination, unknown or with no compatibility entry yet, can be tested with agent test. A model absent from the canonical catalog has no compatibility entry and is never supported.

Only unknown changes automatically. Its first conclusive outcome, from agent test or the platform's own compatibility checks, sets the combination to verified or unsupported; later runs do not replace that status.

In agent version JSON output, each compatibility entry includes the canonical model id, status, and an optional reason, for every model the registry tracks for that version. agent list includes only the supported models, so every status is verified, and omits the reason:

{
  "compatibility": [
    {
      "canonical_model_id": "anthropic/claude-opus-4-8",
      "status": "verified",
      "reason": null
    }
  ]
}

agent list

List agents and their latest supported versions.

ax agent list [AGENT] [--all-versions] [--model <MODEL>]

A version supports a model only when the combination is supported: verified and on the agent's model menu. This is the rule validation and submission use, and the same rule as ax model list. The latest supported version is the newest stable supporting version, which is the version an experiment without a version or saved pin runs for that model alone. An agent with no supporting version is left out. opencode, eve and pi versions are listed only in organizations where that agent is enabled.

A pre-release, such as Codex 0.161.0-alpha.9, is never the latest version. It runs only when an experiment pins that exact version, and a saved pin to one is kept. --all-versions lists pre-releases with (pre-release).

$ ax agent list --model anthropic/claude-haiku-4-5 --all-versions

Supported agents and versions for anthropic/claude-haiku-4-5:

AGENT   VERSION
claude  2.1.283 (latest)
cursor  2026.09.26-dd393fe (latest)
cursor  2026.09.18

--json lists the same rows as the table. Each version's compatibility lists only its supported models, without reasons; agent version shows every status. To get every registered version, as --json returned before --all-versions existed, add --all-versions.

agent test

Test whether an exact agent version works with a model.

ax agent test <AGENT>@<VERSION> --model <MODEL>

The command runs a fixed check: one short turn in which the agent is asked to reply to the model. You choose only the agent version and model. The check runs at Fiveonefour's expense and is not billed to your organization. The agent itself answers, so the check also runs for a model that experiment validation does not offer for that agent, or that the registry marks unsupported.

Every run is a new check, even when the pair already has a saved compatibility status or another check of it is running, and the answer is that check's own. When the model provider rate limits the check (HTTP 429), the check times out, or the provider returns HTTP 503, the check is retried up to three more times before it answers. Other failures answer at once.

The output prints the check's answer first, then the pair's saved status. A pair with no saved result (unknown) saves the answer for everyone: verified when the model replied, unsupported when the agent rejected the model. Later ax model list, ax agent list, experiment validation, and submission use it: a verified pair on the agent's model menu is then listed and runs. A saved verified or unsupported status does not change from a check, so the answer can differ from the saved status. Fiveonefour is notified when a verified pair fails a check.

$ ax agent test cursor@2026.09.26-dd393fe --model anthropic/claude-haiku-4-5
Running a compatibility test (test 01K7P3Q0WJ4M5N6P7Q8R9S0T1V)...

Works: cursor@2026.09.26-dd393fe with anthropic/claude-haiku-4-5 answered in this check.
Saved status: verified.

When the agent rejects the model, a Reason: line quotes the rejection. Other failures are undetermined. The detail explains the failure and whether retrying could help. When the run recorded an error, the detail ends with it, with credentials redacted. The check's run is not in your organization, so this output is the only place to read that run's error. Provider HTTP 500, model-specific 4xx, and unavailable models do not suggest retrying. Authentication failures need credentials fixed first. Network errors, timeouts, and HTTP 408/429/502/503/504/529 can suggest retrying later. Unknown failures retain retry guidance. The human output prints Retry: only when retrying could help; an older server without retryability information retains that hint.

For an undetermined JSON result, retryable tells you whether repeating the check could help, and failure_class uses these full names:

The command exits 0 when the pair works, 3 when it does not, and 4 when the check could not determine either; other errors exit 1. The command prints the check's test id when it starts. If the check does not finish within the command's wait, the error names the test id; running the command again starts a new check. Starting checks is rate limited per user and across Fiveonefour; retries within one check do not count. Repeated provider errors do not turn the pair into saved unsupported support.

agent version

Inspect one exact agent version, its artifact digests, and model compatibility.

ax agent version <AGENT> <VERSION>