Skip to content

Decide how the CLI names itself in commands it tells people and agents to run #576

Open

Description

@shane-kercheval

Problem

When the CLI tells someone what to run next, it names itself as a bare clerk. Clerk's own instructions for agents start the CLI through a package runner such as npx, and on those machines clerk usually isn't on PATH.

For example, an agent that started the CLI with npx -y clerk@latest deploy is told "Run clerk deploy status again shortly". Run as written, that fails with "command not found" on a machine with no global install. On a machine that also has a global install, it runs that install, whose version may be older than the one that gave the advice.

The bare name is hardcoded in about 210 lines of .ts source in packages/cli-core/src (origin/main, 2026-10-21), with no shared helper: error messages, next steps (lib/next-steps.ts), and agent-mode nextAction text. Another 193 lines are in the per-command README files, which describe syntax and aren't run as written. (Counted by searching for `clerk <subcommand> outside comments and tests.)

Clerk's instructions for agents start the CLI this way:

  • The clerk-setup skill (published in the docs as prompts/cli-setup.md) runs everything as npx -y clerk@latest …, and its checker rejects a global install.
  • The docs' "For AI agents" blocks use npx clerk@latest …, and so does the SDK's missing-keys error (packages/shared/src/errors/keySetupGuidance.ts in clerk/javascript).
  • The clerk-cli skill uses a global clerk only when it's present and current. Otherwise it uses the runner that matches the project's lockfile: bunx, npx, pnpm dlx or yarn dlx.

Agents mostly cope today, because these mentions are advice: an agent reads "run clerk deploy status" and runs it with its own runner. That costs a rewrite on every suggestion, and on a machine with both a runner and a global install it can run a stale global clerk.

About 38% of agent machines never used a global install

On those machines a bare clerk probably fails. The numbers below come from CLI telemetry, which reports how each run was launched (install_method), in growth_dbt.int_cli_developer_command_runs. The window is 2026-09-21 to 2026-10-20, agent mode only (the CLI's output wasn't going to a terminal), with background commands (mcp run, update, telemetry) left out. Query in Hex.

How the CLI was launched Agent runs Share of runs Agent machines Share of machines
Global npm install 512,966 57% 6,528 49%
Homebrew 225,229 25% 1,658 13%
npx 119,755 13% 5,617 42%
bunx 8,817 1% 340 3%
npm script 21,897 2% 339 3%
Unknown 7,881 1% 151 1%
All 896,545 100% 13,261

A machine counts once for each launch method it used, so the machine shares add up to more than 100%.

  • About 5,000 agent machines (38%) launched the CLI only through npx or bunx. That's the 5,957 npx and bunx machine counts minus the 962 machines that also used a global install. It's an estimate, because a machine that used both npx and bunx counts twice in the 5,957.
  • 962 agent machines (7.3%) used both a runner and a global install. On these, a suggested bare clerk runs the global copy, which may be a different version from the one giving the advice.
  • Global installs run older versions. Version 3.1.0 (released 2026-08-12) is 20% of global-npm agent runs and 0.3% of npx runs. Version 4.0.0 (released 2026-10-15) is 20% of npx runs and isn't among the five most-used versions on global npm or Homebrew.
  • Runner machines run fewer commands each: about 21 runs per npx machine, against about 79 per global-npm machine. That's why runners are 45% of machine counts but 14% of runs.

The data can't show how often a suggested bare clerk actually fails. A "command not found" never reaches the CLI, so no telemetry is sent. The likely cost is one wasted tool call and a rewrite per suggestion, not a failed setup, because runner-only machines keep running follow-up commands through the runner. That's an inference, not a measurement.

Every count is a floor. Developers can turn telemetry off, and the CLI sends nothing for a machine's first command, so a machine that ran npx clerk init once and nothing else doesn't appear.

Decisions needed

None of these is decided. Each one lists the options and a recommendation.

1. How the CLI names itself for someone who didn't install it globally

The options:

  • A fixed npx -y clerk@latest.
  • The project's runner, from preferredRunner() in packages/cli-core/src/lib/runners.ts.
  • The way the current process was launched.

Recommendation: the way the current process was launched. Build it on currentCliPrefix() in packages/cli-core/src/lib/cli-invocation.ts, which clerk/cli#550 adds. That PR lets agents create a production app with clerk deploy --domain, and it uses the helper to name the CLI in the first commands meant to be run exactly as given (the deploy report's hostSetup and oauth.setup commands). The helper would return:

  • clerk when the clerk on PATH starts the running binary: either it is the binary (a Homebrew install) or it is the npm wrapper script that launches it (a global npm install). feat(deploy): let agents create production app with --domain #550's helper matches the first case but not yet the second; see "Risks" below.
  • npx -y clerk@latest or bunx clerk@latest when the CLI was launched through that runner. detectInstallMethod() in packages/cli-core/src/lib/env-signals.ts already detects these launches for telemetry.
  • The running CLI's quoted absolute path otherwise, which is what feat(deploy): let agents create production app with --domain #550 prints today for every launch that isn't the clerk on PATH.

Why: this is the only form sure to work on the machine that receives it, because that launch path just ran. It also ends the stale-global mix-up on the 962 machines that have both. The other two options fail for large groups:

  • A fixed npx form is wrong for the 82% of agent runs that come from a global install. For them it's slower, and it moves them off their own install.
  • The project's runner needs a lockfile, and many commands run without one. It also probes yarn by running yarn dlx --help.

Tradeoff: the same message reads differently on different machines, so docs can't quote it word for word. The runner form uses @latest to match every published instruction, so a release that lands mid-session can run a newer version than the one that gave the advice. Pinning to the running version (npx -y clerk@4.0.0) avoids that, at the cost of a version number in every message.

A possible simpler rule: print npx -y clerk@latest or bunx clerk@latest when that runner fetched the CLI, and otherwise print clerk whenever any clerk is on PATH. Only when none is does it print the running CLI's absolute path. "Fetched by a runner" means the running binary sits in npx's cache (/_npx/ in its path) or bunx's (bunx-).

  • It needs no npm wrapper check.
  • It also covers asdf and Volta. They put their own wrapper scripts on PATH, which a wrapper check wouldn't recognize, so their users would get the absolute path.
  • The tradeoff: when no runner fetched the CLI and the clerk on PATH is a different install from the running one, the suggestion runs that other install. That is what happens today.

2. Which text uses that form

Recommendation: suggestions in runtime output. That covers error messages, next steps and agent-mode nextAction. Three kinds of text keep their current form:

  • feat(deploy): let agents create production app with --domain #550's hostSetup and oauth.setup commands keep feat(deploy): let agents create production app with --domain #550's rule: clerk or the running CLI's absolute path, never a runner form. Agents run these exactly as given, and they handle secret keys. The absolute path runs the same version that wrote the report, with no download. A runner form would fetch clerk@latest first, which can be a different version. Whether npx ever writes to stdout while fetching, where the Vercel command reads the keys, is unverified.
  • Text the CLI writes into files keeps bare clerk, because someone may read it later on another machine. Today that is the missing-key error clerk init writes into the generated app's code: MISSING_PUBLISHABLE_KEY_ERROR in commands/init/frameworks/helpers.ts, and its Expo version in expo.ts.
  • The command READMEs and --help examples keep bare clerk, because they describe syntax and aren't run as written. That leaves 193 README lines out of the change.

3. What the skills and docs tell agents

Recommendation: the clerk-cli and clerk-setup skills and the docs' "For AI agents" blocks tell agents to run the commands the CLI prints as written, because the CLI prints them in the form it was launched with. Neither skill then needs its own rules for rewriting the CLI's advice.

Risks

The change is low risk, provided it either adds the one check that #550's helper is missing or uses the simpler rule in decision 1 (see the first item below). It changes only text the CLI prints: no command, flag, exit code or released output field changes. With that check, text for Homebrew and global npm installs, 82% of agent runs, stays clerk. Only launches where the clerk on PATH isn't the running CLI get different text: npx, bunx, npm scripts and unknown installs, at most 18% of agent runs. The clerk-cli skill doesn't depend on the wording: its agent-mode reference (skills/clerk-cli/references/agent-mode.md in clerk/skills) tells agents to show nextAction to the user and to branch on state and the exit code.

What could go wrong, most serious first:

  • Without a wrapper check, every global npm user's advice becomes a long file path. A global npm install puts packages/cli/bin/clerk on PATH. That file is a Node script that starts the platform binary from the @clerk/cli-<platform> package. feat(deploy): let agents create production app with --domain #550's helper compares the two files, finds them different, and prints the binary's absolute path, such as '/usr/local/lib/node_modules/clerk/node_modules/@clerk/cli-darwin-arm64/bin/clerk'. feat(deploy): let agents create production app with --domain #550's own test "npm global install, reached through its shim" asserts this. That's acceptable for feat(deploy): let agents create production app with --domain #550's few copy-exactly commands, but applied to every suggestion it would put a path like that in the advice for 57% of agent runs. The commands would still run. The fix is either a check that the clerk on PATH is the npm wrapper of the running binary, tested beside feat(deploy): let agents create production app with --domain #550's case, or the simpler rule in decision 1, which needs no check. Homebrew needs no such check, because its formula installs the binary itself (bin.install "clerk" in scripts/lib/homebrew.ts).
  • A string the change misses keeps printing bare clerk, and the tests won't notice. About 210 strings change. The simplest way to keep the roughly 220 test assertions passing is to fix the prefix to clerk with _setCliPrefix() in test setup, and then a string that skipped the helper passes every test. Nothing that works today breaks: the missed message keeps today's behavior. A lint rule or test that fails on a `clerk <subcommand> literal in runtime source would catch every missed string.
  • On Windows the problem stays unless the runner case comes first. feat(deploy): let agents create production app with --domain #550's helper returns clerk on every Windows launch, because no single quoting works in both PowerShell and cmd.exe. The runner form needs no quoting, so the runner case can run before that Windows check. If it runs after, Windows users who launch through npx keep getting bare clerk.
  • An npm exec run of a project's own clerk dependency is told to run npx -y clerk@latest. detectInstallMethod() reports npx for any npm exec run, so the advice would run the latest version instead of the project's pinned one. Both commands work, and this applies only when the clerk on PATH isn't the running CLI. The simpler rule in decision 1 avoids it, because it checks only where the binary is.

NEXT_STEPS in lib/next-steps.ts is a constant built when the module loads, so its entries become functions or are filled in when printed. That's a structural edit to about 37 strings. There is no next-steps.test.ts; the strings are covered only through the 8 command test files that reference NEXT_STEPS, so a step no command test reaches would go unchecked.

Posted by Claude for Shane

Activity

  1. changed the title [-]Deploy report commands assume clerk is on PATH; decide how generated commands invoke the CLI[/-] [+]Decide how the CLI names itself in commands it tells people and agents to run[/+] on Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions