filecoin-project/filecoin-pin

Add CLI error-scenario smoke test matrix for devs

Open

#470 opened on May 8, 2026

 (7 comments) (0 reactions) (1 assignee)TypeScript (18 forks)github user discovery
enhancementgood first issuehelp wantedready-for-workteam/filecoin-pinteam/fs-wg

Repository metrics

Stars
 (22 stars)
PR merge metrics
 (Avg merge 3d 12h) (18 merged PRs in 30d)

Description

Description

Manually verifying CLI error scenarios across all commands is slow and ad-hoc. During PR #468 (CliFatal sentinel work) we hand-rolled a 15-scenario bash sweep to confirm error output and exit codes. We should turn this into a reusable script or test suite so future CLI changes can re-run the matrix in one command.

Goals

  • Cover every command + every documented error scenario (invalid args, missing flags, bad RPC, no auth, insufficient funds, user cancel, etc.)
  • Capture stdout, stderr, exit code per scenario
  • Run without live RPC where possible (use mocks or no-network failure paths)
  • Clear pass/fail per scenario; easy to add new scenarios

Acceptance criteria

  • pnpm run smoke (or similar) executes the full matrix
  • Output format makes it easy to spot regressions (snapshot diff or labeled per-scenario block)
  • At minimum covers: payments setup/fund/deposit/withdraw/status, add, import, data-set show/list/terminate, provider list/show/ping, rm
  • Argument-validation scenarios (no network, no auth) run in CI
  • Network-dependent scenarios (insufficient funds, bad RPC) are clearly tagged and skipped in CI by default
  • README or CONTRIBUTING.md documents how to run + add scenarios

Notes

  • See PR #468 conversation for the manual sweep template
  • Reuse existing src/test/mocks/synapse-sdk.js where mocking is preferable to live RPC
  • Consider snapshot testing on captured stderr (vitest toMatchInlineSnapshot)
  • Out of scope: replacing existing unit/integration tests

Contributor guide