CLI E2E Testcase Writer

Work on one domain per run. Produce exactly two artifacts for that domain:

• workflow testcase files under

  tests/cli_e2e/{domain}/

• tests/cli_e2e/{domain}/coverage.md

Focus on domain testcase files. Do not change shared E2E support code such as

tests/cli_e2e/core.go

unless the user explicitly asks. Treat

tests/cli_e2e/demo/

as reference only.

Core standard

• Make the testcase scenario-based and self-contained.
• Prove one workflow end to end: create plus follow-up read, or mutate plus teardown.
• Prefer one file per workflow or one closely related feature.
• For mutable flows, prove persisted state with read-after-write assertions, not just exit code.
• Leave prerequisite-heavy paths uncovered when they cannot be proven, and explain why in

  coverage.md

  .

Workflow

1. Explore the live CLI before writing code

lark-cli --help
lark-cli <domain> --help
lark-cli <domain> +<shortcut> -h
lark-cli <domain> <group> --help
lark-cli <domain> <group> <method> -h
lark-cli schema <domain>.<group>.<method>

2. Count leaf commands for the denominator

• A leaf command is one that executes an action — it has no further subcommands.
• If

  lark-cli <domain> <group> --help

  lists no subcommands,

  <group>

  itself is the leaf.
• Count

  task +create

  as one leaf and

  task tasks get

  as one leaf.
• Do not count parameter combinations.
• Reuse coverage already present under

  tests/cli_e2e/{domain}/

  . Do not count

  tests/cli_e2e/demo/

  .

3. Choose the proof surface before editing

Identify the provable risks for the touched workflow: invalid input, missing prerequisite, identity or permission, state transition, output shape, cleanup safety. If only the happy path is testable, document the blocked risk areas in

coverage.md

.

4. Add or update the workflow testcase

• Use

  clie2e.RunCmd(ctx, clie2e.Request{...})

  .
• Put command path and plain flags in

  Args

  ; put JSON in

  Params

  (URL/path parameters) and

  Data

  (request body).
• Prefer one top-level test per workflow with

  t.Run

  substeps.
• Register teardown on

  parentT.Cleanup

  so it survives subtest failures.
• When touching an existing command, verify the JSON response shape is stable: assert status type, field paths, and identifiers consumed by later steps before changing assertions.

5. Run and iterate

Run

go test ./tests/cli_e2e/{domain} -count=1

while iterating and before finishing. If command shape or behavior is unclear, re-check help or schema (step 1) before changing assertions.

6. Refresh the domain outputs

• Update the workflow testcase files.
• Update

  coverage.md

  : recompute the denominator from live help output, mark each command as

  shortcut

  or

  api

  , and keep one command table for the whole domain.

Testcase rules

• Override

  BinaryPath

  ,

  DefaultAs

  , or

  Format

  on

  clie2e.Request

  only when the testcase truly needs it.
• Use

  require.NoError

  ,

  result.AssertExitCode

  ,

  result.AssertStdoutStatus

  ,

  assert

  , and

  gjson

  .
• Shortcut responses (

  {ok: bool}

  ) assert

  true

  ; API responses (

  {code: int}

  ) assert

  0

  .
• Use

  t.Helper()

  only for setup or assertion helpers that are called from multiple tests.
• Use table-driven tests only when the scenario shape repeats across inputs.
• For expected failures, assert stderr content and exit code when the environment makes them deterministic.
• If identity or external fixtures cannot be proven, leave the command uncovered and document the prerequisite rather than faking confidence.

coverage.md

Keep

coverage.md

brief and mechanical. Include:

• a domain-specific H1 title
• a metrics section with denominator, covered count, and coverage rate
• a summary section restating each

  Test...

  workflow, key

  t.Run(...)

  proof points, and main blockers
• one command table for all commands

Recommended structure:

# <Domain> CLI E2E Coverage

## Metrics
- Denominator: N leaf commands
- Covered: N
- Coverage: N%

## Summary
- TestXxx: ... key `t.Run(...)` proof points ...
- Blocked area: ...

## Command Table
| Status | Cmd | Type | Testcase | Key parameter shapes | Notes / uncovered reason |
| --- | --- | --- | --- | --- | --- |
| ✓ | task +create | shortcut | task_status_workflow_test.go::TestTask_StatusWorkflow | basic create; create with due | |
| ✕ | task +assign | shortcut |  | none | requires real user open_id |

• Mark each command

  shortcut

  or

  api

  .
• Write testcase entries in

  go test -run

  friendly form.
• Commands only exercised in

  parentT.Cleanup

  teardown are not counted as covered.
• Do not split covered and uncovered commands into separate sections.

Guardrails

• Run as bot identity only; do not assume

  --as user

  works.
• Do not place new real coverage under

  tests/cli_e2e/demo/

  .
• Do not depend on preexisting remote data.
• Do not fabricate open_ids, chats, docs, or other remote fixtures.
• Prefer deterministic negative cases over tenant-dependent assertions.
• Do not guess

  Params

  or

  Data

  fields when help or schema can tell you the exact shape.
• Do not hardcode obvious defaults unless the command truly requires explicit flags.
• Do not put agent, model, or vendor brand names in visible remote test data; use neutral prefixes such as

  lark-cli-e2e-

  or

  <domain>-e2e-

  .
• A command is covered only when the testcase asserts returned fields or persisted state, not just exit code.
• Cleanup-only execution is not primary coverage, except

  delete

  in the same workflow that created the resource.