Skip to content

README.md

Latest commit

 

History

History
102 lines (75 loc) · 5.61 KB

README.md

File metadata and controls

102 lines (75 loc) · 5.61 KB
Raw

Test Directory

The test directory uses execution lanes first and behavior areas second.

Execution lanes

Directory Vitest project Purpose
installer-integration/ installer-integration Tests that spawn the real installer process
package-contract/ package-contract Tests that import compiled CLI or plugin artifacts
e2e/support/ e2e-support Deterministic tests for E2E fixtures and support code
e2e/live/ e2e-live Opt-in tests that mutate external state

Other *.test.js and *.test.ts files outside the dedicated lanes above belong to the integration project. The project globs in vitest.config.ts must remain disjoint and exhaustive.

Shared test code

  • Put passive inputs in fixtures/.
  • Put deterministic reusable utilities in helpers/.
  • Put stateful harnesses, fake services, and process setup in support/.
  • Keep one-test companion modules with their owning test when practical.

Adding tests

Choose the execution lane from the boundary that the test exercises. Within the integration project, group new tests by the behavior that owns the assertion. For example, process-recovery/ owns sandbox process and forward recovery coverage, channels/ owns channel lifecycle coverage, and credentials/ owns host credential storage and reset coverage. Do not put an ordinary integration test in e2e/ or package-contract/.

Run npm run test:projects:check after adding or moving a test.

Regression evidence

Reproduce a defect before fixing it when feasible. If reproduction is not feasible, record why and preserve the strongest pre-fix evidence. Add regression coverage at the earliest stable behavior boundary that could detect the defect. Add higher-level coverage only for a distinct integration boundary. Include negative and state-safety evidence when the acceptance criteria or risk require it.

Rerun affected tests after an edit or hook autofix changes tested behavior.

When a defect escapes normal controls, record the product cause, detection gap, and smallest durable prevention evidence in the issue or pull request. Search a bounded set of sibling paths for the same failure class. Fix sibling instances only when they share the cause and fit the current scope.

Test contracts

Do not read shipped YAML, JSON, manifests, workflows, or E2E runtime files only to assert literal structure. Use synthetic fixtures for schema tests. Test behavior through the owning consumer or validator.

A direct source-shape assertion requires a reviewed security or compatibility trust-boundary exception. Put this annotation immediately above the test:

// source-shape-contract: security -- Cross-field digest equality protects the shipped trust anchor

Use security or compatibility as the category and state the concrete reason. Add the file, test title, and category to the reviewed allowlist in scripts/find-source-shape-tests.mts. npm run source-shape:check rejects unsupported categories, short or misplaced reasons, missing allowlist entries, and unused entries.

Live E2E assertion ratchet

Run npm run e2e:assertions:scan to inspect direct assertions and assertions reachable through live companion modules. Run npm run e2e:assertions:check to compare the current suite with ci/e2e-assertion-budget.json.

An E2E assertion reduction must classify each removed assertion as already covered by a lower test, moved to a lower test, covered by another retained behavior test, or unnecessary because it has no distinct quality value. Do not move assertions into helpers, aggregate objects, or generated probes. After a valid reduction, run npm run e2e:assertions:update and include the lower baseline in the same change. The ratchet rejects growth and stale baselines.

Maintainer-approved exceptions are recorded in ci/e2e-assertion-growth-exceptions.json. Each entry binds a PR number to SHA-256 digests of the exact base and candidate budget files. Local hooks and candidate CI use the branch policy. The independent GitHub growth check uses only trusted-base policy and the event's PR number; a candidate cannot authorize its own independent check. A new exception must either land on main first or have its expected independent-check failure explicitly waived by a maintainer. Remove the entry after its PR merges.

New test files must use TypeScript. Each plugin test must execute at least one Vitest expect assertion. The repository test configuration owns automatic mock and environment cleanup; restore direct global or environment mutations in the test that owns them.

Follow WRITING.md for behavior-oriented test titles. Put a local issue reference in a final suffix such as (#1234).

macOS host tools

Some tests require GNU command-line tools that macOS does not provide. The macos-vitest job in .github/workflows/platform-vitest-main.yaml owns the authoritative package list. The hosted runner must already provide gtar; the workflow verifies that prerequisite before Homebrew installs the other tools. It then puts the installed GNU binaries first on PATH and exposes gtar as tar only to the Vitest process through a private shim directory. This workflow runs only after pushes to main; candidate-controlled and manually dispatched code does not receive its package credential. WSL installs gnu-coreutils for fixtures that require GNU utility behavior, keeps Ubuntu's default utilities intact, and stops Docker before non-live tests.