Skip to content

DIAGNOSTIC_COMMAND does not actually restrict cargo to build/test/check #37

Description

@kaluli123123

Summary

The cargo alternative in DIAGNOSTIC_COMMAND lists build|test|check, but the restriction has no effect: every cargo <anything> passes the eligibility gate.

cargo(?:\s+(?:build|test|check))?

The subcommand group is optional, so bare cargo already satisfies the alternative, and the trailing (?:\s|$) of the outer group matches the space that precedes whatever subcommand follows. The group can never change the match outcome.

export const DIAGNOSTIC_COMMAND =
/(?:^|[;&|()\s])(?:lake\s+build|lake\s+env\s+lean|lean|coq|cargo(?:\s+(?:build|test|check))?|zig\s+build|pytest|python(?:3)?\s+-m\s+(?:pytest|unittest|py_compile)|ctest|cmake\s+--build|ninja|make|npm\s+test|pnpm\s+test|yarn\s+test|go\s+test|bazel\s+test)(?:\s|$)/i;

Reproduction

Observed on main commit d7ecfc089944f0d04b80122a0a9a6ca0d786f3d0, Node.js v22.22.2.

import { DIAGNOSTIC_COMMAND } from "./src/sol-pi/extensions/evidence-preserving-reducer/config.ts";
for (const c of ["cargo build", "cargo fmt", "cargo publish", "cargo login", "cargo install foo", "cargo"])
  console.log(c.padEnd(20), DIAGNOSTIC_COMMAND.test(c));

Actual behavior

cargo build          true
cargo fmt            true     <- not build|test|check
cargo publish        true     <- not build|test|check
cargo login          true     <- not build|test|check
cargo install foo    true     <- not build|test|check
cargo                true     <- no subcommand at all

For comparison, the neighbouring python(?:3)?\s+-m\s+(?:pytest|unittest|py_compile) alternative has no optional group and does restrict correctly.

Impact

DIAGNOSTIC_COMMAND is the eligibility gate in reduceToolResult(). Passing it is what makes a tool result's body eligible to be archived and sent to the configured reducer model.

SECURITY.md frames remote reduction as something to enable deliberately for diagnostic logs. cargo publish / cargo login / cargo install output is registry and credential-adjacent traffic rather than a build or test log, and LIKELY_SECRET is documented as "a precaution rather than a complete secret scanner".

There is a cost side too: every false positive spends a reducer model call on output the mechanism was never meant to reduce.

Expected behavior

The cargo alternative should admit only its diagnostic subcommands, as the listed alternatives suggest was intended. A toolchain selector (cargo +nightly test) should stay eligible, since it matches today.

I have a focused fix with regression tests ready.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions