Skip to content

validate: tier chain findings by actionability (warning vs info) - #22

Open
dstrodtman wants to merge 2 commits into
mainfrom
djs-260918-chain-benign-tiering
Open

validate: tier chain findings by actionability (warning vs info)#22
dstrodtman wants to merge 2 commits into
mainfrom
djs-260918-chain-benign-tiering

Conversation

@dstrodtman

@dstrodtman dstrodtman commented Sep 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

What

Split chain-validation findings into severities so the actionable ones aren't buried under benign ones:

  • warning — the downstream rule B is specific, or a path-preserving wildcard move (:splat in B's target). B would route A's target to a different destination, so the author should point A there directly.
  • info — a benign overlap that can't actually chain. Two shapes:
    1. Fixed-page catch-all — A's target lands under a broad wildcard catch-all with a splat-less target. force: false means the catch-all fires only if A's target 404s, and a prefix catch-all can't be pointed past.
    2. Preempted splat — A is a /P/* → /Q/:splat move and B is a specific /Q/<tail>, but a lower-position rule already matches the only source that would drive A into B (/P/<tail>), so A never fires there.

_print_findings summarizes info to a count (so a real warning stands out) and gains validate --show-info to list the detail. info never fails --strict; --strict still gates on error only.

Why

The detector flagged every static overlap between a rule's to and another rule's from as a warning, with no notion that a force: false catch-all only fires on a 404, nor that a specific source rule preempts a wildcard. On docs.ray.io's current.yaml that's a wall of 44 warnings — enough to bury any real chain. The module docstring and the "Validator follow-ups" note already flagged this as the conservative-over-reporting tradeoff.

The tiering is structural and offline (no new inputs, no credentials). It doesn't prove A's target resolves to a live page, so a warning is the actionable signal and info is a note. Per-URL precision (resolving substituted :splat, or an opt-in live/source oracle) stays listed as future work.

Effect on anyscale-ray's current.yaml

validate goes from 44 undifferentiated warnings to 0 warning / 44 info. The 39 catch-all overlaps and the 5 /auto_examples/* splat overlaps are all benign: the latter are preempted by specific /auto_examples/<x>.html rules and each resolves in a single hop live (verified), so no chain fires. A genuinely new chain — a specific onward-redirecting target that isn't preempted or behind a catch-all — would still surface as a warning.

Testing

  • pytest — 370 pass (added validate-layer tests for the warning/info split and for splat preemption, plus CLI tests for the count-summary and --show-info).
  • ruff check . — clean.
  • Ran the branch's validate against anyscale-ray's live current.yaml: 0 error, 0 warning, 44 info, exit 0; --show-info lists all 44 with per-finding reasons.

No ticket; happy to link one if wanted.

🤖 Generated with Claude Code
dstrodtman and others added 2 commits September 18, 2026 12:38
@dstrodtman @claude
validate: tier chain findings by actionability (warning vs info)
407e760 
The chain detector flagged every static overlap between a rule's target and
another rule's `from` as a warning, with no notion that a `force: false`
catch-all only fires on a 404. Against a real ruleset that produced a wall of
warnings (44 on docs.ray.io's current.yaml), which buries the few chains an
author can actually act on.

Tier chain candidates instead:

- `warning` when the downstream rule B is specific, or a path-preserving
  wildcard move (`:splat` in B's target). B would route A's target to a
  different destination, so the author should point A there directly.
- `info` when A's target merely lands under a broad wildcard catch-all whose
  target is a fixed page (no `:splat`). force=false means the catch-all fires
  only if A's target 404s, and a prefix catch-all can't be pointed past, so
  the overlap is benign graceful degradation, not a fixable chain.

The split is structural and offline; it doesn't prove A's target resolves to
a live page, so a warning is the actionable signal and info is a note.
`_print_findings` summarizes info to a count so a real warning stands out;
`validate --show-info` lists the detail. Info never fails `--strict`.

On current.yaml this turns 44 undifferentiated warnings into 5 real
warnings (the `/auto_examples/*` splat landing on specific onward-redirecting
example pages) and 39 benign info notes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dstrodtman @claude
validate: info-tier splat overlaps a lower-position rule preempts
1866fbd 
The actionability tiering still warned on a wildcard->specific overlap even
when a lower-position rule preempts the wildcard for the exact source that
would reach B, so the chain can never fire. On docs.ray.io's current.yaml the
five surviving warnings were all this shape: /auto_examples/<x>.html has its
own rule ahead of the /auto_examples/* wildcard, so the /ray-core/examples/<x>
intermediate the splat would hit is never traversed (verified live: each of
the five resolves in a single hop to its final destination).

Reconstruct the source a splat overlap needs (A.from prefix + the splat value
that yields B.from) and, if a lower-position rule matches it across every
version A covers, emit info instead of warning. exact (single-version)
preemptors are ignored so a per-version gap isn't hidden. current.yaml now
validates at 0 warning / 44 info.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant

@dstrodtman