feat(server): non-interactive bundle attestation for /v1/bundle

[lockwobr]

Summary

POST /v1/bundle?attest=true now returns a signed bundle using a non-interactive, server-owned identity (KMS or private-Sigstore keyless), with tool provenance from the aicrd binary attestation shipped in the image. The server never signs with request-supplied identity.

Motivation / Context

The /v1/bundle endpoint never attested or signed bundles: the handler built BundleOptions with no Attester, so a CI pipeline or platform bundling via the API could not get a signed artifact back. This closes the server-side gap in the supply-chain epic. The CLI's default signing path is interactive browser OIDC, a non-starter for a daemon, so this adds non-interactive server signing.

Fixes: #1150
Related: #1149

Type of Change

• New feature (non-breaking change that adds functionality)
• Build/CI/tooling

Component(s) Affected

• API server (cmd/aicrd, pkg/server)
• Bundlers (pkg/bundler, pkg/component/*)
• Core libraries (pkg/errors, pkg/k8s) — pkg/client/v1, pkg/defaults
• Docs/examples (docs/, examples/)
• Other: .goreleaser.yaml, new server-kms-e2e workflow, chainsaw suite

Implementation Notes

Trust boundary: the server signs as itself. Signing identity comes entirely from server env config, never from the request; ?attest=true is only a boolean trigger. Request-supplied / on-behalf-of-caller OIDC is explicitly out of scope.

Two signing modes (mutually exclusive, fail-fast validated at startup):

• KMS: AICR_SIGNING_KEY (awskms:// / gcpkms:// / azurekms:// / hashivault://).
• Keyless against a private Sigstore: AICR_FULCIO_URL + a token source (AICR_IDENTITY_TOKEN_FILE, read fresh per request, or GHA ambient). Shared: AICR_REKOR_URL, AICR_TLOG_UPLOAD (KMS-only), AICR_SIGNING_CONFIG_PATH.

Tool provenance (the aicrd binary attestation): the release build (.goreleaser.yaml) now runs cosign attest-blob on the aicrd binary and ships the result per-architecture in the ko image via KO_DATA_PATH. The server verifies its own attestation once at startup, caches it, and embeds it in each signed bundle. It fails to start if signing is enabled but the attestation is missing or invalid. The attestation path resolves via AICR_BINARY_ATTESTATION_FILE, then KO_DATA_PATH/GOARCH, then next-to-binary; the identity pin is overridable (still org-pinned) via AICR_BINARY_ATTESTATION_IDENTITY_REGEXP, mirroring the CLI's --certificate-identity-regexp.

Additive, CLI-safe: the bundler gains WithVerifiedBinaryAttestation and the client facade gains BundleOptions.BinaryAttestation. With neither set (the CLI path), behavior is byte-identical: no attest is inferred from Attester != nil, and the discover-and-verify-per-run path is unchanged.

Testing

make test    # -race, all packages
make lint    # golangci-lint + yamllint
go test -tags integration ./pkg/server/ -run TestServerSigning   # vs real OpenBAO KMS

• make test green on the changed packages; make lint 0 issues; golangci-lint (incl. -tags integration) 0 issues.
• Go integration test (//go:build integration, skips without VAULT_ADDR) verified live against a real OpenBAO Transit KMS: bundle returned signed, bundleAttested=true.
• New chainsaw suite tests/chainsaw/signing/server-bundle-attestation-vault/ smoke mode passed live (server rejects attest=true when unconfigured with 400; unsigned bundle still returned). The full attested path runs in the new server-kms-e2e.yaml workflow (needs GitHub OIDC to attest the binary), the same CI-only limitation as the existing signing e2es.
• The integration test is build-tagged out of make test, and the chainsaw suite has no chainsaw-test.yaml, so neither is auto-run by make test / make e2e / make qualify; they do not affect local dev.

Note: tests/releasepolicy fails on local macOS due to a pre-existing environment gap (GNU timeout absent in publish-homebrew.sh), unrelated to this change; it fails identically on main and passes in CI. Tracked for a separate portability fix.

Risk Assessment

• Medium — Touches the shared bundler signing path, but strictly additively (CLI path byte-identical), and server signing is opt-in and off unless configured.

Rollout notes: Server signing is disabled by default (no env config → capability off, current behavior preserved). Turning it on in a real deployment depends on the aicrd binary attestation being present in the image, which this PR wires into the release build. No migration.

Checklist

• Tests pass locally (make test with -race) — modulo the pre-existing environmental releasepolicy gap noted above
• Linter passes (make lint)
• I did not skip/disable tests to make CI green
• I added/updated tests for new functionality
• I updated docs if user-facing behavior changed
• Changes follow existing patterns in the codebase
• Commits are cryptographically signed (git commit -S)

[github-actions]

Recipe evidence check

No leaf overlays affected by this PR.

This gate is warning-only and never blocks merge.

[github-actions]

Coverage Report ✅

Metric	Value
Coverage	80.7%
Threshold	80%
Status	Pass

Coverage Badge

![Coverage](https://img.shields.io/badge/coverage-80.7%25-brightgreen)

Merging this branch changes the coverage (2 decrease, 2 increase)

Impacted Packages	Coverage Δ	🤖
github.com/NVIDIA/aicr/pkg/bundler	81.08% (+1.20%)	👍
github.com/NVIDIA/aicr/pkg/bundler/verifier	75.50% (-0.08%)	👎
github.com/NVIDIA/aicr/pkg/client/v1	77.54% (+0.18%)	👍
github.com/NVIDIA/aicr/pkg/defaults	100.00% (ø)
github.com/NVIDIA/aicr/pkg/server	78.66% (-0.53%)	👎

---

Coverage by file

Changed files (no unit tests)

Changed File	Coverage Δ	Total	Covered	Missed	🤖
github.com/NVIDIA/aicr/pkg/bundler/bundler.go	81.81% (+1.55%)	786 (+11)	643 (+21)	143 (-10)	👍
github.com/NVIDIA/aicr/pkg/bundler/verifier/verifier.go	74.00% (-0.07%)	300 (+3)	222 (+2)	78 (+1)	👎
github.com/NVIDIA/aicr/pkg/client/v1/bundle.go	76.19% (+2.42%)	63 (+2)	48 (+3)	15 (-1)	👍
github.com/NVIDIA/aicr/pkg/defaults/timeouts.go	0.00% (ø)	0	0	0
github.com/NVIDIA/aicr/pkg/server/bundle_handler.go	75.56% (+17.49%)	90 (+28)	68 (+32)	22 (-4)	🎉
github.com/NVIDIA/aicr/pkg/server/serve.go	0.00% (ø)	30 (+7)	0	30 (+7)
github.com/NVIDIA/aicr/pkg/server/signing.go	71.11% (+71.11%)	90 (+90)	64 (+64)	26 (+26)	🌟

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds server-side bundle attestation through POST /v1/bundle?attest=true, supporting KMS and keyless signing modes. Signing configuration is validated at startup, and the server verifies and caches its own binary attestation for embedding in signed bundles. Bundler and client APIs accept verified attestation bytes. GoReleaser generates binary attestations, while new CI and OpenBAO-based tests validate signed and unsigned server flows.

Estimated code review effort: 5 (Critical) | ~120 minutes

Suggested labels: theme/ci-dx

Suggested reviewers: mchmarny

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: non-interactive bundle attestation in the server API.
Description check	✅ Passed	The description matches the changeset and explains the server-side attestation feature, signing modes, and supporting updates.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/server-bundle-attestation

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/server/bundle_handler.go`:
- Around line 203-225: Add an authorization check before the attestation setup
in the attestRequested branch, requiring an authorized principal or approved
recipe/source policy before constructing the attester or signing the bundle.
Reject unauthorized requests with the existing error-response mechanism, and
preserve the current signing flow only for requests that pass this authorization
check.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: b3245382-984d-4413-8951-497bb1cc31c6

📥 Commits

Reviewing files that changed from the base of the PR and between 091edf8 and 8f45f19.

📒 Files selected for processing (19)

• .github/workflows/server-kms-e2e.yaml
• .gitignore
• .goreleaser.yaml
• api/aicr/v1/server.yaml
• docs/contributor/api-server.md
• docs/user/api-reference.md
• pkg/bundler/bundler.go
• pkg/bundler/bundler_test.go
• pkg/client/v1/bundle.go
• pkg/client/v1/bundle_test.go
• pkg/defaults/timeouts.go
• pkg/server/bundle_handler.go
• pkg/server/bundle_handler_test.go
• pkg/server/serve.go
• pkg/server/signing.go
• pkg/server/signing_integration_test.go
• pkg/server/signing_test.go
• tests/chainsaw/signing/server-bundle-attestation-vault/README.md
• tests/chainsaw/signing/server-bundle-attestation-vault/run.sh

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/server/serve.go`:
- Around line 111-114: Update the parseSigningConfig error path in serve to
return the existing err directly instead of wrapping it with errors.Wrap.
Preserve the structured error code and message produced by parseSigningConfig.

In `@pkg/server/signing.go`:
- Around line 66-80: Move the parseTLogUpload call out of the unconditional
start of parseSigningConfig and into the KMS signing branch guarded by the
existing hasKMS check. Preserve error propagation when KMS mode is enabled,
while ignoring malformed or stray AICR_TLOG_UPLOAD values when no signing mode
is configured or keyless mode is selected.
- Around line 221-274: Update loadBinaryAttestation to use
aicrerrors.PropagateOrWrap when handling errors returned by verify(ctx),
preserving existing structured error codes while defaulting uncoded failures to
ErrCodeInternal with the current context message.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 451ed38b-bd88-4855-bbb8-266fa12b404a

📥 Commits

Reviewing files that changed from the base of the PR and between 8f45f19 and 78c7d7e.

📒 Files selected for processing (19)

• .github/workflows/server-kms-e2e.yaml
• .gitignore
• .goreleaser.yaml
• api/aicr/v1/server.yaml
• docs/contributor/api-server.md
• docs/user/api-reference.md
• pkg/bundler/bundler.go
• pkg/bundler/bundler_test.go
• pkg/client/v1/bundle.go
• pkg/client/v1/bundle_test.go
• pkg/defaults/timeouts.go
• pkg/server/bundle_handler.go
• pkg/server/bundle_handler_test.go
• pkg/server/serve.go
• pkg/server/signing.go
• pkg/server/signing_integration_test.go
• pkg/server/signing_test.go
• tests/chainsaw/signing/server-bundle-attestation-vault/README.md
• tests/chainsaw/signing/server-bundle-attestation-vault/run.sh

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.goreleaser.yaml:
- Around line 112-137: Extract the duplicated cosign attest-blob retry and
backoff loop from the aicr and aicrd post-build hooks into a shared script such
as hack/cosign-attest-blob.sh that accepts the bundle path and required signing
inputs. Update both hooks to invoke the shared script while preserving their
distinct bundle destinations and aicrd kodata staging behavior.

In `@pkg/server/signing.go`:
- Around line 227-262: Update defaultBinaryAttestationVerifier to read
attestPath once into a bounded buffer, then reuse that exact buffer for both
verifier.VerifyBinaryAttestation and the returned cached data. Remove the
separate os.Open/io.ReadAll path after verification while preserving the
existing size-limit and error handling behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: c24126c7-7fde-4ba2-ae7c-a548451ff56e

📥 Commits

Reviewing files that changed from the base of the PR and between c621912 and 20a021c.

📒 Files selected for processing (20)

• .github/workflows/server-kms-e2e.yaml
• .gitignore
• .goreleaser.yaml
• THIRD_PARTY_NOTICES.md
• api/aicr/v1/server.yaml
• docs/contributor/api-server.md
• docs/user/api-reference.md
• pkg/bundler/bundler.go
• pkg/bundler/bundler_test.go
• pkg/client/v1/bundle.go
• pkg/client/v1/bundle_test.go
• pkg/defaults/timeouts.go
• pkg/server/bundle_handler.go
• pkg/server/bundle_handler_test.go
• pkg/server/serve.go
• pkg/server/signing.go
• pkg/server/signing_integration_test.go
• pkg/server/signing_test.go
• tests/chainsaw/signing/server-bundle-attestation-vault/README.md
• tests/chainsaw/signing/server-bundle-attestation-vault/run.sh

[mchmarny]

CHANGES REQUESTED — 2 merge blockers. Reviewed server signing, binary-provenance staging, handler and client wiring, and the KMS E2E at the exact head. Focused race tests, affected-package lint, shell and workflow validation all passed, and the current live E2E executed full mode; the two fail-open paths identified inline still need correction.