Skip to content

fix(nightly): unblank CONTAINER_IMAGE in GitLab trigger + forward ARTIFACTORY_REPO_NAME#973

Merged
ajcasagrande merged 2 commits into
mainfrom
harrison/increase-release-outputs
May 21, 2026
Merged

fix(nightly): unblank CONTAINER_IMAGE in GitLab trigger + forward ARTIFACTORY_REPO_NAME#973
ajcasagrande merged 2 commits into
mainfrom
harrison/increase-release-outputs

Conversation

@saturley-hall

@saturley-hall saturley-hall commented May 21, 2026

Copy link
Copy Markdown
Member

Summary

  • Last night's nightly (run 26219832007) successfully triggered the GitLab compliance pipeline, but several of the variables forwarded to it were blank — most importantly CONTAINER_IMAGE.
  • Root cause: GitHub Actions silently blanks any cross-job output whose value contains a repo secret. stage-nightly-container was emitting container_image="${NGC_STAGING_IMAGE_BASE}/aiperf-nightly:${CONTAINER_TAG}" as a job output; because NGC_STAGING_IMAGE_BASE is a secret, GitHub stripped it with the annotation Skip output 'container_image' since it may contain secret. The downstream trigger-gitlab-security-scan then read needs.stage-nightly-container.outputs.container_image and got an empty string, so the GitLab POST sent variables[CONTAINER_IMAGE]= (verified against the env dump from the run: CONTAINER_IMAGE: blank, everything else populated).
  • Fix: drop the unused job output and reconstruct the image string inline inside the trigger job from the same secret + non-secret container_tag. This is the same value, but read directly from secrets.NGC_STAGING_IMAGE_BASE (which is masked in logs but not stripped from form-data the way cross-job outputs are).
  • Also forward ARTIFACTORY_REPO_NAME (already exists in the automated-release environment) so the GitLab side no longer has to derive it from ARTIFACTORY_URL.

Test plan

  • Trigger the nightly via workflow_dispatch from main against this branch's workflow file (released to main first, or via temporary push to main behind a feature flag).
  • Confirm the Stage Container (ECR → NGC Staging) job no longer emits the Skip output 'container_image' since it may contain secret. annotation.
  • Confirm in the Trigger GitLab pipeline step that CONTAINER_IMAGE resolves to the full <NGC>/aiperf-nightly:<container_tag> string (it will be masked in logs as ***/aiperf-nightly:<tag>, which is the expected behavior — only the secret prefix is hidden).
  • On the GitLab side, confirm the triggered compliance pipeline now sees non-empty CONTAINER_IMAGE and ARTIFACTORY_REPO_NAME variables.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes

    • Fixed potential security issue by preventing sensitive container image values from being exported between CI/CD workflow jobs
  • Chores

    • Updated container build workflow to reconstruct parameters directly within jobs instead of using exported outputs
    • Modified security scan job configuration to handle additional staging and repository parameters

Review Change Stack

…IFACTORY_REPO_NAME

GitHub Actions blanks cross-job outputs whose values contain a repo secret
("Skip output '<name>' since it may contain secret."), so the previous
nightly forwarded an empty CONTAINER_IMAGE to the GitLab compliance
pipeline. Reconstruct the image string in the trigger job from
NGC_STAGING_IMAGE_BASE + container_tag and drop the now-unused job output.
Also forward ARTIFACTORY_REPO_NAME so GitLab no longer has to derive it.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com>
@github-actions

github-actions Bot commented May 21, 2026

Copy link
Copy Markdown

Try out this PR

Quick install:

pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@6c41c2defd24d7b1ce0dde472ce8081fb5d6c063

Recommended with virtual environment (using uv):

uv venv --python 3.12 && source .venv/bin/activate
uv pip install --upgrade --force-reinstall git+https://github.com/ai-dynamo/aiperf.git@6c41c2defd24d7b1ce0dde472ce8081fb5d6c063

Last updated for commit: 6c41c2dBrowse code

@github-actions github-actions Bot added the fix label May 21, 2026
@coderabbitai

coderabbitai Bot commented May 21, 2026

Copy link
Copy Markdown
Contributor

Walkthrough

The nightly workflow is updated to prevent unintended secret blanking across job outputs. The staging job stops exporting a container_image output containing secrets, and the GitLab security-scan trigger job receives additional parameters to locally reconstruct the container image path instead.

Changes

Nightly Workflow Secret Handling

Layer / File(s) Summary
Remove container image from staging job outputs
.github/workflows/nightly.yml
The stage-nightly-container job removes its outputs.container_image declaration and the copy step stops emitting that output. Comments explain that the secret-containing CONTAINER_IMAGE is intentionally not exported because GitHub blanks cross-job outputs with secrets.
Update GitLab trigger to reconstruct image locally
.github/workflows/nightly.yml
The trigger-gitlab-security-scan job receives staging and artifactory parameters (NGC_STAGING_IMAGE_BASE, RESOLVED_SHA, TIMESTAMP, STAGE_SUBPATH, ARTIFACTORY_REPO_NAME), reconstructs CONTAINER_IMAGE from components in the script, and passes ARTIFACTORY_REPO_NAME to the GitLab pipeline trigger.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

🐰 A secret should never cross the job line,
So we build it fresh where it's safe and fine,
Stage and scan now reconstruct with care,
No blanked outputs, just images to share!
Secrets stay hidden, the workflow flows clear. ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main changes: fixing the CONTAINER_IMAGE blanking issue in the GitLab trigger and forwarding ARTIFACTORY_REPO_NAME.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
.github/workflows/nightly.yml (1)

1025-1031: ⚖️ Poor tradeoff

Consider centralizing the non-secret part of the image path.

The literal aiperf-nightly:${CONTAINER_TAG} here duplicates the same construction in stage-nightly-container at line 889/893/898-900. The repo-name segment and tag are not secrets, so only the NGC_STAGING_IMAGE_BASE prefix is what GitHub Actions blanks on cross-job outputs — you could safely emit the suffix (e.g., image_suffix: aiperf-nightly:${{ steps.copy.outputs.tag }} or even just rely on needs.prepare-nightly.outputs.container_tag plus a shared local constant) as a stage-nightly-container output so the staging job remains the single source of truth for the NGC repo path. Today, if aiperf-nightly is ever renamed (e.g., to track a PyPI dist-name change), both jobs must be updated in lockstep.

Not blocking — the comment at lines 902-907 documents the intent, and the duplication is just two short occurrences in the same file.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/nightly.yml around lines 1025 - 1031, Centralize the
non-secret image suffix by emitting it from the staging job and using it when
reconstructing CONTAINER_IMAGE: add an output (e.g., image_suffix:
"aiperf-nightly:${{ steps.copy.outputs.tag }}" or use the existing
needs.prepare-nightly.outputs.container_tag combined with a shared constant) in
the stage-nightly-container job, then replace the hardcoded
"aiperf-nightly:${CONTAINER_TAG}" in the downstream job with that output
(referring to CONTAINER_IMAGE, NGC_STAGING_IMAGE_BASE, stage-nightly-container,
and container_tag/image_suffix) so the repo-name+tag are defined in one place
and only NGC_STAGING_IMAGE_BASE remains secret-prefixed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.github/workflows/nightly.yml:
- Around line 1025-1031: Centralize the non-secret image suffix by emitting it
from the staging job and using it when reconstructing CONTAINER_IMAGE: add an
output (e.g., image_suffix: "aiperf-nightly:${{ steps.copy.outputs.tag }}" or
use the existing needs.prepare-nightly.outputs.container_tag combined with a
shared constant) in the stage-nightly-container job, then replace the hardcoded
"aiperf-nightly:${CONTAINER_TAG}" in the downstream job with that output
(referring to CONTAINER_IMAGE, NGC_STAGING_IMAGE_BASE, stage-nightly-container,
and container_tag/image_suffix) so the repo-name+tag are defined in one place
and only NGC_STAGING_IMAGE_BASE remains secret-prefixed.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 59f843c9-a08f-4e08-98cc-d0383a92ad96

📥 Commits

Reviewing files that changed from the base of the PR and between 83bdcd9 and b7b4c75.

📒 Files selected for processing (1)
  • .github/workflows/nightly.yml
@codecov

codecov Bot commented May 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@ajcasagrande ajcasagrande enabled auto-merge (squash) May 21, 2026 21:20
@ajcasagrande ajcasagrande merged commit fa0171e into main May 21, 2026
24 checks passed
@ajcasagrande ajcasagrande deleted the harrison/increase-release-outputs branch May 21, 2026 21:20
saturley-hall added a commit that referenced this pull request May 21, 2026
… trigger

Match the existing ARTIFACTORY_PYPI_URL secret naming so the PyPI-repo
scoping is explicit in both the GitHub secret reference and the GitLab
variable forwarded by the nightly trigger.

Follow-up to #973.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3 participants