Skip to content

feat(multi-turn-bench): add api_key and custom headers for multi turn benchmark#44516

Merged
simon-mo merged 3 commits into
vllm-project:mainfrom
jimmy-evo:feat/multi_turn_benchmark_api_key
Jun 9, 2026
Merged

feat(multi-turn-bench): add api_key and custom headers for multi turn benchmark#44516
simon-mo merged 3 commits into
vllm-project:mainfrom
jimmy-evo:feat/multi_turn_benchmark_api_key

Conversation

@jimmy-evo

@jimmy-evo jimmy-evo commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Purpose

add args api-key custom-header for multi-turn benchmark.

add validation for "no-chunk" stream requests: stream request with no ttft counted.


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

@mergify mergify Bot added the performance Performance-related issues label Jun 4, 2026
@simon-mo simon-mo added the ready ONLY add when PR is ready to merge/full CI is needed label Jun 8, 2026
@simon-mo simon-mo enabled auto-merge (squash) June 8, 2026 15:40
… benchmark

Signed-off-by: Jimmy <jinmingyi1998@sina.cn>
auto-merge was automatically disabled June 8, 2026 16:14

Head branch was pushed to by a user without write access

@jimmy-evo jimmy-evo force-pushed the feat/multi_turn_benchmark_api_key branch from 6b8f6bc to 868b5d4 Compare June 8, 2026 16:14
@mergify

mergify Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Hi @jimmy-evo, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

@mergify

mergify Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @jimmy-evo.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Jun 9, 2026
…chmark_api_key

# Conflicts:
#	benchmarks/multi_turn/benchmark_serving_multi_turn.py
@mergify mergify Bot removed the needs-rebase label Jun 9, 2026
parser.add_argument(
"--api-key",
type=str,
default=None,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: LOW

The --api-key argument value will be logged in plaintext at line 1533 (logger.info(args)), which dumps the entire args namespace including api_key. This exposes the bearer token/secret credential in log output.
Helpful? Add 👍 / 👎

💡 Fix Suggestion

Suggestion: Redact the api_key field before logging the args namespace at line 1533. You can either:

  1. Create a sanitized copy of the args for logging: replace logger.info(args) with code that masks or removes the api_key value before printing, e.g.:
    args_to_log = vars(args).copy()
    if args_to_log.get('api_key'):
        args_to_log['api_key'] = '***REDACTED***'
    logger.info(args_to_log)
  2. Or simply omit the full args dump and rely on the individual parameter logging that follows it.
@simon-mo simon-mo merged commit 1c2ffc6 into vllm-project:main Jun 9, 2026
7 of 13 checks passed
@jimmy-evo jimmy-evo deleted the feat/multi_turn_benchmark_api_key branch June 10, 2026 06:16
waqahmed-amd-fi pushed a commit to waqahmed-amd-fi/vllm that referenced this pull request Jun 10, 2026
… benchmark (vllm-project#44516)

Signed-off-by: Jimmy <jinmingyi1998@sina.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: simon-mo <simon.mo@hey.com>
Signed-off-by: Waqar Ahmed <waqar.ahmed@amd.com>
Saddss pushed a commit to Saddss/vllm that referenced this pull request Jun 14, 2026
… benchmark (vllm-project#44516)

Signed-off-by: Jimmy <jinmingyi1998@sina.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: simon-mo <simon.mo@hey.com>
vivek8123 pushed a commit to odh-on-pz/vllm-upstream that referenced this pull request Jun 18, 2026
… benchmark (vllm-project#44516)

Signed-off-by: Jimmy <jinmingyi1998@sina.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: simon-mo <simon.mo@hey.com>
divineearthly pushed a commit to divineearthly/vllm that referenced this pull request Jun 19, 2026
… benchmark (vllm-project#44516)

Signed-off-by: Jimmy <jinmingyi1998@sina.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: simon-mo <simon.mo@hey.com>
Signed-off-by: divineearthly <divineearthly@gmail.com>
tunglinwood pushed a commit to tunglinwood/vllm that referenced this pull request Jun 22, 2026
… benchmark (vllm-project#44516)

Signed-off-by: Jimmy <jinmingyi1998@sina.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: simon-mo <simon.mo@hey.com>
nkzhenhua pushed a commit to nkzhenhua/vllm that referenced this pull request Jun 24, 2026
… benchmark (vllm-project#44516)

Signed-off-by: Jimmy <jinmingyi1998@sina.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: simon-mo <simon.mo@hey.com>
ohsono pushed a commit to ohsono/vllm that referenced this pull request Jul 3, 2026
… benchmark (vllm-project#44516)

Signed-off-by: Jimmy <jinmingyi1998@sina.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: simon-mo <simon.mo@hey.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Performance-related issues ready ONLY add when PR is ready to merge/full CI is needed

2 participants