[Security] Fix image EXIF orientation and tRNS transparency handling#44974
Merged
DarkLight1337 merged 3 commits intoJun 9, 2026
Merged
Conversation
Normalize EXIF orientation via ImageOps.exif_transpose immediately
after Image.open so the pixel data the model sees matches how humans
view the image. Without this, rotated JPEG images introduce
interpretation bias.
Detect tRNS transparency in P/L/RGB-mode PNGs before converting to
RGB. Previously only RGBA images were composited against a
background; P/L/RGB images with tRNS chunks fell through to a bare
image.convert("RGB") which discards transparency semantics, making
hidden overlay elements visible and distorting model input.
Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
Signed-off-by: jperezde <jperezde@redhat.com>
jperezdealgaba
requested review from
DarkLight1337,
NickLucche,
tjtanaa and
ywang96
as code owners
github-actions
Bot
added
the
ready
Contributor
Author
|
@DarkLight1337 It seems it hit a timeout error unrelated to the MR, right? |
ekagra-ranjan
pushed a commit
to ekagra-ranjan/vllm
that referenced
this pull request
Jun 9, 2026
…llm-project#44974) Signed-off-by: jperezde <jperezde@redhat.com> Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
waqahmed-amd-fi
pushed a commit
to waqahmed-amd-fi/vllm
that referenced
this pull request
Jun 10, 2026
…llm-project#44974) Signed-off-by: jperezde <jperezde@redhat.com> Signed-off-by: Waqar Ahmed <waqar.ahmed@amd.com>
Saddss
pushed a commit
to Saddss/vllm
that referenced
this pull request
Jun 14, 2026
…llm-project#44974) Signed-off-by: jperezde <jperezde@redhat.com>
vivek8123
pushed a commit
to odh-on-pz/vllm-upstream
that referenced
this pull request
Jun 18, 2026
…llm-project#44974) Signed-off-by: jperezde <jperezde@redhat.com>
divineearthly
pushed a commit
to divineearthly/vllm
that referenced
this pull request
Jun 19, 2026
…llm-project#44974) Signed-off-by: jperezde <jperezde@redhat.com> Signed-off-by: divineearthly <divineearthly@gmail.com>
tunglinwood
pushed a commit
to tunglinwood/vllm
that referenced
this pull request
Jun 22, 2026
…llm-project#44974) Signed-off-by: jperezde <jperezde@redhat.com>
nkzhenhua
pushed a commit
to nkzhenhua/vllm
that referenced
this pull request
Jun 24, 2026
…llm-project#44974) Signed-off-by: jperezde <jperezde@redhat.com>
ohsono
pushed a commit
to ohsono/vllm
that referenced
this pull request
Jul 3, 2026
…llm-project#44974) Signed-off-by: jperezde <jperezde@redhat.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Fix image preprocessing issues where EXIF orientation is not normalized and PNG tRNS transparency is not properly composited before RGB conversion, causing the model to see pixel data that diverges from how humans view the image.
EXIF orientation: ImageOps.exif_transpose was never called after Image.open, so JPEG images with EXIF orientation tags (e.g. 180-degree rotation) were fed to the model with raw pixel data that doesn't match the visual display, introducing interpretation bias.
tRNS transparency: Only RGBA → RGB conversion was explicitly composited against a background via rgba_to_rgb. P-mode, L-mode, and RGB images carrying a tRNS chunk fell through to a bare image.convert("RGB"), which discards transparency semantics — making otherwise hidden overlay elements visible and distorting model input. This is analogous to the AlphaDog attack class.
Changes:
Add normalize_image() using ImageOps.exif_transpose and call it immediately after Image.open in ImageMediaIO.load_bytes
Add _has_transparency() helper that detects RGBA/LA/PA modes and tRNS in image info
Update convert_image_mode() to promote any image with transparency to RGBA and composite against the background color before converting to RGB
Pass the user-configured rgba_background_color through _convert_image_mode to convert_image_mode for consistency
Test Plan
python -m pytest tests/multimodal/test_image.py -vTest Result
tests/multimodal/test_image.py::test_rgb_to_rgb PASSED
tests/multimodal/test_image.py::test_rgba_to_rgb PASSED
tests/multimodal/test_image.py::test_palette_with_trns_to_rgb PASSED
tests/multimodal/test_image.py::test_l_mode_no_trns_to_rgb PASSED
tests/multimodal/test_image.py::test_exif_transpose_normalizes_orientation PASSED
tests/multimodal/test_image.py::test_normalize_image_no_exif PASSED
======================== 6 passed in 1.01s =========================