Skip to content

fix(asr): omit empty words from timestamp offsets - #16323

Open
patelnav wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
patelnav:fix/empty-word-timestamps
Open

patelnav wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
patelnav:fix/empty-word-timestamps

Conversation

@patelnav

Copy link
Copy Markdown

What does this PR do?

Omit empty word offsets from get_words_offsets. Parakeet TDT v3 can emit ▁ followed by <unk>, which creates a timestamped word: "" entry.

Collection: ASR

Changelog

  • Filter empty or whitespace-only words after offset calculation. Preserve all nonempty entries, including <unk> text and its timestamps.
  • Add seven cases with a real SentencePiece tokenizer. Six fail before the fix; all pass after it.

Validation

  • pytest --confcutdir=tests/collections/asr/decoding tests/collections/asr/decoding/test_word_offsets.py -q: 7 passed on CPU. The flag avoids unrelated test-data downloads.
  • pre-commit run --from-ref origin/main --to-ref HEAD: passed.
  • Replayed four captured Parakeet outputs with the checkpoint tokenizer. All nonempty entries are unchanged.

Full GPU suite not run. No API or configuration changes; no documentation update needed.

Signed-off-by: Nav Patel <navrpatel@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 30, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

1 participant