Skip to content

fix(asr): preserve speaker boundaries in corpus cpWER - #16297

Open
rohan-patnaik wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
rohan-patnaik:fix/corpus-cpwer-aggregation
Open

rohan-patnaik wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
rohan-patnaik:fix/corpus-cpwer-aggregation

Conversation

@rohan-patnaik

@rohan-patnaik rohan-patnaik commented Sep 25, 2026 •

Copy link
Copy Markdown

Important

An outdated branch does not block merging; use Update branch only after consulting the automation team.

What does this PR do ?

Fixes #16093 by computing corpus cpWER from speaker-aware edit/reference counts instead of joined transcripts and ordinary WER.

Collection: ASR (speaker tasks)

Changelog

  • Preserve session cpWER edit/reference counts.
  • Sum counts before division for speaker boundaries and length weighting.
  • Use the result for OfflineDiarWithASR.evaluate()'s average_cpWER.

Usage

No API change.

GitHub Actions CI

170 targeted tests and pre-commit pass. The evaluator regression fails on base (0.0 instead of 0.5); a 500-corpus randomized probe matches brute-force scoring.

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install?
    • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

  • New Feature
  • Bugfix
  • Documentation

Additional Information

Revisits closed, unmerged PR #16094 and follows the cpWER semantics introduced in #15573.

Signed-off-by: Rohan Patnaik <rohan-patnaik@users.noreply.github.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 25, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the ASR label Sep 25, 2026
@svcnvidia-nemo-ci svcnvidia-nemo-ci added the waiting-on-maintainers Waiting on maintainers to respond label Sep 27, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ASR community-request waiting-on-maintainers Waiting on maintainers to respond

2 participants