docs: train a NeMo ASR model on Nebius Serverless Jobs

[SalikovAlex]

What does this PR do ?

Adds a self-contained, single-GPU Conformer CTC training tutorial for Nebius Serverless Jobs, linked from the cloud tutorial index. The default 50-step AN4 run exports a .nemo model, reloads it for evaluation/transcription, and publishes verifiable artifacts to Object Storage.

Collection: ASR / cloud tutorials

Changelog

• Add tutorials/cloud/nebius/ with a small Conformer config, training script, CLI run config, source-context exclusions and an end-to-end README.
• Pin the Linux amd64 manifest of the supported NeMo Speech 26.07.00 container and SHA-256 of the public AN4 archive. Convert AN4 SPHERE files with SoundFile, avoiding a system SoX dependency.
• Bound training to 50 steps by default (maximum 1000) and set a one-hour provider timeout. Include prerequisites, logs, cancellation, artifact retrieval, independent reload, failures and cleanup.
• Stage model exports locally; publish hashes after successful reload/evaluation and copying. Reject occupied output directories and do not publish success on failed copies.
• Add six CPU tests for checksum rejection and output-publication failure/success behavior. No NeMo core, scheduler, endpoint or multi-node changes.

Usage

cd tutorials/cloud/nebius
nebius ai job run train.py --show-context
# Set a unique name and an existing Object Storage bucket before executing:
nebius ai job run train.py --name "$JOB_NAME" --output "$OUTPUT_BUCKET_ID" --timeout 1h -- --max-steps 50

Validation

Passed locally:

• python -m unittest discover -s tutorials/cloud/nebius -p test_train.py -v: six tests (SoundFile 0.13.1, Python 3.11.15).
• Actual public AN4 archive checksum verification and complete conversion: 948 train / 130 test records; every WAV path, mono 16 kHz format, duration and transcript alphabet checked.
• OmegaConf 2.3.0 resolution of all model interpolations after manifest substitution; Python syntax and relative Markdown links checked.
• nebius ai job run ... --show-context with CLI 0.12.279: exactly train.py and conformer.yaml, approximately 3.5 KiB compressed.
• Registry manifest inspection confirmed the pinned Linux amd64 digest for NeMo Speech 26.07.00.
• pre-commit run and pre-commit run --from-ref origin/main --to-ref HEAD: all applicable hooks passed.
• git diff --check; author and DCO sign-off match.

Live validation passed (2026-09-28, eu-north1):

• One H100 80 GB, exactly 50 optimizer steps, 948 AN4 training utterances and evaluation on all 130 test utterances. Job reached COMPLETED.
• Exported .nemo model reloaded successfully; test loss 59.0578727722168, WER 1.0 and empty sample prediction. This is a pipeline smoke test from random weights, not an accuracy benchmark.
• Six artifacts downloaded independently from Object Storage; every SHA-256 matched. model.nemo was 8,509,440 bytes.
• A separate fresh-H100 Job read the persisted artifacts through a read-only mount, verified all six hashes, restored the model, confirmed finite weights and reproduced the sample transcription. It also reached COMPLETED.
• Runtime: NeMo 3.0.0, PyTorch 2.12.0+cu132, CUDA 13.2, NVIDIA driver 580.173.02.
• Direct Compute lookups confirmed deletion of all four test VMs and boot disks, including two failed iterations. Test output objects and Job records were removed after preserving local evidence.

Live testing fixed native bucket-ID submission, configuration interpolation after dataset-section detachment, and serialization of NeMo Hypothesis text. The configuration regression was reproduced before the fix; both regression cases are covered by local tests.

Not tested: model accuracy, multi-node, training resumption, cancellation or provider timeout. Observed normal/failure cleanup is not a universal guarantee. No standalone local Docker execution was performed; container execution and independent reload ran on Nebius.

GitHub Actions CI

This is an external contribution. DCO Signed-off-by is present; it is separate from the cryptographic-signature/trusted-committer gate. A maintainer may need to authorize CI with /ok to test <head-sha>. Local checks above do not claim upstream CI passed.

Before your PR is "Ready for review"

Pre checks:

• Read and followed the contributor guidelines.
• Added necessary local tests.
• Added documentation and the cloud index link.
• Live GPU acceptance checks described above.
• Does the PR affect components that are optional to install? (No core dependency changes; uses the pinned container.)

PR Type:

• New Feature
• Bugfix
• Documentation

Additional Information

Current contribution instructions do not require a preliminary issue. A repository-wide Nebius issue/PR search found no duplicate Serverless tutorial. This contribution was prepared with AI assistance and checked locally; the unrun validation is listed explicitly above.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.