News
- 19 Oct 2026The test sets and evaluation code are now publicly available.
- 6 Oct 2026The challenge Program is now available.
- 6 Oct 2026The NVVSpeech Challenge 2026 leaderboard is available online.
- 3 Sep 2026Accepted papers are now listed on the ISCSLP 2026 website.
- 29 Jul 2026Deadlines extended and CodaBench updated — see Challenge Timeline and email.
- 26 Jul 2026The NVVSpeech Challenge 2026 leaderboard is now open on CodaBench.
- 23 Jul 2026The evaluation sets have been sent via email to the registered teams for their respective tracks.
- 8 Jul 2026Baseline systems for Track 1 and Track 2 are now available on GitHub.
- 23 Jun 2026A dataset resources page has been added, listing publicly available nonverbal speech datasets.
- 22 Jun 2026The challenge website is live and registration is now open via the Google and Tencent forms below.
- 15 Jun 2026The NVVSpeech Challenge has been accepted as a special challenge session at ISCSLP 2026.
Overview
Background and Motivation
Human spoken communication extends beyond linguistic content. In natural conversations, speakers frequently produce nonverbal vocalizations (NVVs), such as laughter, sighs, crying, coughing, breathing, gasps, yawns, and other paralinguistic sounds. These vocalizations convey emotion, attitude, feedback, turn-taking cues, social intent, and, in some cases, physiological states. However, most current spoken language processing systems remain largely transcript-centric: automatic speech recognition systems often omit or normalize nonverbal vocalizations, while text-to-speech systems usually focus on generating fluent linguistic content without reliably modeling or controlling such expressive vocal events.
Recent progress in speech language models, expressive speech synthesis, and multimodal conversational agents makes it timely to revisit speech with nonverbal vocalizations as a core spoken language processing problem. For understanding, systems need to detect, classify, and locate nonverbal vocalizations relative to the accompanying transcript. For generation, systems need to synthesize speech that naturally integrates specified nonverbal vocalizations while preserving speech intelligibility, naturalness, and acoustic continuity between verbal and nonverbal segments.
NVVSpeech challenge aims to promote nonverbal vocalizations as a crucial yet under-explored component of spoken language communication. It defines shared task settings for both Mandarin Chinese and English, provides held-out test sets, baseline implementations, and standardized evaluation protocols, and enables the community to examine the current capability boundaries of speech understanding and generation systems with nonverbal vocalizations. NVVSpeech will help identify remaining technical bottlenecks and establish more reproducible comparisons across systems, while encouraging research on paralinguistic speech understanding, expressive and controllable speech generation, speech language models, and evaluation methods for more human-like spoken interaction.
Citation
If you use the challenge resources or refer to the challenge findings in your work, please cite our challenge overview paper:
@article{xue2026nvvspeech,
title = {The {NVVSpeech} Challenge 2026: Overview and Lessons for Speech Understanding and Generation with Nonverbal Vocalizations},
author = {Xue, Liumeng and Cao, Yuang and Hu, Jingbin and Zhan, Qirui and Chen, Huakang and Zheng, Yibin and Chen, Yunxiang and Liu, Houdun and Feng, Su and Wu, Haibin and Lian, Zheng and Xue, Wei and Wu, Bengu and Bu, Hui and Xie, Lei},
journal = {arXiv preprint arXiv:2610.16076},
year = {2026},
doi = {10.48550/arXiv.2610.16076},
url = {https://arxiv.org/abs/2610.16076}
}
Evaluation Framework
The evaluation framework behind this challenge is described in the NVV-SuperBench paper. If you use its evaluation protocol, please also cite:
@article{xue2026nvvsuperbench,
title = {{NVV-SuperBench}: Beyond Words, Beyond Quality---Benchmarking Nonverbal Vocalizations in Speech Generation},
author = {Xue, Liumeng and Bian, Weizhen and Pan, Jiahao and Wu, Wenxuan and Ren, Yilin and Kang, Boyi and Hu, Jingbin and Ma, Ziyang and Wang, Shuai and Qian, Xinyuan and Lee, Hung-yi and Guo, Yike},
journal = {arXiv preprint arXiv:2604.16211},
year = {2026}
}