Track 1 · Understanding
Detecting and transcribing non-verbal vocalizations in speech.
Objective Ranking
F1 · mNTD · CER/WERFinal score is the mean of the ZH and EN scores.
- 🥇 1st Moss-Transcribe OpenMOSS 66.39
- 🥈 2nd AcoustiX yunn1 64.80
- 🥉 3rd XingChenAGI asr-student 63.72
| # | Team | ZH ↑ | EN ↑ | Bilingual ↑ | Status |
|---|---|---|---|---|---|
| 🥇 1 | Moss-TranscribeOpenMOSS | 65.82 | 66.96 | 66.39 | Qualified |
| 🥈 2 | AcoustiXyunn1 | 66.02 | 63.58 | 64.80 | Qualified |
| 🥉 3 | XingChenAGIasr-student | 62.87 | 64.55 | 63.72 | Qualified |
| 4 | SighLoggerjiasy1 | 62.94 | 64.12 | 63.53 | Qualified |
| 5 | Goldennv2026 | 51.48 | 48.02 | 49.75 | Qualified |
| – | NVV-GenOrganizers | 34.05 | 32.59 | 33.32 | Baseline |
| – | Gemini 3.1 Flash TTS Preview | 53.04 | 52.70 | 52.87 | Commercial |
Per-language metric breakdown (top 5 + non-competing systems)
| System | ZH F1 ↑ | ZH mNTD ↓ | ZH CER ↓ | EN F1 ↑ | EN mNTD ↓ | EN WER ↓ |
|---|---|---|---|---|---|---|
| Moss-TranscribeOpenMOSS | 0.6384 | 0.3644 | 0.1009 | 0.6530 | 0.3869 | 0.1730 |
| AcoustiXyunn1 | 0.6364 | 0.3412 | 0.0961 | 0.6185 | 0.3988 | 0.1758 |
| XingChenAGIasr-student | 0.6202 | 0.3592 | 0.0961 | 0.6151 | 0.4279 | 0.1694 |
| SighLoggerjiasy1 | 0.6179 | 0.3819 | 0.1126 | 0.6178 | 0.4084 | 0.1859 |
| Goldennv2026 | 0.5002 | 0.5019 | 0.2294 | 0.4549 | 0.5847 | 0.2474 |
| Gemini 3.1 Flash TTS Preview | 0.4969 | 0.4760 | 0.2218 | 0.5096 | 0.5223 | 0.2523 |
Track 2 · Generation
Synthesizing speech with non-verbal vocalizations. Final ranking combines objective automatic scoring with a subjective listening test.
Results
Overall = 0.30·Obj + 0.70·Subj- 🥇 1st Tencent-HY someone111 55.51
- 🥈 2nd Moss-Transcribe OpenMOSS 55.20
- 🥉 3rd NVV-Gen Organizers 53.74
- 1st VocalMind Organizers 56.18 Non-competing
- 2nd Tencent-HY someone111 55.51 Official #1
- 3rd Moss-Transcribe OpenMOSS 55.20 Official #2
| # | Team | ZH Final ↑ | EN Final ↑ | Bilingual ↑ | Status |
|---|---|---|---|---|---|
| 🥇 1 | Tencent-HYsomeone111 | 54.00 | 57.02 | 55.51 | Official #1 |
| 🥈 2 | Moss-TranscribeOpenMOSS | 53.32 | 57.07 | 55.20 | Official #2 |
| 3 | NVV-GenOrganizers | 51.31 | 56.18 | 53.74 | Baseline |
| – | VocalMindOrganizers | 53.62 | 58.74 | 56.18 | Non-competing |
Final score $=0.30\times\text{Objective}+0.70\times\text{Subjective}$; Bilingual is the mean of the ZH and EN final scores. VocalMind is the organizers' post-trained system, listed for reference only — it does not enter the official award ranking. Commercial systems have objective scores only and are listed in the Objective tab. Switching to the All systems ranking view re-orders the table by score and changes no official result.
| # | Team | ZH ↑ | EN ↑ | Final ↑ | Status |
|---|---|---|---|---|---|
| 1 | Moss-TranscribeOpenMOSS | 75.97 | 73.77 | 74.87 | Qualified |
| 2 | NVV-GenOrganizers | 77.62 | 70.30 | 73.96 | Baseline |
| 3 | Tencent-HYsomeone111 | 76.22 | 69.66 | 72.94 | Qualified |
| – | Gemini 3.1 Flash TTS Preview | 83.50 | 83.89 | 83.70 | Commercial |
| – | ElevenLabs | 83.03 | 80.41 | 81.72 | Commercial |
| – | VocalMindOrganizers | 79.39 | 72.21 | 75.80 | Non-competing |
LALM automatic evaluation — $\text{Track2Score}=\operatorname{norm}(0.30A+0.25P+0.15N+0.15Q+0.15E)$; Final is the mean of ZH and EN, and forms the objective component of the Overall score. Commercial systems (Gemini 3.1 Flash TTS Preview, ElevenLabs) and VocalMind, the organizers' non-competing system, are shown for comparison; no human listening test was run for the commercial systems, and VocalMind takes no part in the official ranking, so none of them enter the Overall ranking.
LALM per-dimension ratings (1–5)
| System | Lang | A ↑ | P ↑ | N ↑ | Q ↑ | E ↑ |
|---|---|---|---|---|---|---|
| VocalMindOrganizers | ZH | 4.5975 | 4.4775 | 3.8200 | 3.0488 | 4.3100 |
| EN | 4.4488 | 3.9925 | 3.4538 | 3.1113 | 3.8063 | |
| Moss-TranscribeOpenMOSS | ZH | 4.3575 | 4.3500 | 3.7213 | 3.0463 | 4.1938 |
| EN | 4.4638 | 4.1300 | 3.5288 | 3.0750 | 3.9250 | |
| NVV-GenOrganizers | ZH | 4.5225 | 4.4063 | 3.7325 | 3.0263 | 4.2175 |
| EN | 4.3675 | 3.8963 | 3.3650 | 3.0800 | 3.7400 | |
| Tencent-HYsomeone111 | ZH | 4.5125 | 4.2938 | 3.6750 | 3.0775 | 4.0575 |
| EN | 4.3638 | 3.8438 | 3.3325 | 3.1550 | 3.6225 | |
| Gemini 3.1 Flash TTS Preview | ZH | 4.6825 | 4.7413 | 4.0275 | 2.9613 | 4.6775 |
| EN | 4.6921 | 4.7484 | 4.0063 | 3.0200 | 4.7134 | |
| ElevenLabs | ZH | 4.8350 | 4.6900 | 3.9150 | 3.0213 | 4.3838 |
| EN | 4.8075 | 4.5413 | 3.6925 | 3.0713 | 4.1613 |
A NVV Accuracy (30%) · P NVV Perceptual Effect (25%) · N Overall Naturalness (15%) · Q Overall Quality (15%) · E Overall Expression (15%). Rated by the large audio-language model.
| # | Team | ZH ↑ | EN ↑ | Bilingual ↑ | Status |
|---|---|---|---|---|---|
| 1 | Tencent-HYsomeone111 | 44.48 | 51.60 | 48.04 | Qualified |
| 2 | Moss-TranscribeOpenMOSS | 43.61 | 49.91 | 46.76 | Qualified |
| 3 | NVV-GenOrganizers | 40.03 | 50.13 | 45.08 | Baseline |
| – | VocalMindOrganizers | 42.57 | 52.96 | 47.77 | Non-competing |
Human listening test — same $\operatorname{norm}(0.30A+0.25P+0.15N+0.15Q+0.15E)$ rubric on a 0–100 scale; Bilingual is the mean of ZH and EN. VocalMind took part in the listening test but is out of competition.
Human listening per-dimension ratings (1–5, bilingual)
| System | A ↑ | P ↑ | N ↑ | Q ↑ | E ↑ | Overall ↑ |
|---|---|---|---|---|---|---|
| Tencent-HYsomeone111 | 2.835 | 2.485 | 3.233 | 3.513 | 2.919 | 2.997 |
| VocalMindOrganizers | 2.804 | 2.446 | 3.227 | 3.450 | 3.044 | 2.994 |
| Moss-TranscribeOpenMOSS | 2.773 | 2.440 | 3.204 | 3.400 | 2.921 | 2.948 |
| NVV-GenOrganizers | 2.671 | 2.327 | 3.179 | 3.352 | 2.935 | 2.893 |
A NVV Accuracy (30%) · P NVV Perceptual Effect (25%) · N Overall Naturalness (15%) · Q Overall Quality (15%) · E Overall Expression (15%). Bilingual (ZH+EN) means from the 160-item human listening test; Overall is the unweighted dimension mean.
Scoring
Track 1 · Understanding
Objective automatic evaluation only. The per-language score is:
$$\text{Track1Score}=100\times\bigl[\,0.70\,F1_{\text{micro}}+0.20\,(1-\text{mNTD})+0.10\,(1-\min(\text{Err}_{\text{tagged}},1))\,\bigr]$$- Event-based Micro F1 (0.70) — pools NVV events across categories; an event is correct only with the right category and transcript-relative position.
- mNTD (0.20) — placement error between predicted and reference NVV tags; lower is better.
- Tagged-transcript Error (0.10) — tag-aware CER (Mandarin) / WER (English), with each NVV tag treated as one atomic token.
Bilingual score is the mean of the two languages:
$$\text{FinalTrack1Score}=\tfrac{1}{2}\left(\text{Track1Score}_{\text{ZH}}+\text{Track1Score}_{\text{EN}}\right)$$Track 2 · Generation
Each system is rated on five dimensions per language:
$$\text{Track2Score}=100\times\left(0.30A+0.25P+0.15N+0.15Q+0.15E\right)$$- A · NVV Accuracy (0.30) — whether the intended NVV types, numbers, and positions are correctly realized (0 if absent / inaudible).
- P · NVV Perceptual Effect (0.25) — whether NVVs are clearly audible, recognizable, and perceptually effective.
- N · Overall Naturalness (0.15) — naturalness, fluency, and acoustic continuity of the full utterance.
- Q · Overall Quality (0.15) — perceived audio quality, including the absence of noticeable artifacts.
- E · Overall Expression (0.15) — expressive delivery coherent and appropriate for the content, target NVVs, and conversational context.
Bilingual score is the mean of the two languages:
$$\text{FinalTrack2Score}=\tfrac{1}{2}\left(\text{Track2Score}_{\text{ZH}}+\text{Track2Score}_{\text{EN}}\right)$$The five dimensions are scored two ways — automatically by a large audio-language model (LALM), the Objective score, and by a human listening test, the Subjective score. The overall ranking combines them:
$$\text{FinalScore}=0.30\times\text{Objective}+0.70\times\text{Subjective}$$Tie-break: higher NVV Accuracy, then NVV Perceptual Effect, then Overall Naturalness.
Status labels
- Qualified valid competing submission
- Baseline organizers' NVV-Gen reference
- Non-competing organizers' internal system, out of competition
- Commercial external system, out of competition
Each leaderboard has a Competition ranking / All systems ranking toggle. Competition ranking is the official award ranking: only qualified submissions and the NVV-Gen reference carry a position. All systems ranking re-orders every evaluated system — including the non-competing and commercial ones — by score, and changes no official result.