LEVERAGING PUBLIC MEDIA FOR LOW-RESOURCE TEXT-TO-SPEECH: A CASE STUDY OF VIETNAMESE LANGUAGE

Nhi Pham, Erasmus Student

Department of Computer Systems and Technologies Faculty of Electrical Engineering, Electronics and Automation University of Ruse “Angel Kanchev”

E-mail: nhi.pt205190@sis.hust.edu.vn

Dr. Nguyen Thi Thu Trang, PhD

Department of Computer Science School of Information and Communications Technology Hanoi University of Science and Technology

E-mail: trangntt@soict.hust.edu.vn

Assoc. Prof. Adriana Borodzhieva, PhD

Department of Telecommunications Faculty of Electrical Engineering, Electronics and Automation University of Ruse “Angel Kanchev”

ORCID: 0000-0001-5330-4153

E-mail: aborodzhieva@uni-ruse.bg

Abstract

The performance gap between high-resource and low-resource Text-to-Speech (TTS) systems remains a significant challenge in speech synthesis. While mainstream models achieve high naturalness and intelligibility, they rely on massive amounts of high-fidelity, studio-recorded data, a requirement that is often impractical for many low-resource languages. In this work, we introduce an end-to-end data construction pipeline that leverages publicly available media for speech data collection. Using Vietnamese as a case study, we combine this data construction framework with a multilingual transfer learning approach to optimize model training efficacy. Experimental results indicate that integrating diverse, naturalistic public media significantly improves prosodic naturalness and speaker diversity compared to models trained on limited curated datasets. Our findings demonstrate a scalable, cost-effective approach for developing high-quality TTS in low-resource linguistic contexts without the need for specialized recording sessions.

Keywords

Speech Synthesis, Text-to-Speech, Low-Resource Language, Data Construction Pipeline, Transfer Learning

REFERENCES

Azzarelli, A., Gao, G., Kwan, H. M., Zhang, F., Anantrasirichai, N., Moolan-Feroze, O., and Bull, D. (2025). “ViVo: A Dataset for Volumetric Video Reconstruction and Compression”. arXiv preprint, arXiv:2506.00558.
Desplanques, B., Thienpondt, J., and Demuynck, K. (2020). “ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification”. arXiv preprint, arXiv:2005.07143.
Kong, J., Kim, J., and Bae, J. (2020). “HiFi-GAN: Generative Adversarial Networks for Efficient and High-Fidelity Speech Synthesis”. Advances in Neural Information Processing Systems, vol. 33, pp. 17022-17033, https://arxiv.org/abs/2010.05646.
Le, T.-T., Nguyen, L. T., and Nguyen, D. Q. (2024). “Phowhisper: Automatic Speech Recognition for Vietnamese”. arXiv preprint, arXiv:2406.02555.
Lux, F., Koch, J., Meyer, S., Bott, T., Schauffler, N., Denisov, P., Schweitzer, A., and Vu, N. T. (2023). “The IMS Toucan System for the Blizzard Challenge”, arXiv preprint, arXiv:2310.17499.
McInnes, L., Healy, J., and Astels, S. (2017). “hdbscan: Hierarchical Density-Based Clustering”, Journal of Open Source Software, vol. 2 (11), pp. 205, doi:10.21105/joss.00205.
Shi, Y., Bu, H., Xu, X., Zhang, S. & Li, M. (2020). “AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines”, doi: 10.48550/arXiv.2010.11567.
Zen, H., Dang, V., Clark, R., Zhang, Y., Weiss, R. Y., Jia, Y., Chen, Z., and Wu, Y. (2019). “LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech”. arXiv preprint, arXiv:1904.02882.