Sequence-to-sequence Singing Voice Synthesis with Perceptual Entropy Loss [article]

Jiatong Shi, Shuai Guo, Nan Huo, Yuekai Zhang, Qin Jin
2021 arXiv   pre-print
The neural network (NN) based singing voice synthesis (SVS) systems require sufficient data to train well and are prone to over-fitting due to data scarcity. However, we often encounter data limitation problem in building SVS systems because of high data acquisition and annotation costs. In this work, we propose a Perceptual Entropy (PE) loss derived from a psycho-acoustic hearing model to regularize the network. With a one-hour open-source singing voice database, we explore the impact of the
more » ... loss on various mainstream sequence-to-sequence models, including the RNN-based, transformer-based, and conformer-based models. Our experiments show that the PE loss can mitigate the over-fitting problem and significantly improve the synthesized singing quality reflected in objective and subjective evaluations.
arXiv:2010.12024v2 fatcat:wldsjyquwrb5no7ha6bqa3ysie