Joint Singing Voice Separation and F0 Estimation with Deep U-Net Architectures

Andreas Jansson, Rachel M. Bittner, Sebastian Ewert, Tillman Weyde
2019 2019 27th European Signal Processing Conference (EUSIPCO)  
Citation: Jansson, A., Bittner, R. M., Ewert, S. and Weyde, T. ORCID: 0000-0001-8028-9905 (2019). Joint singing voice separation and F0 estimation with deep U-net architectures. Abstract-Vocal source separation and fundamental frequency estimation in music are tightly related tasks. The outputs of vocal source separation systems have previously been used as inputs to vocal fundamental frequency estimation systems; conversely, vocal fundamental frequency has been used as side information to
more » ... ve vocal source separation. In this paper, we propose several different approaches for jointly separating vocals and estimating fundamental frequency. We show that joint learning is advantageous for these tasks, and that a stacked architecture which first performs vocal separation outperforms the other configurations considered. Furthermore, the best joint model achieves state-of-the-art results for vocal-f0 estimation on the iKala dataset. Finally, we highlight the importance of performing polyphonic, rather than monophonic vocal-f0 estimation for many real-world cases.
doi:10.23919/eusipco.2019.8902550 dblp:conf/eusipco/JanssonBEW19 fatcat:dnznsspqwzgjli6yte6sd7hlxm