Deep Audio-Visual Learning: A Survey [article]

Hao Zhu, Mandi Luo, Rui Wang, Aihua Zheng, Ran He
2020 arXiv   pre-print
Audio-visual learning, aimed at exploiting the relationship between audio and visual modalities, has drawn considerable attention since deep learning started to be used successfully. Researchers tend to leverage these two modalities either to improve the performance of previously considered single-modality tasks or to address new challenging problems. In this paper, we provide a comprehensive survey of recent audio-visual learning development. We divide the current audio-visual learning tasks
more » ... to four different subfields: audio-visual separation and localization, audio-visual correspondence learning, audio-visual generation, and audio-visual representation learning. State-of-the-art methods as well as the remaining challenges of each subfield are further discussed. Finally, we summarize the commonly used datasets and performance metrics.
arXiv:2001.04758v1 fatcat:p6ph5cujl5do3pzlpvcce35nvi