An Empirical Analysis of Deep Audio-Visual Models for Speech Recognition [article]

Devesh Walawalkar, Yihui He, Rohit Pillai
2018 arXiv   pre-print
In this project, we worked on speech recognition, specifically predicting individual words based on both the video frames and audio. Empowered by convolutional neural networks, the recent speech recognition and lip reading models are comparable to human level performance. We re-implemented and made derivations of the state-of-the-art model. Then, we conducted rich experiments including the effectiveness of attention mechanism, more accurate residual network as the backbone with pre-trained
more » ... ts and the sensitivity of our model with respect to audio input with/without noise.
arXiv:1812.09336v1 fatcat:grljy67llre2lo3fpqx5tdheiu