Learning a Metric Embedding for Face Recognition using the Multibatch Method [article]

Oren Tadmor and Yonatan Wexler and Tal Rosenwein and Shai Shalev-Shwartz and Amnon Shashua
2016 arXiv   pre-print
This work is motivated by the engineering task of achieving a near state-of-the-art face recognition on a minimal computing budget running on an embedded system. Our main technical contribution centers around a novel training method, called Multibatch, for similarity learning, i.e., for the task of generating an invariant "face signature" through training pairs of "same" and "not-same" face images. The Multibatch method first generates signatures for a mini-batch of k face images and then
more » ... ucts an unbiased estimate of the full gradient by relying on all k^2-k pairs from the mini-batch. We prove that the variance of the Multibatch estimator is bounded by O(1/k^2), under some mild conditions. In contrast, the standard gradient estimator that relies on random k/2 pairs has a variance of order 1/k. The smaller variance of the Multibatch estimator significantly speeds up the convergence rate of stochastic gradient descent. Using the Multibatch method we train a deep convolutional neural network that achieves an accuracy of 98.2% on the LFW benchmark, while its prediction runtime takes only 30msec on a single ARM Cortex A9 core. Furthermore, the entire training process took only 12 hours on a single Titan X GPU.
arXiv:1605.07270v1 fatcat:lawhovo76jc6dac2uztpprrvqq