Parallelized Training of Restricted Boltzmann Machines using Markov-Chain Monte Carlo Methods [article]

Pei Yang, Srinivas Varadharajan, Lucas A. Wilson, Don D. Smith II, John A Lockman III, Vineet Gundecha, Quy Ta
<span title="2019-10-14">2019</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
Restricted Boltzmann Machine (RBM) is a generative stochastic neural network that can be applied to collaborative filtering technique used by recommendation systems. Prediction accuracy of the RBM model is usually better than that of other models for recommendation systems. However, training the RBM model involves Markov-Chain Monte Carlo (MCMC) method, which is computationally expensive. In this paper, we have successfully applied distributed parallel training using Horovod framework to
more &raquo; ... the training time of the RBM model. Our tests show that the distributed training approach of the RBM model has a good scaling efficiency. We also show that this approach effectively reduces the training time to little over 12 minutes on 64 CPU nodes compared to 5 hours on a single CPU node. This will make RBM models more practically applicable in recommendation systems.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1910.05885v1">arXiv:1910.05885v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/7ns5uqhctjexna2dq6gta5hgxy">fatcat:7ns5uqhctjexna2dq6gta5hgxy</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20200901211750/https://arxiv.org/pdf/1910.05885v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/99/00/99008adbd19f01030090c9e79fd82fd9de3a02ed.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/1910.05885v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>