Filters








65 Hits in 0.97 sec

Differentially-Private Federated Linear Bandits [article]

Abhimanyu Dubey, Alex Pentland
2020 arXiv   pre-print
The rapid proliferation of decentralized learning systems mandates the need for differentially-private cooperative learning. In this paper, we study this in context of the contextual linear bandit: we consider a collection of agents cooperating to solve a common contextual bandit, while ensuring that their communication remains private. For this problem, we devise FedUCB, a multiagent private algorithm for both centralized and decentralized (peer-to-peer) federated learning. We provide a
more » ... s technical analysis of its utility in terms of regret, improving several results in cooperative bandit learning, and provide rigorous privacy guarantees as well. Our algorithms provide competitive performance both in terms of pseudoregret bounds and empirical benchmark performance in various multi-agent settings.
arXiv:2010.11425v1 fatcat:mbuav4eq7vfhbbega56pda3h4m

Coreset-Based Neural Network Compression [article]

Abhimanyu Dubey, Moitreya Chatterjee, Narendra Ahuja
2018 arXiv   pre-print
We propose a novel Convolutional Neural Network (CNN) compression algorithm based on coreset representations of filters. We exploit the redundancies extant in the space of CNN weights and neuronal activations (across samples) in order to obtain compression. Our method requires no retraining, is easy to implement, and obtains state-of-the-art compression performance across a wide variety of CNN architectures. Coupled with quantization and Huffman coding, we create networks that provide
more » ... ke accuracy, with a memory footprint that is 832× smaller than the original AlexNet, while also introducing significant reductions in inference time as well. Additionally these compressed networks when fine-tuned, successfully generalize to other domains as well.
arXiv:1807.09810v1 fatcat:r5deensjdjhfjcfdxjkrfjaopi

Coreset-Based Adaptive Tracking [article]

Abhimanyu Dubey, Nikhil Naik, Dan Raviv, Rahul Sukthankar, Ramesh Raskar
2015 arXiv   pre-print
We propose a method for learning from streaming visual data using a compact, constant size representation of all the data that was seen until a given moment. Specifically, we construct a 'coreset' representation of streaming data using a parallelized algorithm, which is an approximation of a set with relation to the squared distances between this set and all other points in its ambient space. We learn an adaptive object appearance model from the coreset tree in constant time and logarithmic
more » ... e and use it for object tracking by detection. Our method obtains excellent results for object tracking on three standard datasets over more than 100 videos. The ability to summarize data efficiently makes our method ideally suited for tracking in long videos in presence of space and time constraints. We demonstrate this ability by outperforming a variety of algorithms on the TLD dataset with 2685 frames on average. This coreset based learning approach can be applied for both real-time learning of small, varied data and fast learning of big data.
arXiv:1511.06147v1 fatcat:zujkqnzvnjgytgarjfdanibhoa

Cooperative Multi-Agent Bandits with Heavy Tails [article]

Abhimanyu Dubey, Alex Pentland
2020 arXiv   pre-print
Correspondence to: Abhimanyu Dubey <dubeya@mit.edu>. Proceedings of the 37 th International Conference on Machine Learning, Online, PMLR 119, 2020. Copyright 2020 by the author(s).  ...  Dubey & Pentland (2019) provide an algorithm for Thompson Sampling for α-stable densities (Borak et al., 2005) , at family of heavy-tailed densities typically with infinite variance.  ... 
arXiv:2008.06244v1 fatcat:2cjp4v6hrnccbdjmfdrhkc2gru

Smaller Models, Better Generalization [article]

Mayank Sharma, Suraj Tripathi, Abhimanyu Dubey, Jayadeva, Sai Guruju, Nihal Goalla
2019 arXiv   pre-print
Dubey Jayadeva arXiv:1908.11250v1 [cs.LG] 29 Aug 2019 MIT Indian  ...  eez142368@iitd.ac.in surajtripathi93@gmail.com Abhimanyu  ... 
arXiv:1908.11250v1 fatcat:lkqvczu4tjferlq2fbuual6jrm

Modeling Image Virality with Pairwise Spatial Transformer Networks [article]

Abhimanyu Dubey, Sumeet Agarwal
2017 arXiv   pre-print
The study of virality and information diffusion online is a topic gaining traction rapidly in the computational social sciences. Computer vision and social network analysis research have also focused on understanding the impact of content and information diffusion in making content viral, with prior approaches not performing significantly well as other traditional classification tasks. In this paper, we present a novel pairwise reformulation of the virality prediction problem as an attribute
more » ... diction task and develop a novel algorithm to model image virality on online media using a pairwise neural network. Our model provides significant insights into the features that are responsible for promoting virality and surpasses the existing state-of-the-art by a 12% average improvement in prediction. We also investigate the effect of external category supervision on relative attribute prediction and observe an increase in prediction accuracy for the same across several attribute learning datasets.
arXiv:1709.07914v1 fatcat:xavvzwxeqbgbnitp44bbmqmcdy

Kernel Methods for Cooperative Multi-Agent Contextual Bandits [article]

Abhimanyu Dubey, Alex Pentland
2020 arXiv   pre-print
Correspondence to: Abhimanyu Dubey <dubeya@mit.edu>. Proceedings of the 37 th International Conference on Machine Learning, Online, PMLR 119, 2020.  ... 
arXiv:2008.06220v1 fatcat:4dkbeqn3fnc5zejptl5bznyole

Evaluating Generative Adversarial Networks on Explicitly Parameterized Distributions [article]

Shayne O'Brien, Matt Groh, Abhimanyu Dubey
2018 arXiv   pre-print
The true distribution parameterizations of commonly used image datasets are inaccessible. Rather than designing metrics for feature spaces with unknown characteristics, we propose to measure GAN performance by evaluating on explicitly parameterized, synthetic data distributions. As a case study, we examine the performance of 16 GAN variants on six multivariate distributions of varying dimensionalities and training set sizes. In this learning environment, we observe that: GANs exhibit similar
more » ... formance trends across dimensionalities; learning depends on the underlying distribution and its complexity; the number of training samples can have a large impact on performance; evaluation and relative comparisons are metric-dependent; diverse sets of hyperparameters can produce a "best" result; and some GANs are more robust to hyperparameter changes than others. These observations both corroborate findings of previous GAN evaluation studies and make novel contributions regarding the relationship between size, complexity, and GAN performance.
arXiv:1812.10782v1 fatcat:vkeo3bw42vajfbewfefx7645l4

MemeSequencer: Sparse Matching for Embedding Image Macros [article]

Abhimanyu Dubey, Esteban Moro, Manuel Cebrian, Iyad Rahwan
2018 arXiv   pre-print
Approaches that take context into account, such as the work of Singh and Lee [50] , and Dubey and Agarwal [24] perform much better, however, they face challenges in decoupling text information.  ...  SkipThought Features [33] 52.95 SVM + Finetuned ResNet18 + Word2VecPooling [40] 55.68 SVM + Finetuned ResNet18 + SkipThought Features [33] 56.35 Xiao and Lee [57] 62.67 Singh and Lee [50] 61.98 Dubey  ... 
arXiv:1802.04936v1 fatcat:ratefwcsgfh45c6fhynb2gta44

Provably Efficient Cooperative Multi-Agent Reinforcement Learning with Function Approximation [article]

Abhimanyu Dubey, Alex Pentland
2021 arXiv   pre-print
., 2019; Dubey & Pentland, 2020b) , as well as in the distributed settings (Hillel et al., 2013; Wang et al., 2019) .  ...  Differentially-private algorithms have also been proposed (Dubey & Pentland, 2020b) . Reinforcement Learning with Function Approximation.  ... 
arXiv:2103.04972v1 fatcat:okyksfg4yffnrk6gs3hi6i4wkq

Examining Representational Similarity in ConvNets and the Primate Visual Cortex [article]

Abhimanyu Dubey, Jayadeva, Sumeet Agarwal
2016 arXiv   pre-print
We compare several ConvNets with different depth and regularization techniques with multi-unit macaque IT cortex recordings and assess the impact of the same on representational similarity with the primate visual cortex. We find that with increasing depth and validation performance, ConvNet features are closer to cortical IT representations.
arXiv:1609.03529v1 fatcat:igklyrtrgbf3xjt6j7oresjhj4

Maximum-Entropy Fine-Grained Classification [article]

Abhimanyu Dubey, Otkrist Gupta, Ramesh Raskar, Nikhil Naik
2018 arXiv   pre-print
Fine-Grained Visual Classification (FGVC) is an important computer vision problem that involves small diversity within the different classes, and often requires expert annotators to collect data. Utilizing this notion of small visual diversity, we revisit Maximum-Entropy learning in the context of fine-grained classification, and provide a training routine that maximizes the entropy of the output probability distribution for training convolutional neural networks on FGVC tasks. We provide a
more » ... retical as well as empirical justification of our approach, and achieve state-of-the-art performance across a variety of classification tasks in FGVC, that can potentially be extended to any fine-tuning task. Our method is robust to different hyperparameter values, amount of training data and amount of training label noise and can hence be a valuable tool in many similar problems.
arXiv:1809.05934v2 fatcat:7sisnsbvrraytjp3bkhdlybxhq

Thompson Sampling on Symmetric α-Stable Bandits [article]

Abhimanyu Dubey, Alex Pentland
2019 arXiv   pre-print
Thompson Sampling provides an efficient technique to introduce prior knowledge in the multi-armed bandit problem, along with providing remarkable empirical performance. In this paper, we revisit the Thompson Sampling algorithm under rewards drawn from symmetric α-stable distributions, which are a class of heavy-tailed probability distributions utilized in finance and economics, in problems such as modeling stock prices and human behavior. We present an efficient framework for posterior
more » ... , which leads to two algorithms for Thompson Sampling in this setting. We prove finite-time regret bounds for both algorithms, and demonstrate through a series of experiments the stronger performance of Thompson Sampling in this setting. With our results, we provide an exposition of symmetric α-stable distributions in sequential decision-making, and enable sequential Bayesian inference in applications from diverse fields in finance and complex systems that operate on heavy-tailed features.
arXiv:1907.03821v2 fatcat:adndtf42g5c3bndlf5vlso2tge

No Peek: A Survey of private distributed deep learning [article]

Praneeth Vepakomma, Tristan Swedish, Ramesh Raskar, Otkrist Gupta, Abhimanyu Dubey
2018 arXiv   pre-print
We survey distributed deep learning models for training or inference without accessing raw data from clients. These methods aim to protect confidential patterns in data while still allowing servers to train models. The distributed deep learning methods of federated learning, split learning and large batch stochastic gradient descent are compared in addition to private and secure approaches of differential privacy, homomorphic encryption, oblivious transfer and garbled circuits in the context of
more » ... neural networks. We study their benefits, limitations and trade-offs with regards to computational resources, data leakage and communication efficiency and also share our anticipated future trends.
arXiv:1812.03288v1 fatcat:7w3oheeljrgejit6dsj54ay5nq

Adaptive Methods for Aggregated Domain Generalization [article]

Xavier Thomas, Dhruv Mahajan, Alex Pentland, Abhimanyu Dubey
2021 arXiv   pre-print
Domain generalization involves learning a classifier from a heterogeneous collection of training sources such that it generalizes to data drawn from similar unknown target domains, with applications in large-scale learning and personalized inference. In many settings, privacy concerns prohibit obtaining domain labels for the training data samples, and instead only have an aggregated collection of training points. Existing approaches that utilize domain labels to create domain-invariant feature
more » ... epresentations are inapplicable in this setting, requiring alternative approaches to learn generalizable classifiers. In this paper, we propose a domain-adaptive approach to this problem, which operates in two steps: (a) we cluster training data within a carefully chosen feature space to create pseudo-domains, and (b) using these pseudo-domains we learn a domain-adaptive classifier that makes predictions using information about both the input and the pseudo-domain it belongs to. Our approach achieves state-of-the-art performance on a variety of domain generalization benchmarks without using domain labels whatsoever. Furthermore, we provide novel theoretical guarantees on domain generalization using cluster information. Our approach is amenable to ensemble-based methods and provides substantial gains even on large-scale benchmark datasets. The code can be found at: https://github.com/xavierohan/AdaClust_DomainBed
arXiv:2112.04766v2 fatcat:xsi3whmwevfitjevpivjxkr2a4
« Previous Showing results 1 — 15 out of 65 results