Filters








174 Hits in 1.3 sec

Cross-stitch Networks for Multi-task Learning [article]

Ishan Misra and Abhinav Shrivastava and Abhinav Gupta and Martial Hebert
2016 arXiv   pre-print
Multi-task learning in Convolutional Networks has displayed remarkable success in the field of recognition. This success can be largely attributed to learning shared representations from multiple supervisory tasks. However, existing multi-task approaches rely on enumerating multiple network architectures specific to the tasks at hand, that do not generalize. In this paper, we propose a principled approach to learn shared representations in ConvNets using multi-task learning. Specifically, we
more » ... pose a new sharing unit: "cross-stitch" unit. These units combine the activations from multiple networks and can be trained end-to-end. A network with cross-stitch units can learn an optimal combination of shared and task-specific representations. Our proposed method generalizes across multiple tasks and shows dramatically improved performance over baseline methods for categories with few training examples.
arXiv:1604.03539v1 fatcat:qjhwexuju5fhjg4anqoj7tmmgu

Cross-Stitch Networks for Multi-task Learning

Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, Martial Hebert
2016 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)  
Multi-task learning in Convolutional Networks has displayed remarkable success in the field of recognition. This success can be largely attributed to learning shared representations from multiple supervisory tasks. However, existing multi-task approaches rely on enumerating multiple network architectures specific to the tasks at hand, that do not generalize. In this paper, we propose a principled approach to learn shared representations in ConvNets using multitask learning. Specifically, we
more » ... ose a new sharing unit: "cross-stitch" unit. These units combine the activations from multiple networks and can be trained end-to-end. A network with cross-stitch units can learn an optimal combination of shared and task-specific representations. Our proposed method generalizes across multiple tasks and shows dramatically improved performance over baseline methods for categories with few training examples.
doi:10.1109/cvpr.2016.433 dblp:conf/cvpr/MisraSGH16 fatcat:lefrg2pwzfev5kckjntlu377ve

Growth codes

Abhinav Kamra, Vishal Misra, Jon Feldman, Dan Rubenstein
2006 Computer communication review  
Sensor networks are especially useful in catastrophic or emergency scenarios such as floods, fires, terrorist attacks or earthquakes where human participation may be too dangerous. However, such disaster scenarios pose an interesting design challenge since the sensor nodes used to collect and communicate data may themselves fail suddenly and unpredictably, resulting in the loss of valuable data. Furthermore, because these networks are often expected to be deployed in response to a disaster, or
more » ... ecause of sudden configuration changes due to failure, these networks are often expected to operate in a "zero-configuration" paradigm, where data collection and transmission must be initiated immediately, before the nodes have a chance to assess the current network topology. In this paper, we design and analyze techniques to increase "persistence" of sensed data, so that data is more likely to reach a data sink, even as network nodes fail. This is done by replicating data compactly at neighboring nodes using novel "Growth Codes" that increase in efficiency as data accumulates at the sink. We show that Growth Codes preserve more data in the presence of node failures than previously proposed erasure resilient techniques.
doi:10.1145/1151659.1159943 fatcat:3fdnnbkshndijat3y6qrckcwta

Learning by Asking Questions [article]

Ishan Misra, Ross Girshick, Rob Fergus, Martial Hebert, Abhinav Gupta, Laurens van der Maaten
2017 arXiv   pre-print
Acknowledgments: The authors would like to thank Arthur Szlam, Jason Weston, Saloni Potdar and Abhinav Shrivastava for helpful discussions and feedback on the manuscript; Soumith Chintala and Adam Paszke  ... 
arXiv:1712.01238v1 fatcat:l6j6gsfggfauvirxnbbifxcwhe

Applying artificial vision models to human scene understanding

Elissa M. Aminoff, Mariya Toneva, Abhinav Shrivastava, Xinlei Chen, Ishan Misra, Abhinav Gupta, Michael J. Tarr
2015 Frontiers in Computational Neuroscience  
How do we understand the complex patterns of neural responses that underlie scene understanding? Studies of the network of brain regions held to be scene-selective-the parahippocampal/lingual region (PPA), the retrosplenial complex (RSC), and the occipital place area (TOS)-have typically focused on single visual dimensions (e.g., size), rather than the high-dimensional feature space in which scenes are likely to be neurally represented. Here we leverage well-specified artificial vision systems
more » ... o explicate a more complex understanding of how scenes are encoded in this functional network. We correlated similarity matrices within three different scene-spaces arising from: (1) BOLD activity in scene-selective brain regions; (2) behavioral measured judgments of visually-perceived scene similarity; and (3) several different computer vision models. These correlations revealed: (1) models that relied on mid-and high-level scene attributes showed the highest correlations with the patterns of neural activity within the scene-selective network; (2) NEIL and SUN-the models that best accounted for the patterns obtained from PPA and TOS-were different from the GIST model that best accounted for the pattern obtained from RSC; (3) The best performing models outperformed behaviorally-measured judgments of scene similarity in accounting for neural data. One computer vision method-NEIL ("Never-Ending-Image-Learner"), which incorporates visual features learned as statistical regularities across web-scale numbers of scenes-showed significant correlations with neural activity in all three scene-selective regions and was one of the two models best able to account for variance in the PPA and TOS. We suggest that these results are a promising first step in explicating more fine-grained models of neural scene understanding, including developing a clearer picture of the division of labor among the components of the functional scene-selective brain network.
doi:10.3389/fncom.2015.00008 pmid:25698964 pmcid:PMC4316773 fatcat:jprzkzvqpndxnak2g7fm53gqlu

ClusterFit: Improving Generalization of Visual Representations [article]

Xueting Yan and Ishan Misra and Abhinav Gupta and Deepti Ghadiyaram and Dhruv Mahajan
2019 arXiv   pre-print
Pre-training convolutional neural networks with weakly-supervised and self-supervised strategies is becoming increasingly popular for several computer vision tasks. However, due to the lack of strong discriminative signals, these learned representations may overfit to the pre-training objective (e.g., hashtag prediction) and not generalize well to downstream tasks. In this work, we present a simple strategy - ClusterFit (CF) to improve the robustness of the visual representations learned during
more » ... pre-training. Given a dataset, we (a) cluster its features extracted from a pre-trained network using k-means and (b) re-train a new network from scratch on this dataset using cluster assignments as pseudo-labels. We empirically show that clustering helps reduce the pre-training task-specific information from the extracted features thereby minimizing overfitting to the same. Our approach is extensible to different pre-training frameworks -- weak- and self-supervised, modalities -- images and videos, and pre-training tasks -- object and action classification. Through extensive transfer learning experiments on 11 different target datasets of varied vocabularies and granularities, we show that ClusterFit significantly improves the representation quality compared to the state-of-the-art large-scale (millions / billions) weakly-supervised image and video models and self-supervised image models.
arXiv:1912.03330v1 fatcat:ydpyseofvvdkhfec27ggeshrse

Watch and Learn: Semi-Supervised Learning of Object Detectors from Videos [article]

Ishan Misra, Abhinav Shrivastava, Martial Hebert
2015 arXiv   pre-print
We present a semi-supervised approach that localizes multiple unknown object instances in long videos. We start with a handful of labeled boxes and iteratively learn and label hundreds of thousands of object instances. We propose criteria for reliable object detection and tracking for constraining the semi-supervised learning process and minimizing semantic drift. Our approach does not assume exhaustive labeling of each object instance in any single frame, or any explicit annotation of negative
more » ... data. Working in such a generic setting allow us to tackle multiple object instances in video, many of which are static. In contrast, existing approaches either do not consider multiple object instances per video, or rely heavily on the motion of the objects present. The experiments demonstrate the effectiveness of our approach by evaluating the automatically labeled data on a variety of metrics like quality, coverage (recall), diversity, and relevance to training an object detector.
arXiv:1505.05769v1 fatcat:cpshxqbebneuxgjmfyakw2k4g4

Data persistence in sensor networks

Abhinav Kamra, Jon Feldman, Vishal Misra, Dan Rubenstein
2005 Performance Evaluation Review  
doi:10.1145/1101892.1101901 fatcat:hgygbgkqs5gtjhdnjhkpj6wh5a

CountTorrent

Abhinav Kamra, Vishal Misra, Dan Rubenstein
2007 Proceedings of the 5th international conference on Embedded networked sensor systems - SenSys '07  
We study the problem of aggregate querying over sensor networks where the network topology is continuously evolving. We develop scalable data aggregation techniques that remain efficient and accurate even as nodes move, join or leave the network. We present a novel distributed algorithm called CountTorrent, that enables fast estimation of certain classes of aggregate queries such as COUNT and SUM. CountTorrent does not require a static routing infrastructure, is easily implemented in a
more » ... ed setting, and can be used to inform all network nodes of the aggregate query result, instead of just the query initiator as is done in traditional query aggregation schemes. We evaluate its robustness and accuracy compared to previous aggregation approaches through simulations of dynamic and mobile sensor network environments and experiments on micaz motes. We show that in networks where the nodes are stationary, CountTorrent can provide 100% accurate aggregate results even in the presence of lossy links. In mobile sensor networks where the nodes constantly move and hence the network topology changes continuously, CountTorrent provides a close (within 10 − 20%) estimate of the accurate aggregate query value to all nodes in the network at all times.
doi:10.1145/1322263.1322269 dblp:conf/sensys/KamraMR07 fatcat:pbneufcgjrhejl6szoviewzbsy

Controlling the performance of 3-tiered web sites

Abhinav Kamra, Vishal Misra, Erich Nahum
2004 Performance Evaluation Review  
doi:10.1145/1012888.1005744 fatcat:uq3q44ze5je7zatxck6xscq42u

Scaling and Benchmarking Self-Supervised Visual Representation Learning [article]

Priya Goyal, Dhruv Mahajan, Abhinav Gupta, Ishan Misra
2019 arXiv   pre-print
Self-supervised learning aims to learn representations from the data itself without explicit manual supervision. Existing efforts ignore a crucial aspect of self-supervised learning - the ability to scale to large amount of data because self-supervision requires no manual labels. In this work, we revisit this principle and scale two popular self-supervised approaches to 100 million images. We show that by scaling on various axes (including data size and problem 'hardness'), one can largely
more » ... or even exceed the performance of supervised pre-training on a variety of tasks such as object detection, surface normal estimation (3D) and visual navigation using reinforcement learning. Scaling these methods also provides many interesting insights into the limitations of current self-supervised techniques and evaluations. We conclude that current self-supervised methods are not 'hard' enough to take full advantage of large scale data and do not seem to learn effective high level semantic representations. We also introduce an extensive benchmark across 9 different datasets and tasks. We believe that such a benchmark along with comparable evaluation settings is necessary to make meaningful progress. Code is at: https://github.com/facebookresearch/fair_self_supervision_benchmark.
arXiv:1905.01235v2 fatcat:rmnrw5i35jeaba3fdu5tnisdwy

Data-driven exemplar model selection

Ishan Misra, Abhinav Shrivastava, Martial Hebert
2014 IEEE Winter Conference on Applications of Computer Vision  
We consider the problem of discovering discriminative exemplars suitable for object detection. Due to the diversity in appearance in real world objects, an object detector must capture variations in scale, viewpoint, illumination etc. The current approaches do this by using mixtures of models, where each mixture is designed to capture one (or a few) axis of variation. Current methods usually rely on heuristics to capture these variations; however, it is unclear which axes of variation exist and
more » ... are relevant to a particular task. Another issue is the requirement of a large set of training images to capture such variations. Current methods do not scale to large training sets either because of training time complexity [31] or test time complexity [26] . In this work, we explore the idea of compactly capturing taskappropriate variation from the data itself. We propose a two stage data-driven process, which selects and learns a compact set of exemplar models for object detection. These selected models have an inherent ranking, which can be used for anytime/budgeted detection scenarios. Another benefit of our approach (beyond the computational speedup) is that the selected set of exemplar models performs better than the entire set.
doi:10.1109/wacv.2014.6836080 dblp:conf/wacv/MisraSH14 fatcat:io35rfhlh5bx3pky57rxkjbqhi

Inflammatory Mediators and the Failing Heart: A Translational Approach

Abhinav Diwan, Tony Tran, Arunima Misra, Douglas Mann
2003 Current molecular medicine  
The inflammatory response regulates congestive heart failure (CHF) development. T cell activation plays an important role in tissue inflammation. We postulate that CD28 or B7 deficiency inhibits T cell activation and attenuates CHF development by reducing systemic, cardiac, and pulmonary inflammation. We demonstrated that chronic pressure overload-induced end-stage CHF in mice is characterized by profound accumulation of activated effector T cells (CD3 + CD44 high cells) in the lungs and a mild
more » ... but significant increase of these cells in the heart. In knockout mice lacking either CD28 or B7, there was a dramatic reduction in the accumulation of activated effector T cells in both hearts and lungs of mice under control conditions and after transverse aortic constriction. CD28 or B7 knockout significantly attenuated transverse aortic constriction-induced CHF development, as indicated by less increase of heart and lung weight and less reduction of left ventricle contractility. CD28 or B7 knockout also significantly reduced transverse aortic constrictioninduced CD45 + leukocyte, T cell, and macrophage infiltration in hearts and lungs, lowered proinflammatory cytokine expression (such as tumor necrosis factor-α and interleukin-1β) in lungs. Furthermore, CD28/B7 blockade by CTLA4-Ig treatment (250 μg/mouse every 3 days) attenuated transverse aortic constriction-induced T cell activation, left ventricle hypertrophy, and left ventricle dysfunction. Our data indicate that CD28/B7 deficiency inhibits activated effector T cell accumulation, reduces myocardial and pulmonary inflammation, and attenuates the development of CHF. Our findings suggest that strategies targeting T cell activation may be useful in treating CHF. (Hypertension. 2016;68:688-696. Heart by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from by guest on July 22, 2018 http://hyper.ahajournals.org/ Downloaded from Wang et al T-Cell Activation and Congestive Heart Failure 689 T cells) in the lungs. To demonstrate a role for CD28 function in CHF development, we tested the hypothesis that transverse aortic constriction (TAC)-induced LV failure and lung remodeling would be attenuated in CD28 or B7 knockout (KO) mice. Our findings indicate that genetic disruption of either CD28 or B7 protein significantly attenuates TAC-induced cardiac and pulmonary inflammation, lung remodeling, as well as LV hypertrophy and dysfunction. The beneficial effect of CD28 or B7 deletion was associated with a dramatic reduction of TACinduced T cell activation in the heart and lungs. Materials and Methods Detailed methods are available in the online-only Data Supplement. Primary antibodies and primers used in this study are listed in Tables S1 and S2 in the online-only Data Supplement. Mice and TAC Procedure Animals and Experimental Design CD28 KO (B6.129S2-Cd28 tm1Mak /J; stock No 002666) and B7 KO (B6.129S4-Cd80 tm1Shr Cd86 tm2Shr /J; stock No. 003610) mice and C57BL/6J wild-type (WT) mice were obtained from Jackson Laboratory. The fusion protein CTLA4-Ig (abatacept, 250 μg), which inhibits CD28/B7 interactions, was administered intraperitoneally every 3 days beginning 1 day before TAC. An isotype control antibody human IgG was used as a vehicle control. Male mice 4 to 5 weeks of age were subjected to a procedure using a 27G needle to create the TAC, a model that mimics clinical systemic hypertension and aortic stenosis, as previously described. 2 Data were collected 8 weeks after TAC. LV hypertrophy and cardiac function were assessed. Summary CD28/B7 signaling contributes to chronic pressure overloadinduced cardiac and pulmonary inflammation and the development of congestive heart failure, suggesting that strategies targeting Tcell activation may be useful for congestive heart failure treatment.
doi:10.2174/1566524033361537 pmid:12630562 fatcat:vwdzhb3c65a5pkuoj2lzwttyqe

Between-Class Covariance Correction For Linear Discriminant Analysis in Language Recognition

Abhinav Misra, Qian Zhang, Finnian Kelly, John H.L. Hansen
2016 Odyssey 2016  
Linear Discriminant Analysis (LDA) is one of the most widely-used channel compensation techniques in current speaker and language recognition systems. In this study, we propose a technique of Between-Class Covariance Correction (BCC) to improve language recognition performance. This approach builds on the idea of Within-Class Covariance Correction (WCC), which was introduced as a means to compensate for mismatch between different development data-sets in speaker recognition. In BCC, we compute
more » ... igendirections representing the multimodal distributions of language i-vectors, and show that incorporating these directions in LDA leads to an improvement in recognition performance. Considering each cluster in the multi-modal i-vector distribution as a separate class, the between-and within-cluster covariance matrices are used to update the global between-language covariance. This is in contrast to WCC, for which the within-class covariance is updated. Using the proposed method, a relative overall improvement of +8.4% Equal Error Rate (EER) is obtained on the 2015 NIST Language Recognition Evaluation (LRE) data. Our approach offers insights toward addressing the challenging problem of mismatch compensation, which has much wider applications in both speaker and language recognition.
doi:10.21437/odyssey.2016-10 dblp:conf/odyssey/MisraZKH16 fatcat:pk7cnyvdsnhuzkvqvf46xjj3oe

Growth codes

Abhinav Kamra, Vishal Misra, Jon Feldman, Dan Rubenstein
2006 Proceedings of the 2006 conference on Applications, technologies, architectures, and protocols for computer communications - SIGCOMM '06  
Sensor networks are especially useful in catastrophic or emergency scenarios such as floods, fires, terrorist attacks or earthquakes where human participation may be too dangerous. However, such disaster scenarios pose an interesting design challenge since the sensor nodes used to collect and communicate data may themselves fail suddenly and unpredictably, resulting in the loss of valuable data. Furthermore, because these networks are often expected to be deployed in response to a disaster, or
more » ... ecause of sudden configuration changes due to failure, these networks are often expected to operate in a "zero-configuration" paradigm, where data collection and transmission must be initiated immediately, before the nodes have a chance to assess the current network topology. In this paper, we design and analyze techniques to increase "persistence" of sensed data, so that data is more likely to reach a data sink, even as network nodes fail. This is done by replicating data compactly at neighboring nodes using novel "Growth Codes" that increase in efficiency as data accumulates at the sink. We show that Growth Codes preserve more data in the presence of node failures than previously proposed erasure resilient techniques.
doi:10.1145/1159913.1159943 dblp:conf/sigcomm/KamraMFR06 fatcat:7ydo7okbvnejbhvgierk6wljvy
« Previous Showing results 1 — 15 out of 174 results