9,567 Hits in 5.1 sec

Classification Representations Can be Reused for Downstream Generations [article]

Saisubramaniam Gopalakrishnan, Pranshu Ranjan Singh, Yasin Yazici, Chuan-Sheng Foo, Vijay Chandrasekhar, ArulMurugan Ambikapathi
2020 arXiv   pre-print
classifier for the downstream task of sample generation.  ...  We show that these latent space representations can be smartly manipulated (using convex combinations of n samples, n≥2) to yield meaningful sample generations.  ...  Conclusion and Future Work The answer to the question: 'Can classification latent space representations be reused for the downstream task of generation?'  ... 
arXiv:2004.07543v1 fatcat:jheawxxqyrbn3jwv4imdo6c2wq

Transfer Learning for Segmentation Problems: Choose the Right Encoder and Skip the Decoder [article]

Jonas Dippel, Matthias Lenga, Thomas Goerttler, Klaus Obermayer, Johannes Höhne
2022 arXiv   pre-print
In this work, we investigate the impact of transfer learning for segmentation problems, being pixel-wise classification problems that can be tackled with encoder-decoder architectures.  ...  It is common practice to reuse models initially trained on different data to increase downstream task performance.  ...  can be generalized in downstream segmentation tasks.  ... 
arXiv:2207.14508v1 fatcat:tdy6fcvxwnhd7m4v65cpyb5rya

Prompt Tuning for Discriminative Pre-trained Language Models [article]

Yuan Yao, Bowen Dong, Ao Zhang, Zhengyan Zhang, Ruobing Xie, Zhiyuan Liu, Leyu Lin, Maosong Sun, Jianyong Wang
2022 arXiv   pre-print
., ELECTRA, can be effectively prompt-tuned.  ...  The source code and experiment details of this paper can be obtained from  ...  DPT for Question Answering. Besides text classification, DPT can also be applied for the question answering task.  ... 
arXiv:2205.11166v1 fatcat:djvuui475jd37hn4mcebbcte2a

Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation [article]

Akiko Eriguchi, Melvin Johnson, Orhan Firat, Hideto Kazawa, Wolfgang Macherey
2018 arXiv   pre-print
Further, our system can perform classification in a new language for which no classification data was seen during training, showing that zero-shot classification is possible and remarkably competitive.  ...  , encoder representation power, and model generalization on zero-shot performance.  ...  We hypothesize that a system with enhanced generalization might be better suited for zero-shot classification since it is a measure of the ability of the model to generalize to a new task.  ... 
arXiv:1809.04686v1 fatcat:rmw6bqdjefdl7izrxj5df7xwvu

Dive into Self-Supervised Learning for Medical Image Analysis: Data, Models and Tasks [article]

Chuyan Zhang, Yun Gu
2022 arXiv   pre-print
However, for a specific downstream task, there is still a lack of an instruction book on how to select suitable pretext tasks and implementation details.  ...  , (3) the applicability of upstream tasks to downstream tasks and (4) the stacking effect of SSL and commonly used policies for deep learning, including data resampling and augmentation.  ...  Thus, the extent of feature reuse can be indicative of the efficiency of a SSL algorithm in downstream tasks.  ... 
arXiv:2209.12157v1 fatcat:ocv2l3fxmjczdf3bnkarlgq2qa

CAiD: Context-Aware Instance Discrimination for Self-supervised Learning in Medical Imaging [article]

Mohammad Reza Hosseinzadeh Taher, Fatemeh Haghighi, Michael B. Gotway, Jianming Liang
2022 arXiv   pre-print
their performance on medical downstream tasks.  ...  Our preliminary analysis showed that high global similarity of medical images in terms of anatomy hampers instance discrimination methods for capturing a set of distinct features, negatively impacting  ...  As such, while f θ is a regular encoder, f ξ can be a momentum encoder (He et al., 2020) or share weights with f θ (Zbontar et al., 2021) ; sim(.) can be contrastive loss (He et al., 2020) , cosine  ... 
arXiv:2204.07344v1 fatcat:svxk5yzsmfdyddgy4frgmpxg5i

InferCode: Self-Supervised Learning of Code Representations by Predicting Subtrees [article]

Nghi D. Q. Bui, Yijun Yu, Lingxiao Jiang
2020 arXiv   pre-print
Current learning techniques, however, have a major drawback that these models are mostly trained on datasets labeled for particular downstream tasks, and code representations may not be suitable for other  ...  cross-language code search or reused under a transfer learning scheme to continue training the model weights for supervised tasks such as code classification and method name prediction.  ...  in producing embeddings for any code unit that can be parsed into syntax trees, and (3) general enough so that its trained representations for code can perform well for various downstream tasks.  ... 
arXiv:2012.07023v2 fatcat:jxhfs2a6qfeabehgjvt4bavkfe

MIGS: Methylation Interpolated Gene Signatures Determine Associations Between Differential Methylation and Gene Expression [article]

Christopher E Schlosberg, Nathan D VanderKraats, John R Edwards
2016 bioRxiv   pre-print
We find that methylation changes at the TSS and downstream ~2kb are most predictive of expression change.  ...  MIGS will be an invaluable tool to analyze genome-wide methylation data as MIGS produces a longer and more accurate list of genes with methylation-associated expression changes.  ...  Acknowledgements We would like to acknowledge Tao Ju for providing assistance on best practices for feature representation and Kilian Weinberger for confirmation of evaluation framework of our machine  ... 
doi:10.1101/063941 fatcat:qbtxr6tixzdrrdsfc3eiwb2o2i

Self-Supervised Vision Transformers for Malware Detection [article]

Sachith Seneviratne, Ridwan Shariffdeen, Sanka Rasnayaka, Nuran Kasthuriarachchi
2022 arXiv   pre-print
Experimental results using 1.2 million Android applications across a hierarchy of 47 types and 696 families, shows that self-supervised learning can achieve an accuracy of 97% for the binary classification  ...  Our proposed model is also able to outperform state-of-the-art techniques for multi-class malware classification of types and family with macro-F1 score of .497 and .491 respectively.  ...  A model trained for solving these pretext tasks learn representations that can be reused for solving other downstream tasks of interest, such as image classification.  ... 
arXiv:2208.07049v1 fatcat:hjdecyc75jb5lnyzp5vrmvkaia

Self-Supervised Contrastive Learning of Protein Representations By Mutual Information Maximization [article]

Amy X Lu, Haoran Zhang, Marzyeh Ghassemi, Alan M Moses
2020 bioRxiv   pre-print
Pretrained embedding representations of biological sequences which capture meaningful properties can alleviate many problems associated with supervised learning in biology.  ...  Our model, CPCProt, achieves comparable performance to state-of-the-art self-supervised models for protein sequence embeddings on various downstream tasks, but reduces the number of parameters down to  ...  logistic regression and kNN for downstream classification, this gap in accuracy decreases ( Table 2 ).  ... 
doi:10.1101/2020.09.04.283929 fatcat:lx2m7ndlsvbojfmxagltqm624a

Zero-Shot Program Representation Learning [article]

Nan Cui, Yuze Jiang, Xiaodong Gu, Beijun Shen
2022 arXiv   pre-print
However, gathering training samples can be costly and infeasible for domain-specific languages such as Solidity for smart contracts.  ...  This enables the representation model to efficiently fit the scarce task-oriented data while reusing pre-trained knowledge.  ...  The classification generates a probability score, which can be used for ranking results of code search. 3) Method Name Prediction: a task that suggests the function name for a given code snippet [42]  ... 
arXiv:2204.08360v1 fatcat:fy24wasbm5a4zdiimebthvx4ja

Improving Generalizability of Protein Sequence Models with Data Augmentations [article]

Hongyu Shen, Layne C. Price, Taha Bahadori, Franziska Seeger
2021 bioRxiv   pre-print
In rarer cases, we even find that information-destroying augmentations, such as randomly shuffling entire protein sequences, can improve downstream performance.  ...  For each TAPE validation task, we demonstrate improvements to the baseline scores when the learned protein representation is fixed between tasks.  ...  However, with other common data types there are simple transformations that can be applied to the data in order to improve a model's ability to generalize: for instance, vision models use cropping, rotations  ... 
doi:10.1101/2021.02.18.431877 fatcat:ytln6ofzjvelhapkdg5s572vcu

InferCode: Self-Supervised Learning of Code Representations by Predicting Subtrees

Nghi D. Q. Bui, Yijun Yu, Lingxiao Jiang
2021 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)  
This pre-trained model can then be applied to downstream unsupervised tasks such as code clustering, code clone detection, cross-language code search, or be reused under a transfer learning scheme to continue  ...  Even though some techniques generate representations from unlabeled code, they are far from being satisfactory when applied to the downstream tasks.  ...  We also thank the anonymous reviewers for their insightful comments and suggestions, and thank the authors of related work for sharing data.  ... 
doi:10.1109/icse43902.2021.00109 fatcat:qesroisw3zepdfxlswqjuq3uaa

Model Reprogramming: Resource-Efficient Cross-Domain Machine Learning [article]

Pin-Yu Chen
2022 arXiv   pre-print
In data-rich domains such as vision, language, and speech, deep learning prevails to deliver high-performance task-specific models and can even learn general task-agnostic representations for efficient  ...  model finetuning, where the source and target domains can be vastly different.  ...  the input transformation and output mapping layers can be implemented at the edge device for reprogramming.  ... 
arXiv:2202.10629v2 fatcat:p4gwfzfsljafnkhff6diovwqfy

Self-Supervised Pretraining with DICOM metadata in Ultrasound Imaging

Szu-Yeu Hu, Shuhang Wang, Wei-Hung Weng, Jingchao Wang, Xiaohong Wang, Arinc Ozturk, Quan Li, Viksit Kumar, Anthony E. Samir
2020 Machine Learning in Health Care  
In this work, we leverage DICOM metadata from ultrasound images to help learn representations of the ultrasound image.  ...  We demonstrate that the proposed method outperforms the approaches without using metadata across a variety of downstream tasks.  ...  We showed that incorporating DICOM metadata as weak labels can improve the quality of representation learning and improve the performance of the downstream segmentation and classification tasks.  ... 
dblp:conf/mlhc/HuWWWWOLKS20 fatcat:asdbnekbxvge3ajamxqawve4oy
« Previous Showing results 1 — 15 out of 9,567 results