Leveraging a corpus of natural language descriptions for program similarity

Meital Zilberstein, Eran Yahav
2016 Proceedings of the 2016 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software - Onward! 2016  
Program similarity is a central challenge in many programmingrelated applications, such as code search, clone detection, automatic translation, and programming education. We present a novel approach for establishing the similarity of code fragments by: (i) obtaining textual descriptions of code fragments captured in millions of posts on questionanswering sites, blogs and other sources, and (ii) using natural language processing techniques to establish similarity between textual descriptions,
more » ... thus between their corresponding code fragments. To improve precision, we use a simple static analysis that extracts type signatures, and combine the results of textual similarity with similarity of the signatures. Because our notion of code similarity is based on similarity of textual descriptions, our approach can determine semantic relatedness and similarity of code across different libraries and even across different programming languages, a task considered extremely difficult using traditional approaches. To evaluate our approach, we use data obtained from the popular question-answering site, STACK-OVERFLOW. To obtain a ground-truth to compare against, we developed a crowdsourcing system, LIKE2DROPS, that allows users to label the similarity of code fragments. We used the system to collect similarity classifications for a massive corpus of 6,500 program pairs. Our results show that our technique is effective in determining similarity, and achieves more than 85% precision, recall and accuracy.
doi:10.1145/2986012.2986013 dblp:conf/oopsla/ZilbersteinY16 fatcat:ijjuoeeqgndbxpmodlseyukle4