Beyond the Code Itself: How Programmers Really Look at Pull Requests

Denae Ford, Mahnaz Behroozi, Alexander Serebrenik, Chris Parnin
2019 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS)  
Developers in open source projects must make decisions on contributions from other community members, such as whether or not to accept a pull request. However, secondary factors-beyond the code itself-can influence those decisions. For example, signals from GitHub profiles, such as a number of followers, activity, names, or gender can also be considered when developers make decisions. In this paper, we examine how developers use these signals (or not) when making decisions about code
more » ... ns. To evaluate this question, we evaluate how signals related to perceived gender identity and code quality influenced decisions on accepting pull requests. Unlike previous work, we analyze this decision process with data collected from an eye-tracker. We analyzed differences in what signals developers said are important for themselves versus what signals they actually used to make decisions about others. We found that after the code snippet (x = 57%), the second place programmers spent their time fixating is on supplemental technical signals (x = 32%), such as previous contributions and popular repositories. Diverging from what participants reported themselves, we also found that programmers fixated on social signals more than recalled.
doi:10.1109/icse-seis.2019.00014 dblp:conf/icse/FordBSP19 fatcat:ab6wzn53efbyxoa6jtnpnbbn2m