Effectiveness of Using Guided Peer Code Review to Support Learning of Programming Concepts in a CS2 Course: A Pilot Study
2020 ASEE Virtual Annual Conference Content Access Proceedings
Introduction This section provides a brief statement of the problem, research motivation, our proposed solution and high-level contributions. Problem statement: Researchers at North Dakota State University (NDSU) have discovered that students enrolled in CS1/CS2 courses struggle at understanding programming concepts, make common mistakes that are recurring and don't think about the code quality as they learn to program. Researchers have discussed exploring program comprehension (helping
... on (helping students understand how a program works) and guiding their learning towards program generation (creating a part or whole program)  . In a programming domain, program comprehension tasks can include modifying a program's functionality, identifying incorrect behavior, or simply answering questions about the program. While program comprehension has been extensively studied in industrial settings, not much educational research can be found on the topic [1, 13]. Based on the empirical studies in industry, a common approach involves a programmer reading through the code line-by-line, which allows programmers to build their knowledge base methodically [1, 13] . A specific type of method called peer code review (PCR) has been found to be useful at not just building high-level abstractions but also enabling knowledge sharing among programmers. PCR is a well-defined quality assurance activity that is designed for the sharing of knowledge and for the purpose of improving code quality [1, 2, 3]. Classified at the evaluation level of Bloom's taxonomy , PCR involves students systematically checking (mainly by viewing and reading) parts of someone else's source code for errors, with the aim of creating better code when they develop their own. Research motivation: Our initial study supports the idea that PCR can be used to support programming pedagogy  in higher education. Based on a feasibility study, it was found that finding errors in someone else's code reduced the likelihood of committing similar errors when students went on to develop their own code. However, students were not reporting many errors when they did not have a list of questions that guides them to identify errors, which resulted in students not pay attention to detail when reading someone else's code. Students indicated that they needed more guidance on what types of errors to look for when performing the code review. Including some set of expectations that can help students map those expectations to the source code under review. Research objective: We aimed to empirically investigate whether guided PCR could be effective in teaching CS2 students programming concepts. Additionally, we investigated the possibility of helping students to retain information (from PCR sessions) to times when they develop their own code. Furthermore, we also wanted to investigate if pair-based review can help with knowledge retention or sharing among students. To accomplish this, guided PCR sessions were conducted where students were asked to follow a provided checklist in order to systematically check for programming errors when reading source code on line-to-line basis. Research methodology: We designed and implemented four different PCR sessions over the course of four weeks in one CS2 classroom. During each week, students were given a piece of code covering a specific data structure and were asked to review and find errors in the code. The provided code pieces were seeded with five categories of errors: initialization/declaration, method call/ definition, array/linkedList/trees/, output and flow of control. We analyzed data gathered from the guided PCR sessions, reflection sessions conducted after each PCR session, work conducted by students (assignments, quizzes, and exams completed as part of the course), and a feedback survey conducted at the end of study. Contribution: This paper provides empirical evidence of the impact the guided PCR had on CS2 students' understanding of programming concepts. While prior research has examined the effects of PCR on students' motivation to learn and course attendance, it lacks evidence concerning the usefulness of guided PCR at improving students' conceptual knowledge in their courses. We also provide evidence on how PCR can be used to enable knowledge sharing among students and foster in-class discussion. We also hope that this study will lead to the development of a more robust error checklist and a larger repository of programs that are seeded with realistic defects and can be used by other CS1/CS2 instructors.