Using multi-source feedback in the classroom: lessons learned
31st Annual Frontiers in Education Conference. Impact on Engineering and Science Education. Conference Proceedings (Cat. No.01CH37193)
For the past two years, we have been tracking two cohorts of industrial engineering students as they progress through their curriculum. We are particularly interested in measuring their acquisition of three EC-2000 outcomes: teamwork, communications and problem solving skills. To do this, we have utilized multi-source feedback in conjunction with closed-form attitudinal surveys as part of a study to measure outcomes through triangulation. In this paper, we will discuss our use of multi-source
... edback in the classroom and compare the results that we have obtained to results obtained from the attitudinal surveys. We will also present those effects we have found that may be due to differences in gender and learning styles. Specifically, our experiment involves following two cohorts of approximately 50 students each through a three-course sequence in i ndustrial engineering that spans the third, fourth and fifth semesters. Each course requires the use of open-ended problem solving (outcome "e"), oral and written communication skills (outcome "g"), and relies heavily on teamwork (outcome "d") both in and out of the classroom. Outcome measurements are obtained using two assessment methods -multi-source feedback via the Team Developer™ and closed-form questionnaires via the Pittsburgh Sophomore/Junior Engineering Learning and Curriculum Evaluation Instruments © and the Pittsburgh Senior Exit Survey © . In addition, we have further enhanced information about student teams with a short survey that assesses overall team maturity and performance, as well as obtains students' perceptions about the overall learning environment. Cooperative learning and teaming techniques are placing the students themselves in the best position to provide one another with meaningful, multi-source feedback about their technical and interpersonal performances. Yet, the applications of multi-source feedback processes in the classroom have been limited, in part, due to large time and resource requirements for development and implementation. It is proposed that a formal, multi-source, peer feedback approach should provide students and educators with important benefits including: help in reinforcing key learning objectives and sending strong messages to students when and where performance should be improved. Recent research on the use of peer feedback systems has suggested that students are likely to demo nstrate changes in behavior and skill acquisition simply by completing the feedback instrument. We will discuss the consistency of student peer and self-ratings over the course of the three semesters for the replicated cohorts and compare them to the survey results. We will describe how learning outcomes change when the process is repeated during specific courses. We will also look at the impact of both gender and learning styles on peer and selfratings. In addition, we have postulated some additional measures of team performance. To better enhance these measures that complement the results of the Team Developer, we have constructed an additional survey instrument. We have correlated these performance measures with the Team Developer results. Finally, we will discuss student reaction to use of the Team Developer as both a learning tool and an educational intervention. Student concerns revolve around how serious they perceive that their peers are in completing the instrument. However, results to date indicate that most students believe the Team Developer is a valuable tool that has helped them identify potential weaknesses.