The SAVE Science project is an attempt to address the shortcomings of current assessments of science. The project has developed two virtual worlds that each have a mystery or natural phenomenon requiring scientific explanation; by recording students' behavior as they investigate the mystery, these worlds can be used to assess their understanding of the scientific method. Currently, however, the scoring of the assessment depends either on manual grading of students' written responses, or, on

more »

... iple choice questions. This paper presents an automated grader that can combine with SAVE Science's virtual worlds to provide a cheap mechanism for assessments of the ability to apply scientific methodology. In experiments on over 300 middle school students, our best automated grader improves by over 50% relative to the closest system from previous work in predicting grades supplied by human judges.