re THE EFFECTS OF SINGLE AND DOUBLE PLAY UPON LISTENING TEST OUTCOMES AND COGNITIVE PROCESSING THE EFFECTS OF SINGLE AND DOUBLE PLAY UPON LISTENING TEST OUTCOMES AND COGNITIVE PROCESSING: THE EFFECTS OF SINGLE AND DOUBLE PLAY UPON LISTENING TEST OUTCOMES AND COGNITIVE PROCESSING

John Field, John Field, John Field
unpublished
The convention of allowing test-takers to hear a recording twice is a controversial topic within L2 listening assessment. Arguably, it compensates for the lack of visual and contextual information when an audio recording is used; but it can also be argued that, in most real-world circumstances, listeners have only one opportunity to make sense of what is said. Hearing a recording twice has been shown to increase scores, fostering the impression that it renders a test ʻeasier'. However, little
more » ... . However, little is known about the precise effects of double play upon test-taker behaviour, the focus of this study. Pre-sessional students (N=73) took a retired IELTS listening test which featured either a multiplechoice format or a gap-filling one. They assumed that they would hear the recording only once, as is customary in IELTS, but were then allowed a second hearing. Scores after the first play were compared with those after the second. There was a general increase, although it varied considerably across individuals. Analysis showed that incorrect or blank answers were replaced by correct ones in relatively few cases. Much more frequent was the persistence of correct answers or incorrect ones. An important issue was whether double play advantaged any particular proficiency level, and thus potentially eroded scoring differentials. In fact, test-takers were found to benefit regardless of level. The data was also analysed to establish whether test format was a factor in the increased scores. Scores from the constructed response task improved more after a second play than those from the selected response one. However, this was mainly because the test-takers experienced difficulty in providing correct answers to gap-filling items after only a single play. A second phase of the study consisted of face-to-face encounters with individual participants (N=36), who took the same IELTS tests and provided verbal reports on how they had arrived at their answers. This was followed by a semi-structured interview in which participants were asked about their experience of the double play presentation and about how they had made use of the opportunity of hearing the recording twice. The data obtained suggested that, for the majority of the participants, cognitive behaviour differed markedly when they were permitted a second hearing. Important features were a reduction in listening anxiety and greater familiarity with the recorded material, which made it easier to locate the required information. Even on the second play, there was still a heavy attentional focus on word-level decoding in order to check or add answers. But, in the case of many participants, this was accompanied by a wider perspective on the content of the recording and the speaker's goals. These participants managed to acquire the kind of overview that is an important component of academic and professional listening. Conclusions are drawn as to the validity of employing double play in future listening test design. Specific reference is made to the structure and needs of the Aptis listening test. THE EFFECTS OF SINGLE AND DOUBLE PLAY UPON LISTENING TEST OUTCOMES AND COGNITIVE PROCESSING: JOHN FIELD ASSESSMENT RESEARCH AWARDS AND GRANTS | PAGE 3 Author John Field is Senior Lecturer in the CRELLA research unit at the University of Bedfordshire, UK. He is especially known for his work on second language listening and his Listening in the Language Classroom (CUP, 2008) has become a standard work in the field. His background in psycholinguistics (on which he has written widely) informs much of his thinking and he is currently applying it to the notion of cognitive validity in L2 testing. He undertakes research into how the conventions, input, formats and items used in listening tests shape test-taker behaviour. He also advises test providers on approaches which reflect the nature of the listening skill more accurately and enable it to be assessed more validly. In another life, he was a materials writer and teacher trainer: writing coursebook series for Saudi Arabia and Hong Kong, radio programmes for the BBC World Service and TV programmes for the Open University of China. He continues to advise publishers on materials design. Acknowledgements I would like to express my sincere thanks to the British Council for their faith in granting me and the CRELLA Institute the first of the awards in the Assessment Research scheme, and for their patience and support in what has proved to be a long and quite complex project. I would especially like to thank Mina Patel for the personal interest she has shown in the project and the considerate and efficient way she has administered it on behalf of the Council. The project would have been impossible without the support and hard work of my three research collaborators: Colin Campbell and Jonathan Smith, who collected and collated the group test data, and Sheila Thorn of the Listening Business, who painstakingly transcribed the verbal reports. I also owe a great debt to the International Study and
fatcat:nj3xzt4qhrao5i2glp26s7pdgy