Topic variation in performance testing: the case of the Chemistry TEACH test for international teaching assistants [thesis]

Dean S. Papajohn
Test method facets are particularly important in the design. development. and use of language tests because these allow for a certain amount of control in testing (Bachman 1990) . Topic of input is an important feature within the framework of test method facets. Recent studies have focused on general topics versus field speciftc topics (Douglas and Selinker 1992. Smith 1992). Yet. field speciftc performance tests for international teaching assistants often provide different topics for each
more » ... nee. assuming equivalency between topics. The comparison of general topics versus field speciftc topics is unable to capture the full effect of topic. 1b1s study reports research into topic features and the effect topic variation has on a performance test for international teaching assistants. the chemistry TEACH test. Results Indicate a relationship between topic of input as defined by the topic features of concepts. math. and calculations and test scores on the chemistry iEACH. Do dJ1ferences in the fifteen chemistry TEACH topics affect test scores? Bachman invites us to research further the facets that compose his framework of test method facets (Bachman 1990). As ofyet the effect of topic on language performance testing has not been thoroughly studied. As test developers are delving further into the areas of context and topic, as can be seen in the current revising of the Test of Spoken English (TSE), a better understanding of topic is urgently needed. This investigation aims to provide evidence of the effect of topic variation in performance testing which will be useful to language performance test developers. -------------------------4 LITERATURE REVIEW The ITA problem has been documented in a number of sources (e.g., Smith et al. 1992) . The awareness of the ITA problem bas prompted many universities to institute ITA assessment. Performance testing bas been widely used for ITA assessment. However, performance testing, like any other testing, bas its limitations, and criteria to evaluate performance testing need to be carefully considered. Bachman (1990) offers his theories of communicative competence and test method facets as guides in language testing research. This includes the topic material or the nature of the language of the input. Language researchers have studied the effect of topic variation in performance testing while chemistry educators have studied the effect of topic presentation on communication of chemistry knowledge. The ITA Problem The communication problem between ITAs and their students bas concerned many people and universities including ISU. For a long time the "ITA problem" was blamed on the ITA's accent and flawed English ability. Other factors contributing to the ITA's problem are a non-standard variety of English, and the ITA's unfamiliarity with American classroom register, teaching strategies, and classroom management (Kaplan 1989) . Undergraduates have also been cited for contributing to the ITA problem (vom Saal 1987). Certain areas of the United States, particularly rural areas, do not have much contact With intercultural communication. This is true for many parts of Iowa and for many of the undergraduates at ISU. The seriousness of the ITA problem is reflected in the suit brought against the University of Pittsburgh. An ITA was blamed for inadequately explaining safety procedures in a lab. One student spilled concentrated nitric acid on both legs of another student causing second degree burns. The injured student blamed the problem on the ITA's lack of English ability (Scbmitz 1993). Although the case was settled out of court, universities have taken notice of their responsibility in the assessment and placement of IT As.
doi:10.31274/rtd-180814-5433 fatcat:ywzhulibsfgy7acxwvzhgttaoe