Second language acquisition researchers have been using Grammaticality Judgment Tests (GJTs) since the mid-1970s in order to assess the linguistic competence of second language learners in their L2. A number of researchers (e.g., Gass, 1994; Ellis,1991) have raised serious questions concerning the reliability of this type of test as a measure of L2 learners" linguistic competence. The purpose of this study was to examine the reliability of GJTs used in foreign language context and also to

more »

... e the relationship between timed GJTs and delayed GJTs. After administering a standard language proficiency test (OPT), 30 advanced out of a pile of 80 EFL learners were selected for this study. Participants were asked to make judgments about 34 sentences included in a computerized GJT. The grammatical structure chosen for this study was verb complements. After second administration of the same computerized GJT, various methods were used in order to examine the reliability of timed GJTs. The results of test-retest analysis and internal consistency reliability revealed that the GJT used in this study had low level of reliability. Moreover, the analysis of response patterns showed that participants were not stable in their judgments and also they were reluctant to use not sure response when they were uncertain. Therefore, their judgments did not exactly reflect their grammatical knowledge. And finally the relationship between timed GJT and delayed GJT was weak which indicated that participants may have used different types of knowledge under different tests administration conditions. The results of this study suggest that the GJT used in this study is not a reliable measure of EFL learners" knowledge about verb complements and researchers should use this kind of test with more caution.