This paper presents a first attempt of an application-driven evaluation exercise of WSD. We used a CLIR testbed from the Cross Lingual Evaluation Forum. The expansion, indexing and retrieval strategies where fixed by the organizers. The participants had to return both the topics and documents tagged with WordNet 1.6 word senses. The organization provided training data in the form of a pre-processed Semcor which could be readily used by participants. The task had two participants, and the

more »

... er also provided an in-house WSD system for comparison. The results do not improve over the baseline, which is not surprising given the simplistic CLIR strategy used. Other than that the exercise was succesful, and provides the foundation for more ambitious follow-up exercises where the participants would be able to build up on the WSD results already available.