Asking probing questions via web probing has recently been advocated as a promising method for evaluating survey questions. In comparison to standard face-to-face (f2f) cognitive interviewing, the increasing availability of internet non-probability panels allows for recruiting respondents in a quicker and more cost-effective way and a realization of larger sample sizes. In the present study, we examine whether web probing is a potential alternative to standard cognitive interviewing, in

more »

... ar: Does web probing produce similar results as f2f cognitive interviewing with regard to the problems detected and the item revisions suggested? The study compares the findings of 508 respondents drawn from a non-probability online panel who completed an online survey including four items from the International Social Survey Programme 2013 and 2014 with the results obtained via f2f cognitive interviewing with 20 participants. Findings indicate that web probing and cognitive interviewing detect very similar problems and lead to the same suggestions for item revisions. However, web probing itself has some limitations. Practical implementations and directions for future research are discussed. introduction and research questions Cognitive interviewing is a qualitative method that aims to reveal information from respondents about the cognitive processes they use when answering survey questions and to identify problems with questions (Willis 2005). Conventionally, cognitive interviewing involves conducting face-to-face (f2f) interviews with small sample sizes of five to 30 respondents (Willis 2005). The semi-structured, in-depth interviews are conducted by specially trained cognitive interviewers on the basis of an interview protocol which contains the questions to be tested in the cognitive interview and the techniques to be adopted, in particular think-aloud and follow-up questions (probing). The technique of probing is used to elicit information about how respondents interpret questions or define specific terms and how respondents arrive at their answers. In addition to the scripted probing questions included in the interview protocol, emergent probes can be asked to follow up on respondents' comments during the interview. Probing questions are administered either