Can we trust street survey data? A rasch analysis using the example of soccer identification

Abstract

Not only in street surveys (e.g., due to be in the hurry, self-presentation reasons) response biases (e.g., tendency to the middle, favoring of the extremes) can often occur. These response patterns can damage a successful testing of hypotheses and it is appropriate to exclude these response patterns (resp. persons) from further content-related considerations. Aim of this study is to show how to identify response biases in a large dataset by using an item response model (IRT, i.e., the Mixed-Rasch model, MRM, Rost, 1990) for that purposes (see also Eid & Zickar, 2007). As data example responses are used from a street survey in the field of fan identification. In the context of the soccer world championships 2002, 2006, 2010, 2014 street surveys were conducted every day of the event for 1 hour in public in Muenster, Germany. The level of identification with the German national team was measured by a one-dimensional identification questionnaire (7 questions, 5-point Likert-scale; Strauss, 1995; Wann & Branscombe, 1993). A total of 10,571 participants completed the entire survey. The one-dimensional Rasch model did not fit the data. The MRM revealed two latent classes: Class 1 (66%) showed the assumed one-dimensional model of identification, while other classes (34%) revealed response biases (mostly favor of extremes). Subsequent correlational analyses using additional external variables showed how these persons can be characterized. This study demonstrates the potential damaging effects if response biases occur in a relevant extent and how researchers can deal with by using IRT models.