Gathering initial content and substantive validity evidence for a scale to measure negativity bias in sport: Leveraging expert review and cognitive interviews

Abstract

Negativity bias refers to the disproportionate psychological impact of negative experiences, stimuli, and events compared to those that are neutral or positive. This phenomenon is pervasive across diverse contexts, including sport. Despite relevance to sport, there is an absence of research in sport contexts, including a lack of empirically validated measurement tools. Accordingly, the purpose of this study was to develop and collect initial content and substantive validity evidence for a sport-specific scale to measure negativity bias. There were three sequential stages: (1) initial item generation, (2) expert review, and (3) cognitive interviews. An initial pool of 84 items was developed, guided by negativity bias literature in other domains and previous empirical evidence from the research team. Next, experts (N = 12) with PhDs in areas relevant to the scale content evaluated the content validity of the measure, including the content relevance, representativeness, and technical quality. Items were refined in response to the expert feedback. Cognitive interviews were then conducted with competitive athletes (N = 14) to gain insight into how they interpreted and responded to scale items (i.e., substantive validity). The interviews involved asking the athletes to verbalize their thoughts as they read and answered each question. Items were refined iteratively between and following interviews. Establishing initial validity evidence through multiple methods is essential for confidence in the scale’s readiness for further psychometric evaluation. This project outlines the procedures and importance of expert reviews and cognitive interviews – two often overlooked steps in collecting content and substantive validity evidence, respectively.