Reliability generalization and the measurement of psychological need satisfaction experienced when exercising: The tale of two instruments

Abstract

Objective: The purpose of this study was to use reliability generalization procedures to (a) test the level of heterogeneity evident in the reliability of scores derived from the Psychological Need Satisfaction in Exercise Scale (PNSE; Wilson et al., 2006) and the Basic Psychological Needs in Exercise Scale (BPNES; Vlachopoulos & Michailidou, 2006), and (b) identify factors impacting score reliability estimates for both instruments. Methods: Primary research studies published in English using all subscales of the PNSE and/or BPNES were identified via an exhaustive literature search. Data were coded and extracted for further analysis in an iterative and sequential fashion. Results: A total of 234 reliability coefficients were extracted from 45 published studies including coefficients of stability (n = 18) and internal consistency (n = 216). The unweighted coefficients of internal consistency ranged from 0.69 to 0.97 (M = 0.88; SD = 0.07) whereas the coefficients of stability ranged from 0.52 to 0.97 (M = 0.84; SD = 0.14). Age was correlated with coefficients of internal consistency for relatedness only (r12 = 0.38, p < .05) whereas sample size was inversely correlated with internal consistency estimates (r12 values ranged from -0.42 to -0.69, p < .05). Multivariate analysis indicated that internal consistency score reliability estimates differed significantly as a function of the instrument and language used to collect the data across studies (partial η2 values = 0.37 to 0.93) but remained unaffected by the sample’s clinical status. Discussion: Overall, the observed heterogeneity in reliability coefficients plus the impact of select study characteristics on score reliability estimates compels researchers to pay greater attention to instrument selection when measuring psychological need satisfaction in exercise.