Using 360° virtual reality as a decision-making assessment tool in sport

Abstract

Objectives: To examine the reliability, construct validity and ecological validity of 360° VR and match broadcast footage for off-field decision-making assessment in Australian football umpires.

Design: Validation assessments with test re-test reliability.

Methods: Two video-based tests of 60 clips each were developed to assess Australian football umpire decision-making, including 360° video of small-sided Australian football games and match broadcast footage of AFL games. Elite (n = 13) and amateur (n = 15) umpires participated in two testing sessions, in a randomised, counterbalanced design. Test re-test reliability was assessed using Cohen's Kappa for individual clips and Intraclass Correlation Coefficients for test scores. Video tests were assessed for construct validity. Ecological validity of the decision-making processes was assessed for each method.

Results: 31 clips met the minimum Kappa criteria for the 360° VR test and 28 clips for match broadcast. Results indicated strong reliability for the 360° VR (ICC = 0.89) and match broadcast (ICC = 0.89) tests. For both video modes, elite umpires performed significantly better in decision-making accuracy than amateur (p < 0.05). For ecological validity of the decision-making processes, 360° VR was rated significantly higher than match broadcast vision (p < 0.05) overall.

Conclusions: This is the first study to examine the reliability and validity of 360° VR footage as an off-field decision-making assessment tool in sport. As match broadcast vision is commonly used to assess decision-making in athletes and officials, results suggest that 360° VR is also an appropriate assessment tool. Although both video modes demonstrate similar reliability and construct validity, 360° VR was considered more specific to in-game decision-making processes, suggesting stronger ecological validity.

Keywords: Decision-making assessment; Perceptual-cognitive skill; Test validation; Virtual reality.