External validation of Global Evaluative Assessment of Robotic Skills (GEARS)

Monty A. Aghazadeh, Isuru S. Jayaratna, Andrew J. Hung, Michael M. Pan, Mihir M. Desai, Inderbir S. Gill, Alvin C. Goh

Research output: Contribution to journalArticlepeer-review

41 Scopus citations

Abstract

Background: We demonstrate the construct validity, reliability, and utility of Global Evaluative Assessment of Robotic Skills (GEARS), a clinical assessment tool designed to measure robotic technical skills, in an independent cohort using an in vivo animal training model. Methods: Using a cross-sectional observational study design, 47 voluntary participants were categorized as experts (>30 robotic cases completed as primary surgeon) or trainees. The trainee group was further divided into intermediates (≥5 but ≤30 cases) or novices (<5 cases). All participants completed a standardized in vivo robotic task in a porcine model. Task performance was evaluated by two expert robotic surgeons and self-assessed by the participants using the GEARS assessment tool. Kruskal–Wallis test was used to compare the GEARS performance scores to determine construct validity; Spearman’s rank correlation measured interobserver reliability; and Cronbach’s alpha was used to assess internal consistency. Results: Performance evaluations were completed on nine experts and 38 trainees (14 intermediate, 24 novice). Experts demonstrated superior performance compared to intermediates and novices overall and in all individual domains (p < 0.0001). In comparing intermediates and novices, the overall performance difference trended toward significance (p = 0.0505), while the individual domains of efficiency and autonomy were significantly different between groups (p = 0.0280 and 0.0425, respectively). Interobserver reliability between expert ratings was confirmed with a strong correlation observed (r = 0.857, 95 % CI [0.691, 0.941]). Experts and participant scoring showed less agreement (r = 0.435, 95 % CI [0.121, 0.689] and r = 0.422, 95 % CI [0.081, 0.0672]). Internal consistency was excellent for experts and participants (α = 0.96, 0.98, 0.93). Conclusions: In an independent cohort, GEARS was able to differentiate between different robotic skill levels, demonstrating excellent construct validity. As a standardized assessment tool, GEARS maintained consistency and reliability for an in vivo robotic surgical task and may be applied for skills evaluation in a broad range of robotic procedures.

Original languageEnglish (US)
Pages (from-to)3261-3266
Number of pages6
JournalSurgical Endoscopy
Volume29
Issue number11
DOIs
StatePublished - Nov 1 2015

Keywords

  • Clinical competence
  • Education
  • Robotics
  • Validation studies

ASJC Scopus subject areas

  • Surgery

Fingerprint

Dive into the research topics of 'External validation of Global Evaluative Assessment of Robotic Skills (GEARS)'. Together they form a unique fingerprint.

Cite this