Using simulators to assess laparoscopic competence: Ready for widespread use?

Review of the available articles demonstrates the lack of standardization in tasks, metrics, and level of validation. Many studies that claim to show the benefits of simulation have significant design flaws. Several simulators possess the ability to differentiate between surgeons of varying experience (construct validity), which is especially convincing when fine gradations in experience can be detected by the simulator, rather than just the ability to differentiate 2 groups with very disparate surgical capability (eg, medical students and master surgeons). 71 Likewise, the demonstration in some simulators that performance improves with practice may reflect simple acclimatization to the simulator rather than the development of a specific technical skill. The demonstration of criterion-based validity (comparison to another available measure of the same attribute) in some simulators furthers the evidence for their validity. This is particularly true for those simulators that provide evidence for effectiveness, in that training in the simulator improves performance of a more complex laparoscopic task, correlates with operating room performance, or improves operating room performance. Face and content validity vary widely among the simulators, from essentially psychometric tasks (MIST-VR, Rosser drills, Southwestern Centre for Minimally Invasive Surgery), to tasks that more closely model laparoscopic skills (MISTELS, LapSim). None of the reviewed simulators model entire procedures. The MIST-VR and MISTELS tasks are the most extensively studied simulators and have been used outside of the host institution, providing evidence for generalizability. The benefits of VR over physical simulation have not been established, and limited comparison studies are available. Performance measures vary widely. Although speed alone appears to be an insufficient measure, the extent to which complex motion analysis adds to outcomes-based assessment is unproven. The benefits of simulator training seem to be greatest for the most inexperienced surgeons, in acclimatizing to a 2-dimensional environment, new instrumentation, and the fulcrum effect.