A method for going from 2D laparoscope to 3D acquisition of surface landmarks by a novel computer vision approach

Purpose: This paper presents a method to use the Smart Trocars—our new surgical instrument recognition system—or any accurate localization system of surgical instrument for acquiring intraoperative surface data. Complex laparoscopic surgeries need a proper guidance system which requires registering the preoperative data from a CT or MRI scan to the intraoperative patient state. The Smart Trocar can be used to localize the instruments when it comes to contact with the soft tissue surface. Method: Two successive views through the laparoscope at different angles with the 3D localization of a fixed tool at one single location using the Smart Trocars can point out visible features during the surgery and acquire their location in 3D to provide a depth map in the region of interest. In other words, our method transforms a standard laparoscope system into a system with three-dimensional registration capability. Result: This method was initially tested on a simulation for uncertainty assessment and then on a rigid model for verification with an accuracy within 2 mm distance. In addition, an in vivo experiment on pig model was also conducted to investigate how the method might be used during a physiologic respiratory cycle. Conclusion: This method can be applied in a large number of surgical applications as a guidance system on its own or in conjunction with other navigation techniques. Our work encourages further testing with realistic surgical applications in the near future.