Surgical Procedural Knowledge Modeling

• Main publications : 45,51,52,53,54,55,56,61,62,67,81,82,83
• Involved peoples : PhD : B. Trelhu (2007-2011), L. Riffaud (2009-2012), F. Lalys (2009-2012), Post-doc : C. Penet (2014), G. Forestier (2012)
• Main grants : Inserm, Région Bretagne, LabEx CAMI
• Main collaborations : Université of Haute Alsace (Mulhouse), ICCAS University of Leipzig (Germany), Karlsruhe Institute of Technology, Karlsruhe (Germany), MNI Mc Gill Univ. Montreal (Canada), CAMIT Kyushu University Fukuoka (Japan)
• Awards : 2nd Best PhD thesis University of Rennes 1, 2015 Best paper award at Artificial Intelligence in Medicine Europe conference (AIME) Avia, Italy June 2015
• Visiting guest researchers: M. Uemura (Kyushu University Japan, 3*2 weeks), D. Katic (Karlsruhe Institute of Technology, Germany, 3 months)
• Valorisation: Two APP protected software (ProcSide, T&F)
• Industrial collaboration: Carl Zeiss (Contract 1 2009-2012 – Contract 2 2012-2015)

The concept of surgical process models (SPM), we introduced about 15 years ago (45,46,47), is now emerging as a key element of the operating room of the future, assuming that an optimal surgical support system needs to be adapted to the different surgical times with an explicit understanding of the surgical process (named situation awareness). Surgical process models are descriptions of the surgical or interventional procedure formalized and represented into structured forms. Different granularity levels of the procedure may be studied from a high-level objective-based list of main surgical or interventional phases (46,47), to a list of the physical actions performed by the surgeons and recorded during surgery (48), to the spatial description of each physical gesture and tool (49,50). For example, a classical representation of the surgical actions performed during a surgical process consists in a sequential list of n-uplets describing each single surgical action by a verb, the operator performing the action, the surgical tool, the targeted anatomical, pathological and/or functional structure(s), the body part concerned by the performance of this action, the starting and ending times of the action, the needed information for performing this action, and some additional attributes characterizing the action. We studied two granularity levels: surgical steps and actions. We first redesigned the ontology for describing this symbolic data. This will be described in Axis 2. Then, we developed a software for observer based descriptions of individual surgical procedures from live observations or from video, resulting in individual SPMs. More than 100 procedures were described from live observations in Operating Rooms (OR). We then proposed original methods for analysing these descriptions based on sequence alignment approaches (51). We demonstrated the added value of such approaches for analysing surgical practice among surgeons and among international surgical departments (52). To the best of our knowledge, for the first time, we shown detailed quantitative and objective significant differences in surgical practice for spinal surgery and between 3 surgical centers in Canada, Germany and France (53). We also analysed the impact of different imaging techniques on processes for interventional neuroradiology (54). Observer based acquisition of individual descriptions of surgeries is a strong deadlock of this methodology. It is costly in term of human resources. To overcome such issue, we studied approaches for automatic recognition of surgical steps and actions from video images of cataract and pituitary surgery. The approach was based on computer vision and machine learning. We demonstrated the feasibility of such fully automatic approach with recognition rate of about 95% for steps and 70% for actions (55,56). This work was done in collaboration with Carl Zeiss and was awarded by the 2nd Best PhD thesis of University of Rennes 1.