• Pierre Jannin - Leader
• Florent Lalys - PhD student
• Xavier Morandi - Medical expert
• Laurent Riffaud - Medical expert

We are using machine learning based approaches along with computer vision techniques to automatically recognize the surgical phases from surgical videos (PhD of Florent Lalys). In collaboration with Carl Zeiss Medical Systems (http://wwww.zeiss.com/), we are studying the use of this approach for automatic labelling and indexation of surgical videos.

In order to better understand and describe surgical procedures using surgical process models, the field of surgical workflow segmentation has recently emerged. It aims at recognizing high-level surgical tasks in the Operating Room with the help of sensor- or human-based systems. Our novel approach focused on the automatic recognition of phases by microscope image analysis, which has never been done before. We used a hybrid method that combines supervised classification and a discrete Hidden Markov Model. We first performed feature extraction and selection on surgical microscope frames to create image databases. Then, machine learning algorithms along with data dimension reduction were assessed. The entire workflow is shown on the figure.

Then, our framework was tested on two various datasets, including one specific types of neurosurgical interventions and one type of ophthalmological surgery. The figure show the different surgical phases that have been defined by surgeons.

With this system, we are now able to recognize the surgical phase of every unknown image by computing its signature and then classifying it with machine learning techniques.This will enable the system to be used in clinical applications such as post-operative video indexation.

Definition of 6 surgical phases in Hypophyse surgeries :
Definition of 8 surgical phases in Cataract surgeries :

Here is a demonstration of the system, in the context of cataract surgeries :
<flashplayer width=600 height=400>file=https://www.irisa.fr/visages/old/team/lalys/demo_these.flv</flashplayer>