You are here: indexactivitiestheme1projectskbmrspm

Knowledge-based multi-level modelling and recognition of Surgical Processes


  • Olga Dergachyova - PhD student Funded by ANR within the Investissement d'Avenir program (Labex CAMI)

General purpose

Surgical context-aware system (CAS) is a foundation stone of operating room of the future allowing the improvement of the patient care, as well as the assistance to the surgical team in order to prevent medical erros and facilitate clinical routine. CAS requires an accurate recognition of the surgical workflow and the detection of events having place in the OR. The surgical workflow can be examined in different levels of granularity (e.g. phases, steps or actions). The existing methods for surgical workflow detection rarely associate input data with the underlying semantics and often focus on one granularity level only. The goal of this project is to implicate the formalized knowledge in the detection process and to perform a multi-level analysis of surgical procedures to enable various clinical applications.



The system for surgical workflow detection designed within the project aims for covering four aspects: 1) knowledge integration, 2) recognition of surgical processes in multiple levels, 3) use of multi-modal input data, 4) real-time performance. Formalized knowledge allows to give a symantic meaning to the input data comming from sensors and better understand the performed actions. This knowledge can be represented in form of Ontology standardizing all the notions of the domain. Also Surgical Process Models can be used to guide the recognition process of surgical procedures. Workflow detection performed in multiple granularity levels is necessary for a wide range of clinical applications from estimation of remaining time to robotic assitance. The top-down approach has been chosen for hierarchical modelling of surgical process. The multi-modal sensors getting a data of different nature (e.g. signal or capturing inverenement) can provide additional information and thus improve the recognition capacity. The input data for this project includes GRB and RGBD videos of the surgical feild and the surgical team in order to analyse the actions of the surgeon and his/her interactions with other team members. The algorithms used in the method allow a real-time computation for on-line use. Additionally, the project comprises new metrics and methods for an informative analysis and rigoureus validation addapted expressly for surgical workflow detection.

Main Collaborators

inserm rennes1 ltsi