You are here: indexactivitiestheme1projectsvideo

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
activities:theme1:projects:video:index [2012/02/13 17:20]
28006287
activities:theme1:projects:video:index [2016/01/05 14:43] (current)
janin
Line 4: Line 4:
  
   * [[members:​pierre.jannin:​index|Pierre Jannin]] - Leader   * [[members:​pierre.jannin:​index|Pierre Jannin]] - Leader
-  * [[members:​florent.lalys:​index|Florent Lalys]] - PhD student+  * [[members:​florent.lalys:​index|Florent Lalys]] - Post doc
   * [[members:​laurent.riffaud:​index|Laurent Riffaud]] - Medical expert   * [[members:​laurent.riffaud:​index|Laurent Riffaud]] - Medical expert
   * [[members:​david.bouget:​index|David Bouget]] - PhD student   * [[members:​david.bouget:​index|David Bouget]] - PhD student
Line 13: Line 13:
  
 The need for a better integration of the new generation of Computer-Assisted-Surgical (CAS) systems has been recently emphasized. One necessity to achieve this objective is to retrieve data from the Operating Room (OR) with different sensors, then to derive models from these data. Recently, the use of videos from cameras in the OR has demonstrated its efficiency. In this project, we propose a framework to assist in the development of systems for the automatic recognition of high level surgical tasks using microscope videos analysis. We are using machine learning based approaches along with computer vision techniques to automatically recognize surgical tasks from surgical videos (PhD of [[members:​florent.lalys:​index|Florent Lalys]]). In collaboration with Carl Zeiss Medical Systems (http://​wwww.zeiss.com/​),​ we are studying the use of this approach for automatic indexation of surgical videos. The need for a better integration of the new generation of Computer-Assisted-Surgical (CAS) systems has been recently emphasized. One necessity to achieve this objective is to retrieve data from the Operating Room (OR) with different sensors, then to derive models from these data. Recently, the use of videos from cameras in the OR has demonstrated its efficiency. In this project, we propose a framework to assist in the development of systems for the automatic recognition of high level surgical tasks using microscope videos analysis. We are using machine learning based approaches along with computer vision techniques to automatically recognize surgical tasks from surgical videos (PhD of [[members:​florent.lalys:​index|Florent Lalys]]). In collaboration with Carl Zeiss Medical Systems (http://​wwww.zeiss.com/​),​ we are studying the use of this approach for automatic indexation of surgical videos.
- 
-{{:​activities:​theme1:​projects:​video:​workflow_tmi.jpg|}} 
  
 ====== Description ====== ====== Description ======
Line 20: Line 18:
 The idea is to combine state-of-the-art computer vision techniques with time series analysis. The first step of the framework consisted in the definition of several visual cues for extracting semantic information,​ therefore characterizing each frame of the video. Five different pieces of image-based classifiers were therefore implemented. Specific pre-processing steps may also be applied for dedicated visual cue detection. Time series classification algorithms were then applied to model time-varying data. Dynamic Time Warping (DTW) and Hidden Markov Models (HMM) were tested. This association combined the advantages of all methods for better understanding of the problem. ​ The idea is to combine state-of-the-art computer vision techniques with time series analysis. The first step of the framework consisted in the definition of several visual cues for extracting semantic information,​ therefore characterizing each frame of the video. Five different pieces of image-based classifiers were therefore implemented. Specific pre-processing steps may also be applied for dedicated visual cue detection. Time series classification algorithms were then applied to model time-varying data. Dynamic Time Warping (DTW) and Hidden Markov Models (HMM) were tested. This association combined the advantages of all methods for better understanding of the problem. ​
  
-Then, our framework was validated on two various datasets, including one specific types of neurosurgical interventions and one type of ophthalmological surgery. The figure show the different surgical phases that have been defined by surgeons.+{{:​activities:​theme1:​projects:​video:​workflow_tmi.jpg|}}
  
-With this systemwe are now able to recognize the surgical phase of every unknown image by computing its signature ​and then classifying it with machine learning techniques.This will enable ​the system to be used in clinical applications such as post-operative video indexation.+Thenour framework was validated on two various datasets, including one specific types of neurosurgical interventions ​and one type of ophthalmological surgeryThe following figures show the different surgical phases that have been defined by surgeons.
  
-\\ 
-\\ 
-\\ 
-\\ 
-\\ 
-\\ 
 \\ \\
 Definition of 6 surgical phases in Hypophyse surgeries :  Definition of 6 surgical phases in Hypophyse surgeries : 
 {{ :​activities:​theme1:​projects:​video:​hypophyse.png?​560x250 }}\\ {{ :​activities:​theme1:​projects:​video:​hypophyse.png?​560x250 }}\\
 Definition of 8 surgical phases in Cataract surgeries : Definition of 8 surgical phases in Cataract surgeries :
-{{ :​activities:​theme1:​projects:​video:​cataract.png?512x188 ​}}+{{ :​activities:​theme1:​projects:​video:​8_etape_cataracte.jpg?600 }}\\
  
 +With this system, we are now able to recognize the surgical phase of every unknown image by computing its signature and then classifying it with machine learning techniques.This will enable the system to be used in clinical applications such as post-operative video indexation.
  
-====== ​Demonstration ​======+==== Results ​==== 
 + 
 + 
 + 
 +=== Demonstration ​===
  
 Here is a demonstration of the system, in the context of cataract surgeries :  ​ Here is a demonstration of the system, in the context of cataract surgeries :  ​
Line 43: Line 40:
 <​flashplayer width=600 height=400>​file=https://​www.irisa.fr/​visages/​old/​team/​lalys/​demo_these.flv</​flashplayer>​\\ <​flashplayer width=600 height=400>​file=https://​www.irisa.fr/​visages/​old/​team/​lalys/​demo_these.flv</​flashplayer>​\\
  
 +====== Publications ======
 +
 +<​html>​
 +<span id="​halpublispm">​
 +
 +<​p><​iframe width="​99%"​ scrolling="​no"​ height="​2000"​ frameborder="​0"​ src="
 +https://​haltools.archives-ouvertes.fr/​Public/​afficheRequetePubli.php?​annee_publideb=1996&​collection_exp=LTSI%3BUNIV-RENNES1&​solrQuery=%28authFullName_t%3A%22jannin%22+AND+authFullName_t%3A%22lalys%22+AND+authFullName_t%3A%22riffaud%22%29&​CB_auteur=oui&​CB_titre=oui&​CB_article=oui&​langue=Anglais&​tri_exp=annee_publi&​tri_exp2=typdoc&​tri_exp3=date_publi&​ amp;ord re_aff=TA&​Fen=Aff&​css=https://​halur1.univ-rennes1.fr/​css/​publis-HAL.css></​iframe></​p>​
 +
 +</​span>​
 +</​html>​
inserm rennes1 ltsi