STRAS
Overview of the project
STRAS is a teleoperated robotic prototype for assistance to flexible endoscopic surgery. The current version of the prototype (v2) has already been used in more than 20 procedures on animal models, mainly for simulated ESD (Endoscopic Submucosa Dissection) procedures. These medical procedures will be of major importance for the treatment of early colorectal cancers in the coming years. They are conventionally realized using standard colonoscopes, but due to the difficulty of the task, at the limit between gastro-enterology and surgery, only a few experts are actually able to perform the procedure in western countries.
STRAS provides a large set of advantages for physicians during intraluminal colorectal procedures. The platform, based on Karl Storz® Anubis® system, allows the simultaneous standard use of two (and up to three) flexible instruments in a triangulated, laparoscopic-like, configuration. All mobilities (12 in total) of the endoscope and the instruments are robotized in a modular patented architecture (see below), allowing easy setup and manipulation of the medical components. STRAS provides great stability of the endoscope and instruments and major improvements in the comfort, intuitivness, and precision of control thanks to dedicated, patent-pending (see below), master interfaces. As demonstrated in many in vivo trials, STRAS can be easily teleoperated by a single surgeon for performing complex procedures, resulting in shorter and less tiring interventions.
STRAS was developed by ICube (lab of the university of Strasbourg and the French National Center for Research (CNRS)) in partnership with the IRCAD (medical partner, Dr B. Dallemagne), with the technical support from Karl Storz (Tuttlingen, Germany). The project was funded by the French Minister of Economy (FUI Project ISIS) and by SATT Conectus through a "Prematuration" project.
The work is currently brought forward by the same partners with the financial support from SATT Conectus with the objective to reach clinical trials. Telerobotlabs (Genova, Italy) acts as an outsourcing company for mechanical realization.
The video at this link shows some of the capabilities of the system. This other video explains the current developments realized on STRAS.
Scope
Flexible endoscopy has long been used for diagnosis purposes only. But with the general trend in surgery to try to minimize invasiveness it has come a major tool for surgical procedures as shown by the development of specific flexible surgical platforms. Unfortunately these systems are not directly adapted to surgery. Several surgeons have to coordinate to manipulate the available DOFs, which result in difficult operations and potential high functioning costs. We envision a solution in the development of robotic tools for flexible endoscopic surgery. The aim of the ISIS project is to show that by using robotics it is possible for a single surgeon to perform surgical procedures intuitively with flexible systems. The chosen path to this goal is to build on existing asserted medical systems, such as flexible endoscopes manufactured by Karl Storz, rather than developing complete new systems. Issues arise at several levels (see the following for more details) :
- Small-size mechanical design for developing light, compact and modular slave systems
- Low-level control of cable-driven systems with significant non-linearities
- Cartesian control of flexible instrument made difficult by imperfect models and lack of measurement solutions
Overall the project includes mechanical development, low-level control, high level strategies and measurement solutions development.
Keywords: Flexible endoscopy, continuum robots, teleoperation
Patents
- M. de Mathelin, F. Le Bastard, F. Nageotte, P. Zanne, L. Zorn
Dispositif d'interface maître pour système endoscopique et installation comprenant un tel dispositif ( Master interface device for a motorised endoscopic system and installation comprising such device ). Dépôt de brevet français ( déposant : U. Strasbourg, CNRS et IRCAD ), n° FR1450560, le 23 janvier 2014, publication n° FR3016512, le 24 juillet 2015. Demande PCT, n° PCT/EP2015/051189, le 22 janvier 2015, publication n° WO 2015/110495, le 30 juillet 2015
- L. Zorn, M. de Mathelin, P. Zanne, F. Nageotte
Dispositif d’instrumentation motorisé et modulable et système d’endoscope comprenant un tel dispositif ( Motorised and modular instrumentation device and endoscopy system comprising such a device ). Dépôt de brevet français ( déposant : U. Strasbourg et CNRS ), n° FR12/52109, le 8 mars 2012, publication n° FR2987734, le 13 septembre 2013, délivrance , le 11 avril 2014. Demande PCT, n° PCT/FR2013/050487, le 8 mars 2013, publication n° WO2013132194, le 12 septembre 2013. Dépôt de brevet européen le 8 mars 2013, n° EP13786533.3, publication le 14 janvier 2015, n° EP2822446. Dépôt de brevet US le 8 mars 2013, n° US14/382,915, publication le 25 décembre 2014, n° US2014378761.
Related publications
- L. Zorn, F. Nageotte, P. Zanne, A. Legner , B. Dallemagne , J. Marescaux , M. de Mathelin
A Novel Telemanipulated Robotic Assistant for Surgical Endoscopy: Preclinical Application to ESD, IEEE Transactions on Biomedical Engineering, April 2018 (Abstract IEEExplore)
- P. Cabras, F. Nageotte, P. Zanne, C. Doignon
An adaptive and fully automatic method for estimating the 3D position of bendable instruments using endoscopic images, International Journal of Medical Robotics and Computer Assisted Surgery
- P. Cabras, D. Goyard , F. Nageotte, P. Zanne, C. Doignon
Comparison of Methods for Estimating the Position of Actuated Instruments in Flexible Endoscopic Surgery, International Conference on Intelligent Robots and Systems (IROS), Chicago, United States, septembre 2014
- A. de Donno, F. Nageotte, P. Zanne, L. Zorn, M. de Mathelin
Master / Slave Control of Flexible Instruments for Minimally Invasive Surgery, IEEE International Conference on Intelligent Robots and Systems, Tokyo, Japan, novembre 2013
- A. de Donno, L. Zorn, P. Zanne, F. Nageotte, M. de Mathelin
Introducing STRAS: A New Flexible Robotic System for Minimally Invasive Surgery, IEEE International Conference on Robotics and Automation (ICRA 2013), Karlsruhe, Germany, mai 2013
Persons involved in the project
- PhD students: T. Poignonec (from October 2019), R. Aleluia Porto (since December 2016)
- Research engineers: Ph. Zanne (Low-level software development, electrical design and realization), L. Goffin (GUI Software development and software validation)
- Researchers: F. Nageotte, M. de Mathelin
- Past engineers : L. Zorn (mechatronic design and realization), F. LeBastard (Mechatronic design)
- Past PhD students: P. Cabras (vision-based measurement), A. De Donno (teleoperation), B. Bardou (teleoperation and control), L. Ott (vision-based control)
- Past Master students : T. Poignonec (backlash compensation), X.T. Ha (visual tracking), M.A. Falek (telemanipulation analysis), N. Shepeleva (vision-based measurement), D. Goyard (learning and vision-based measurements), F. Le Bastard (mechanical transmission modelling)
Contacts
Florent Nageotte, Philippe Zanne, Lucile Zorn
Partners
The EASE / STRAS project is developed in collaboration with Karl Storz GmBH and surgeons from the IRCAD (Dr B. Dallemagne, Dr S. Peretta) and the IHU (A. Legner). EASE prototype has been partly outsourced to TelerobotLabs (Genova, Italy). Axilum Robotics provides support for software and regulations.
R. aleluia Porto is parlty funded by CAMI labex. P. Cabras was funded by CAMI labex and his thesis project was in collaboration with Agathe team (ISIR Paris). STRAS v2 realization was partly outsourced to Sandmann company (Bennwihr Gare, France) and Opticab.
Recent facts
- April 2019: An article demonstrating the medical interest of the STRAS platform through preclinical trials has been publised in "Gastroenterology"
- April 28, 2018: An article in the magazine "L'Express" speaks about the STRAS robot
- April 27, 2018: Rafael Aleluia Porto receives the award for the best presentation at the FMTS days in Strasbourg
- April 2018: Published paper in IEEE Transactions on biomedical Engineering appears in "TBME Highlights"
- March 28 / 29, 2018: The novel prototype was successfully tested on animals (4 ESD realized).
- March 19, 2018: Tests on animals at the IRCAD. 3 ESD successfully realized.
- November 2017: Presentation at Surgetica conference in Strasbourg
- September 2017: Finalist "best paper award" at CRAS Workshop in Montpellier
In details
Visual control of flexible endoscopes
STRAS : a robotic system for intraluminal surgery
During the ISIS project, we have developed a robotic prototype based on the Anubis platform from Karl Storz. The proposed system is modular and consists of three kinds of modules:
- an endoscope module, which motorizes the four-ways deflection of the main endoscope tip
- instruments modules, which enables the two-ways deflection of instruments. They come in two flavours: electrical instruments modules for instruments needing electrical pluging, such as electro-coagulation tools and mechanical instruments needing an opening / closing capability, such as scisors or graspers.
- Translation and Rotation modules (T/RMs) which allow the rotation and translation of the instruments inside the channels of the main endoscope.
TR/Ms and the endoscope modules are mounted onto passive positioning arms, which can be attached to the operating table. With this structure only two T/RMs are needed, whatever the number of instruments required for the surgery.
The whole motorization is located at the proximal side, which makes the robotic system easily compatible with medical constraints.
This slave system can be teleoperated using two master interfaces, enabling a single user to control all degrees of freedom. First tests have been performed in the lab, which have shown that STRAS allows a better and more comfortable control of the flexible instruments than in the manual version.
Control of flexible systems
Flexible systems used in the STRAS prototype have very particular behaviours due to their kinematic properties (singularities at the straight position) and their mechanical actuation (cable driven systems) which makes their control difficult. We are investigating how to improve the control of these systems at two levels: At low level we try to understand the local behaviour of the flexible systems and at higher level we analyze the effect of the choices of master / slave control modes and mappings on the overall control capabilities.
Visual measurement
Because of frictions, backlash and deadzones, it is difficult to estimate the position of instruments using only internal sensors such as motor encoders. On the other hand there are no commercially available sensors which can be integrated inside small diameter instruments and which would allow to measure small enough radii of curvature. For improving the control of flexible instruments, we work on the use of the endoscopic images for measuring the positions and motions of the instruments. This information can be fused with encoders measurements to improve the obtained accuracy.
In the PhD thesis of Paolo Cabras, we developed image processing and computer vision tools allowing to measure the position of flexible instruments using mainly the embedded endoscopic camera. The approach is based on the use of color markers attached to the bendable part of the flexible instruments. We developed graph-based techniques which allow to extract and reconstruct the image structure of the instruments despite possible partial occultations, the strong specularities caused by direct lighting and numerous reflections onto organs. Because of the space necessary for allowing instruments to slide and rotate inside the guide channels, it is difficult to accurately modelize the position and orientation of instruments with respect to the embedded camera. Consequently, standard model-based approaches fail to estimate the position of instruments even when features are correctly extracted. Therefore we proposed an adaptive method, which allows to automatically adapt to change in position and orientation of the instrument in the channels and provide good estimation capabilities.
We also investigated the possibility to use learning-based approches for pose estimation. Learning techniques can avoid modeling at all the link between positions and images and learn it from demonstrations if a ground truth can be made available during a training stage. We adapted several learning techniques, such as RBF and LWR, to the estimation problem and were able to obtain similar accuracy than with models, but with the avantage of using simpler image features.
Major past facts (see also section "Recent Facts")
- Mai 2 2016: Presentation of STRAS to 250 engineers from Siemens, during Siemens days at IRCAD
- Feb. 24 2016: PhD defense of Paolo Cabras on the pose estimation of flexible instruments of STRAS from monocular vision. The comitee was composed by A. Casals (President), G. Morel (reviewer), N. Andreff (reviewer), M. de Mathelin (examiner), C. Doignon (PhD supervisor), F. Nageotte (PhD Advisor)
- Jun. 26 2015: STRAS successfully tested for single port procedures: Simulated appenditectomy, cholecystectomy, dissection of the gastro-oesophageal junction
- May 11: STRAS presented to J.M Le Guen, State secretary for relations with the parliament and physician
- Dec. 2014 : Success of in vivo trials of ESD on pigs
- Sep. 2014 : Start of a new project, called "EASE", for developing a clinical version of STRAS. EASE is Funded by SATT Conectus with support of Karl Storz and IRCAD
- Sep 2014 : new project for developing a clinical version of STRAS funded by SATT Conectus with support of Karl Storz
- Sep. 1/2 2014 : in vivo experiments at the IRCAD
- Feb. 12 2014 : Alsace 20 (TV broadcaster) has filmed STRAS v2 for broadcast program Lab20 to be seen on friday feb. 14 and on the website of Alsace 20 from feb. 17.
- Feb. 5 2014 : In vivo tests of ESD with a novice surgeon
- Jan. 30 2014 : Demonstration of STRAS v2 to M. F. Hollande, president of the republic during his visit at IRCAD
- Jan. 30 2014 : Demonstration of STRAS v2 to Mrs Storz, CEO of Karl Storz
- Jan. 10 2014 : Demonstration of STRAS v2 to a Karl Storz delegation
- Jan. 7 2014 : Demonstration of STRAS v2 to a delegation of Covidien
- Dec. 19 2013 : Second operation tests on animals
- Dec. 13 2013 : A. De Donno successfully defended his PhD. The defense took place in the "Salle des Actes" at the faculty of Medicine in Strasbourg. The jury was composed by Ph. Cinquin (President), A. Menciassi (reviewer), J. Szewczyk (reviewer), M. de Mathelin and F. Nageotte.
- Dec. 5 2013 : First in vivo tests of STRAS v2. Our medical partners at IRCAD successfully performed an ESD in an animal.
- Nov. 4 2013 : F. Nageotte presented control strategies for STRAS at IROS in Tokyo.
- Oct. 29 2013 : First ex vivo tests of STRAS v2. Successful fake ESD in a pig stomach.
- Oct. 24 2013 : First lab tests of the new version of STRAS (v2) on chicken breast.
- Oct 2013 : Second version of STRAS has been assembled
- May 2013 : A. De Donno presented STRAS at IEEE ICRA in Karlsruhe
- Presentation of STRAS robotic system during visits of Siemens HealthCare CEO (march 12), Mrs Sybil Storz, CEO of Karl Storz (march 13) and M. Louis Gallois (Commissaire général à l'investissement) (march 14).
- Nov. 30 2012 : Presentation of STRAS robotic system at the IRCAD during Sanofi CEO visit
- Nov. 29 2012 : first trials by IRCAD surgeons
- oct. 2012 : presentation of surgical simulator at the "Fête de la Science" in Strasbourg
- Oct. 2012 : Paolo Cabras has joined the project as PhD student working on image processing tools for system control
- 2011 : Presentation of STRAS system at the IRCAD lab during visit of Laurent Wauquiez, minister of research and higher education