Herzchirurgie Teil der Chirurgischen Klinik (Zentrum)
Herzchirurgie Teil der Chirurgischen Klinik (Zentrum)
Kliniken & Institute … Kliniken Chirurgische Klinik… Herzchirurgie Forschung AG Artificial…

AG Artificial Intelligence in Cardiovascular Medicine

Belongs to Klinik für Herzchirurgie and Klinik für Kardiologie, Angiologie, Pneumologie

open position announcement

We are currently seeking for a 2 years postdoc for deep learning using ground breaking medical image data sets in the form of photon counting CT images. The topic will be on explainable Deep Neural Network-based Coronary Artery Disease Characterization in a collaboration with Prof. Schoenberg from Universität Mannheim.
Please consider to apply here and mention the specific project. Application deadline: 31.01.2022

News

Dec 06, 2021 - The Miccai 2021 "AdaptOR Challenge: Deep Generative Model Challenge for Domain Adaption in Surgery" is re-opened and datasets are released under CC BY-NC-SA Licence.

November 13, 2021: We congratulate our PhD students Lalith Sharan and Roger Karl for winning the INFORMATICS4LIFE Poster Award 2021.

November 11,2021: Our talk from the Visual Intelligence Seminar Norway is online now.

October 10,2021: Register for the ELLIS Life Annual Symposium 2021 on Oct 29, 2021, organized
by the ELLIS Unit Heidelberg. We will give a talk on our latest work on AI & Medical Image Processing.

August 31, 2021 Our journal paper "Mutually improved endoscopic image synthesis and landmark detection in unpaired image-to-image translation" from our PhD student Lalith Sharan was accepted in IEEE JBHI! Congratulations!

About Us

Modern hospitals generate a vast amount of heterogeneous digital data e.g. in form of multimodal medical images, diagnostic reports, genetic information or real-time sensor streams. One core idea of precision medicine is to utilize such information for making predictions that assist in optimal treatment selection on a case-by-case basis. However, there persists an unmet clincial need to integrate such records for automatic processing, querying and adequate visualization along the entire treatment path.

The goal of our newly established working group is to leverage methods from the field of Artificial Intelligence for the analysis of heterogenous data collected from patients with cardiovascular diseases. We will especially focus on exploiting the potential of multimodal time-resolved cardiac images, such as Echocardiography, MRI, CT and Endoscopy for objective decision support in diagnosis and treatment. Beyond that, we continuously work in the direction of increasing the safety of surgical and interventional cardiovascular procedures. For example, building customized surgical training modules and intraoperative assistance systems are an integral part of our research topics.

It is our strong belief that research can only thrive through collaboration, hence we follow a translational approach and work very closely together with clinical partners. This enables us to address highly relevant clinical questions at the interface of cardiac surgery, cardiac intervention and cardiology. The recently established „Informatics 4 life“ consortium provides us with the optimal conditions to achieve this mission and to contribute towards the heart center of the future.

Head

Portrait of Jun. Prof. Dr. Sandy Engelhardt
Jun. Prof. Dr. Sandy Engelhardt
Focus

Medical Image Processing, Deep Learning, Computer-assisted Surgery


+49 6221 56-37173

Show profile

Projects

Augmented Reality and Deep Generative Models in Endoscopy

We have the ambitious goal to radically improve surgical training. In order to achieve that, we enhance surgical training simulators with augmented reality concepts to make surgical training more realistic. We coined this approach ‚Hyperrealism‘. Furthermore, we aim to create more value from endoscopic data streams by analyzing suture stitching patterns, 3D reconstructed surfaces and surgical phases.

Surgical Simulation with 3D Printing

Another core research topic of us is to build patient-specific replica from medical images. These virtual models are visualized and used for surgical training to make training more effective and procedures safer for the patient. We currently focus on how to use these models in hemodynamic environments.

Deep Cardiac MRI processing

Magnetic resonance imaging (MRI) is a valuable tool to assess geometrical, contraction-, and tissue-related pathological changes non-invasively. By the help of machine learning techniques, we characterize these properties from time-resolved image series to automatically predict therapy response

Analysis of CCTA Images of Patients with Coronary Artery Disease

Invasive heart catheters are a common approach to diagnose coronary artery disease. However, 3D-computed tomography (CT) can provide equal information on stenotic regions and plaque by being less invasive at the same time. Our aim is to analyse these data sets by the help of radiomics and deep learning methods.

Federated Deep Learning of TAVI Outcome (FLOTO)

The predictive ability of DL models highly depends on the quantity and heterogeneity of training data, but gathering large multi-institutional datasets in medicine is opposed by strong privacy regulations. Within the project group AI/ML of DZHK (German Center for Cardiovascular Research) we are organizing a federated learning initiative called FLOTO, which aims at sharing the model weights instead of the data from patients that have received transcatheter aortic valve implantation (TAVI).

Augmented Reality and Deep Generative Models in Endoscopy

tempCycleGAN: Generative Adversarial Network Improves Realism During Surgical Training - MICCAI 2018 (Youtube)

From intraoperative empirical valve inspection towards quantitative optical tracking analysis (Video)

Surgical Simulation with 3D Printing

Training Reconstructive Mitral Valve Surgery on Patient-Specific Mitral Valve Models (Youtube)

Temporal Views of Flattened Mitral Valve Geometries (Youtube)

Deep Cardiac MRI processing

3D CMR domain adaptation

Team

Head of research group

Research Fellows

Medical students

  • Portrait of Jimmy Chen
    Jimmy Chen
    Focus

    Surgical Process Modelling for Patients with Mitral Regurgitation


  • Portrait of Samantha Fischer
    Samantha Fischer
    Focus

    Simulating Mitral Valve Repair Surgeries with Patient-Specific Silicone Valve Models


  • Portrait of Arman Ghanaat
    Arman Ghanaat
    Focus

    Deep learning-based coronary artery analysis of coronary CT angiography-data


  • Julian Kuhm
    Focus

    Cardiac Motion Vector Field Visualization and Clustering


  • Josephin Marx
    Focus

    Creation of patient-specific 3D models for mitral valve reconstruction and interventional procedures: comparison of pathologies and forms of therapy


Research assistant

  • Meret Anne Siemer, Mag. iur. (Heidelberg)
    Focus

    Project Maker4Care; Medical Device Regulation


Student members

  • Portrait von Julian Brand
    Julian Brand

    Bachelor's Thesis

    Focus

    extension of a simulator for surgical interventions on mitral valves with augmented reality visualizations


  • Lukas Burger, B.Sc.

    Student Assistant

    Focus

    Depth sensing with infrared stereo and structured light approaches


  • Maximilian Hehl

    Student Assistant


  • Halvar Kelm

    Student Assistant


Alexander Rogausch
Antonia Stern
Florian Ritzmann
Halvar Kelm
Jean-Luc Busch
Jonathan Kloss
Robert Kreher
Simon Sauerzapf
Ulrike Schnaithmann

Recent publications

Laves, M.H., Tölle, M., Schlaefer, A., Engelhardt, S.
Posterior Temperature Optimization in Variational Inference for Inverse Problems.
In: Bayesian Deep Learning Workshop (NeurIPS) 2021
PDF| BibTeX  | Repo | Preprint

Sharan, L., Romano, G., Brand, J., Kelm, H., Karck, M., De Simone, R., Engelhardt, S.
Point detection through multi-instance deep heatmap regression for sutures in endoscopy
In: International Journal of Computer Assisted Radiology and Surgery 2021
PDF| BibTeX  | Repo

Cordes, J., Enzlein, T., Marsching, C., Hinze, M., Engelhardt, S., Hopf, C., Wolf, I.
M2aia—Interactive, fast, and memory-efficient analysis of 2D and 3D multi-modal mass spectrometry imaging data
In: GigaScience, Volume 10, Issue 7, July 2021
PDF| BibTeX

Toelle, M., Laves, M., Schlaefer, A.
A Mean-Field Variational Inference Approach to Deep Image Prior for Inverse Problems in Medical Imaging
In: MIDL 2021
PDF

Sharan, L., Romano, G., Koehler, S., Kelm, H., Karck, M., De Simone, R., Engelhardt, S.
Mutually improved endoscopic image synthesis and landmark detection in unpaired image-to-image translation
In: IEEE JBHI 2021
PDF| BibTeX  | Preprint| Repo

Friedrich, S., Groß, S., König, I. R., Engelhardt, S., Bahls, M., Heinz, J., Huber, C., Kaderali, L., Kelm, M., Leha, A., Rühl, J., Schaller, J., Scherer, C., Vollmer, M., Seidler, T., Friede T.
Applications of AI/ML approaches in cardiovascular medicine: A systematic review with recommendations
In: European Heart Journal - Digital Health 2021
PDF| BibTeX

Koehler, S., Hussain, T., Blair, Z., Huffaker, T., Ritzmann, F., Tandon, A., Pickardt, T., Sarikouch, S., Latus, H., Greil, G., Wolf, I., Engelhardt, S.
Unsupervised Domain Adaptation from Axial to Short-Axis Multi-Slice Cardiac MR Images by Incorporating Pretrained Task Networks
In: IEEE Transactions on Medical Imaging 2021
PDF| BibTeX  |  Preprint|  Repo

Stern, A., Sharan, L., Romano, G., Koehler, S., Karck, M., De Simone, R., Wolf, I., Engelhardt, S.
Heatmap-based 2D Landmark Detection with a Varying Number of Landmarks
In: Bildverarbeitung für die Medizin (BVM), Informatik aktuell. Springer Vieweg, Wiesbaden 2021
PDF| BibTeX  |  Preprint

Garrow, C. R., Kowalewski, K., Li, L., Wagner, M., Schmidt, M. W., Engelhardt, S., Hashimoto, D. A., Kenngott, H. G., Bodenstedt, S., Speidel, S., Müller-Stich, B. P., Nickel, F.
Machine Learning for Surgical Phase Recognition: A Systematic Review
In: Annals of Surgery 2020
PDF | BibTeX

Engelhardt, S., Sharan, L., Karck, M., De Simone, R., Wolf, I.
Generative Adversarial Networks for Stereoscopic Hyperrealism in Surgical Training
In: Bildverarbeitung für die Medizin (BVM) 2020
PDF | BibTeX | BVM2020Talk

Lichtenberg, N., Eulzer, P., Romano, G., Brcic, A., Karck, M., Lawonn, K., de Simone, R., Engelhardt, S.
Mitral valve flattening and parameter mapping for patient-specific valve diagnosis
In: International Journal of Computer Assisted Radiology and Surgery 2020
PDF | BibTeX

Wang, D.D., Qian, Z., Vukicevic, M., Engelhardt, S., Kheradvar, A., Zhang, C., Little, S.H., Verjans, J., Comaniciu, D., O’Neill, W.W., Vannan M.A.
3D Printing, Computational Modeling, and Artificial Intelligence for Structural Heart Disease
In: JACC: Cardiovascular Imaging, Volume 14, Issue 1, January 2021
PDF | BibTeX

Xiong, Z., Xia, Q., Hu, Z., Huang, N., Bian, C., Zheng, Y., Vesal, S., Ravikumar, N., Maier, A., Yang, X., Heng, P., Ni, D., Li, C., Tong, Q., Si, W., Puybareau, E., Khoudli, Y., Géraud, T., Chen, C., Bai, W., Rueckert, D., Xu, L., Zhuang, X., Luo, X., Jia, S., Sermesant, M., Liu, Y., Wang, K., Borra, D., Masci, A., Corsi, C., Vente, C., Veta, M., Karim, R., Preetha, C. J., Engelhardt, S., Qiao, M., Wang, Y., Tao, Q., Nuñez-Garcia, M., Camara, O., Savioli, N., Lamata, P., Zhao, J.
A Global Benchmark of Algorithms for Segmenting the Left Atrium from Late Gadolinium-Enhanced Cardiac Magnetic Resonance Imaging
In: Medical Image Analysis, Volume 67, January 2021
PDF | BibTeX | Preprint

Sharan, L., Burger, L., Kostiuchik, G., Wolf, I., Karck, M., De Simone, R., Engelhardt, S.
Domain gap in adapting self-supervised depth estimation methods for stereo-endoscopy
In Current Directions in Biomedical Engineering (CDBME) 2020 - (3rd Place CURAC Best Poster Award)
PDF | BibTeX

Kreher, R., Groscheck, T., Qarri, K., Preim, B., Schmeisser, A., Rauwolf, T., Christian, R., Engelhardt, S.
A Novel Calibration Phantom for Combining Echocardiography with Electromagnetic Tracking
In: Current Directions in Biomedical Engineering (CDBME) 2020
PDF | BibTeX | Calibration Phantom

Koehler, S., Tandon, A., Hussain, T., Latus, H., Pickardt T., Sarikouch, S., Beerbaum, B., Greil, G., Engelhardt, S., and Wolf, I.
How well do U-Net-based segmentation trained on adult cardiac magnetic resonance imaging data generalize to rare congenital heart diseases for surgical planning?
In: Proc. SPIE 11315, Medical Imaging 2020: Image-Guided Procedures, Robotic Interventions, and Modeling, 113151K (16 March 2020)
PDF | BibTeX | Preprint

Preetha, C.J., Wehrtmann, F.S., Sharan, L., Fan, C., Kloss, J., Müller-Stich, B.P., Nickel, F., Engelhardt, S.
Towards augmented reality-based suturing in monocular laparoscopic training
In: Proc. SPIE 11315, Medical Imaging 2020: Image-Guided Procedures, Robotic Interventions, and Modeling, 113150X (16 March 2020)
PDF | BibTeX | Preprint

Eulzer, P., Engelhardt, S., Lichtenberg, N., De Simone, R., Lawonn, K.
Temporal Views of Flattened Mitral Valve Geometries
In IEEE Trans Vis Comput Graph 2020
PDF | BibTeX | SciVis2019Talk | SciVis2019Preview | Supplemental Material Video

Pfeiffer, M., Funke, I., Robu, R. M., Bodenstedt, S., Strenger, L., Engelhardt, S., Roß, T., Clarkson, M.J., Gurusamy, K., Davidson, B.R., Maier-Hein, L., Riediger, C., Welsch, T., Weitz, J., Speidel, S.
Generating large labeled data sets for laparoscopic image processing tasks using unpaired image-to-image translation
In: Shen D. et al. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2019
PDF | BibTeX | Preprint | Dataset | Code

Engelhardt, S., Sharan, L., Karck, M., De Simone, R., Wolf, I.
Cross-Domain Conditional Generative Adversarial Networks for Stereoscopic Hyperrealism in Surgical Training
In: Shen D. et al. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2019
PDF | BibTeX | Preprint | Supplemental Material Video

Engelhardt, S., Sauerzapf, S., Preim, B., Karck, M., Wolf, I., De Simone, R.
Flexible and Comprehensive Patient-Specific Mitral Valve Silicone Models with Chordae Tendinae Made From 3D-Printable Molds
In: IJCARS Special Issue IPCAI 2019
PDF | BibTeX | Preprint | Video

Engelhardt S., De Simone R., Full P.M., Karck M., Wolf I.
Improving Surgical Training Phantoms by Hyperrealism: Deep Unpaired Image-to-Image Translation from Real Surgeries
In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2018
PDF | BibTeX | Preprint

Bernard, O., Lalande, A., Zotti, C., Cervenansky, F., Yang, X., Heng, P.A., Cetin, I., Lekadir, K., Camara, O., Gonzalez Ballester, M.A.; Sanroma, G., Napel, S., Petersen, S., Tziritas, G., Grinias, E., Khened, M., Kollerathu, V.A., Krishnamurthi, G., Rohé, M.M., Pennec, X; Sermesant, M., Isensee, F., Jäger, P., Maier-Hein K.H., Full, P.M., Wolf, I., Engelhardt, S.,  Baumgartner, C.F., Koch, L.M., Wolterink, J.M., Išgum, I., Jang, Y., Hong, Y., Patravali, J., Jain, S., Humbert, O., Jodoin, P-M.
Deep Learning Techniques for Automatic MRI Cardiac Multi-structures Segmentation and Diagnosis: Is the Problem Solved?
In: IEEE Transactions on Medical Imaging 2018
PDF | BibTeX | Preprint