biomedical image processing research topics

Biomedical Image Processing

© 2011
Thomas Martin Deserno 0

Inst. Medizinische Informatik, RWTH Aachen, Aachen, Germany

You can also search for this editor in PubMed Google Scholar

Covers major aspects and methods of biomedical imaging
Contributing authors are internationally recognized experts from all over the world
Integrates physics and biomedicine
Provides a reference for researchers and practicioners
Graduate students benefit from the didactic introduction and detailed explanations of the techniques
Includes supplementary material: sn.pub/extras

Part of the book series: Biological and Medical Physics, Biomedical Engineering (BIOMEDICAL)

83k Accesses

130 Citations

9 Altmetric

This is a preview of subscription content, log in via an institution to check access.

Access this book

Subscribe and save.

Get 10 units per month
Download Article/Chapter or eBook
1 Unit = 1 Article or 1 Chapter
Cancel anytime
Available as PDF
Read on any device
Instant download
Own it forever
Compact, lightweight edition
Dispatched in 3 to 5 business days
Free shipping worldwide - see info
Durable hardcover edition

Tax calculation will be finalised at checkout

Other ways to access

Licence this eBook for your library

Institutional subscriptions

About this book

Medical Image Processing

Medical Image Processing: Mathematical Modelling and Numerical Resolution

Image Processing in Biomedical Science

Image processing
Information storage and retrieval
Medical imaging
Pattern recognition

Table of contents (22 chapters)

Front matter, fundamentals of biomedical image processing.

Thomas M. Deserno

Image Formation

Fusion of pet and mri for hybrid imaging.

Zang-Hee Cho, Young-Don Son, Young-Bo Kim, Seung-Schik Yoo

Cardiac 4D Ultrasound Imaging

Jan D’hooge

Image Enhancement

Morphological image processing applied in biomedicine.

Roberto A. Lotufo, Leticia Rittner, Romaric Audigier, Rubens C. Machado, André V. Saúde

Medical Image Registration

Daniel Rueckert, Julia A. Schnabel

Feature Extraction and Selection

Texture in biomedical images.

Maria Petrou

Multi-Scale and Multi-Orientation Medical Image Analysis

Bart M. ter Haar Romeny

Feature Extraction and Selection for Decision Making

Agma J. M. Traina, Caetano Traina Jr., André G. R. Balan, Marcela X. Ribeiro, Pedro H. Bugatti, Carolina Y. V. Watanabe et al.

Segmentation

Parametric and non-parametric clustering for segmentation.

Hayit Greenspan, Tanveer Syeda-Mahmood

Region-Based Segmentation: Fuzzy Connectedness, Graph Cut and Related Algorithms

Krzysztof Chris Ciesielski, Jayaram K. Udupa

Model-Based Segmentation

Tobias Heimann, Hervé Delingette

Classification and Measurements

Classication and measurements, melanoma diagnosis.

Alexander Horsch

CADx Mammography

Lena Costaridou

Quantitative Medical Image Analysis for Clinical Development of Therapeutics

Mostafa Analoui

From the reviews:

“The book … contains useful information for anyone interested in medical and biomedical image processing. … this book is a rich and valuable compendium of chapters that cover the current art in biomedical image processing. This is a reference book or textbook for imaging informatics professionals, radiologists, physicists, researchers, scientists, and students interested in the rapidly changing world of imaging and image processing tools and options.” (Janice Honeyman-Buck, Journal of Digital Imaging, Vol. 25, 2012)

“The book is indeed comprehensive, covering all steps in image processing: from image formation via image enhancement, image visualization to image analysis and image management. … It is ‘comprehensive but short, up-to-date but essential and detailed but illustrative’. Also it can be used both by novices (with a mathematical and physics background) and experts, as the editor expressed as a goal. I therefore can recommend this textbook to all interested in learning more about or getting an overview of biomedical image processing methods.” (Arie Hasman, International Journal of Medical Informatics, Vol. 80, 2011)

“A text that is suitable for clinicians, scientists, and engineers interested in the important topic of how clinical and biomedical images can best be processed for features quantification, and to enhance important visual detail. … The chapters are directed at both students and professors. The aim of each chapter is to cover recent advances and to provide up-to-date information, which is done well. … The text is eminently readable, and would be quite suitable as an undergraduate course text.” (Edward J. Ciaccio, BioMedical Engineering OnLine, Vol. 10 (101), 2011)

Editors and Affiliations

Thomas Martin Deserno

Bibliographic Information

Book Title : Biomedical Image Processing

Editors : Thomas Martin Deserno

Series Title : Biological and Medical Physics, Biomedical Engineering

DOI : https://doi.org/10.1007/978-3-642-15816-2

Publisher : Springer Berlin, Heidelberg

eBook Packages : Physics and Astronomy , Physics and Astronomy (R0)

Hardcover ISBN : 978-3-642-15815-5 Published: 02 March 2011

Softcover ISBN : 978-3-642-26730-7 Published: 21 April 2013

eBook ISBN : 978-3-642-15816-2 Published: 01 March 2011

Series ISSN : 1618-7210

Series E-ISSN : 2197-5647

Edition Number : 1

Number of Pages : XXXVII, 595

Topics : Biological and Medical Physics, Biophysics , Image Processing and Computer Vision , Pattern Recognition , Medicine/Public Health, general , Imaging / Radiology

Publish with us

Policies and ethics

Find a journal
Track your research

Biomedical Image Segmentation and Analysis

Loading... Editorial 01 February 2023 Editorial: Biomedical image segmentation and analysis Naveen Aggarwal 936 views 0 citations

Loading... Original Research 05 December 2022 Early and accurate detection of melanoma skin cancer using hybrid level set approach Mahmoud Ragab , 4 more and Romany F. Mansour 24,041 views 10 citations

Original Research 14 November 2022 Medical ultrasound image speckle reduction and resolution enhancement using texture compensated multi-resolution convolution neural network Muhammad Moinuddin , 4 more and Jong Chul Ye 3,379 views 4 citations

Technology and Code 07 October 2022 Short-pulsed micro-magnetic stimulation of the vagus nerve Hongbae Jeong , 2 more and Giorgio Bonmassar 4,086 views 5 citations

Loading... Systematic Review 30 September 2022 Deep learning techniques for cancer classification using microarray gene expression data Surbhi Gupta , 2 more and Ashutosh Sharma 9,589 views 28 citations

Original Research 30 August 2022 Expression of long noncoding RNA uc.375 in bronchopulmonary dysplasia and its function in the proliferation and apoptosis of mouse alveolar epithelial cell line MLE 12 Tianping Bao , 6 more and Zhaofang Tian 1,757 views 3 citations

Original Research 08 July 2022 Differential Gene Analysis of Trastuzumab in Breast Cancer Based on Network Pharmacology and Medical Images Yuan Lu , 5 more and Yueyun Liu 2,655 views 3 citations

Original Research 10 May 2022 Dynamic Target Tracking Method Based on Medical Imaging Guofeng Qin , 5 more and Ruihan Wang 2,541 views 1 citations

Original Research 13 April 2022 Fast Speckle Noise Suppression Algorithm in Breast Ultrasound Image Using Three-Dimensional Deep Learning Xiaofeng Li , 2 more and Yanbo Wei 2,143 views 14 citations

Help | Advanced Search

Electrical Engineering and Systems Science > Image and Video Processing

Title: biomedical image segmentation: a systematic literature review of deep learning based object detection methods.

Abstract: Biomedical image segmentation plays a vital role in diagnosis of diseases across various organs. Deep learning-based object detection methods are commonly used for such segmentation. There exists an extensive research in this topic. However, there is no standard review on this topic. Existing surveys often lack a standardized approach or focus on broader segmentation techniques. In this paper, we conducted a systematic literature review (SLR), collected and analysed 148 articles that explore deep learning object detection methods for biomedical image segmentation. We critically analyzed these methods, identified the key challenges, and discussed the future directions. From the selected articles we extracted the results including the deep learning models, targeted imaging modalities, targeted diseases, and the metrics for the analysis of the methods. The results have been presented in tabular and/or charted forms. The results are presented in three major categories including two stage detection models, one stage detection models and point-based detection models. Each article is individually analyzed along with its pros and cons. Finally, we discuss open challenges, potential benefits, and future research directions. This SLR aims to provide the research community with a quick yet deeper understanding of these segmentation models, ultimately facilitating the development of more powerful solutions for biomedical image analysis.

Subjects:	Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
Cite as:	[eess.IV]
	(or [eess.IV] for this version)
	Focus to learn more arXiv-issued DOI via DataCite

Submission history

Access paper:.

HTML (experimental)
Other Formats

References & Citations

Google Scholar
Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

biomedical-image-processing

Here are 159 public repositories matching this topic..., dana-farber-aios / pathml.

Tools for computational pathology

Updated Aug 19, 2024

cambridgeltl / visual-med-alpaca

Visual Med-Alpaca is an open-source, multi-modal foundation model designed specifically for the biomedical domain, built on the LLaMa-7B.

Updated Mar 11, 2024

seung-lab / connected-components-3d

Connected components on discrete and continuous multilabel 3D & 2D images. Handles 26, 18, and 6 connected variants; periodic boundaries (4, 8, & 6)

Updated Jul 26, 2024

jvanvugt / pytorch-unet

Tunable U-Net implementation in PyTorch

Updated May 5, 2020

seung-lab / euclidean-distance-transform-3d

Euclidean distance & signed distance transform for multi-label 3D anisotropic images using marching parabolas.

Updated Jul 17, 2024

OpenEIT / OpenEIT

Python based dashboard for real-time Electrical Impedance Tomography including image reconstruction using Back Projection, Graz Consensus and Gauss Newton methods

Updated Jul 6, 2023

tkuanlun350 / 3DUnet-Tensorflow-Brats18

3D Unet biomedical segmentation model powered by tensorpack with fast io speed

Updated Oct 8, 2018

zudi-lin / pytorch_connectomics

PyTorch Connectomics: segmentation toolbox for EM connectomics

Updated Aug 3, 2024

ELEKTRONN / elektronn3

A PyTorch-based library for working with 3D and 2D convolutional neural networks, with focus on semantic segmentation of volumetric biomedical image data

Updated Dec 6, 2023

mlcommons / GaNDLF

A generalizable application framework for segmentation, regression, and classification using PyTorch

Updated Sep 3, 2024

anjanatiha / Pneumonia-Detection-from-Chest-X-Ray-Images-with-Deep-Learning

Detecting Pneumonia in Chest X-ray Images using Convolutional Neural Network and Pretrained Models

Updated Dec 16, 2018
Jupyter Notebook

seung-lab / kimimaro

Skeletonize densely labeled 3D image segmentations with TEASAR. (Medial Axis Transform)

Updated Aug 2, 2024

sacmehta / YNet

Y-Net: Joint Segmentation and Classification for Diagnosis of Breast Biopsy Images

Updated Dec 30, 2020

seung-lab / cloud-volume

Read and write Neuroglancer datasets programmatically.

Updated Aug 29, 2024

yuanqing811 / ISIC2018

ISIC 2018: Skin Lesion Analysis Towards Melanoma Detection

Updated Dec 8, 2022

BiaPyX / BiaPy

Open source Python library for building bioimage analysis pipelines

Updated Sep 4, 2024

mattmacy / torchbiomed

Datasets, Transforms and Utilities specific to Biomedical Imaging

Updated Jul 3, 2018

mirzaevinom / promise12_segmentation

Codes that I have written to complete promise12 prostate segmentation competition.

Updated Dec 12, 2017

seung-lab / dijkstra3d

Dijkstra's Shortest Path for 6, 18, and 26-Connected 3D (Volumetric) Image Volumes

Updated Jul 6, 2024

hoang-ho / Skin_Lesions_Classification_DCNNs

Transfer Learning with DCNNs (DenseNet, Inception V3, Inception-ResNet V2, VGG16) for skin lesions classification

Updated Nov 14, 2019

Improve this page

Add a description, image, and links to the biomedical-image-processing topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the biomedical-image-processing topic, visit your repo's landing page and select "manage topics."

Biomedical Imaging

The current plethora of imaging technologies such as magnetic resonance imaging (MR), computed tomography (CT), position emission tomography (PET), optical coherence tomography (OCT), and ultrasound provide great insight into the different anatomical and functional processes of the human body. While such imaging technologies have improved significantly over the years to provide improved resolution and signal-to-noise ratio (SNR), as well as reduced acquisition speed, there are still many fundamental trade-offs between these three aspects due to operational, financial, and physical constraints. As such, the acquired data can be largely unusable in raw form due to factors such as noise, technology-related artifacts, poor resolution, and contrast. Furthermore, given the complexities of biomedical imaging data, it is often difficult for research scientists and clinicians to interpret and analyze the acquired data in a meaningful and efficient fashion. Researchers in the VIP group are developing novel and exciting ways to address the issues associated with biomedical imaging to assist clinicians, radiologists, pathologists, and clinical research scientists in better visualizing, diagnosing, and understanding various diseases affecting the human body.

Related people

Alexander Wong , David A. Clausi , Paul Fieguth

Robert Amelard , Keyvan Kasiri , Daniel S. Cho , Andre Carrington , Ameneh Boroomand , Farnoud Kazemzadeh , Shahid Haider , M. Javad Shafiee , Jiange Grace Liu , Shelley Wang , Devinder Kumar , Edward Li , Audrey Chung , Brendan Chwyl , Jason Deglint

Akshaya Mishra , Ed Jernigan , Justin Eichel , Nezam Kachouie , Peter Iles , Shimon Schwartz , Wen Zhang , Shannon Puddister , Jeffrey Glaister , Shiva Zaboli , Chenyi Liu , Andrew Cameron , Dorothy Lui , Christian Scharfenberger , Zohreh Azimifar

Related demos

Skin Cancer Detection

Bias Field Correction in Endorectal Diffusion Imaging

Enhanced Low-dose Computed Tomography

Correlated Diffusion Imaging

Photoplethysmographic Imaging

Multiplexed Optical High-coherence Interferometry

Related publications

Journal articles.

Haider, S. , A. Cameron , P. Siva , D. Lui , M. J. Shafiee , A. Boroomand , N. Haider, and A. Wong , " Fluorescence microscopy image noise reduction using a stochastically-connected random field model ", Nature Scientific Reports , Submitted. Details

Boroomand, A. , A. Wong , E. Li, D. Cho , B. Ni, and K. Bizheva, " Multi-penalty conditional random field approach to super-resolved reconstruction of optical coherence tomography images ", Biomedical Optics Express , Accepted. Details

Liu, C. , A. Wong , A. A. Moayed, P. Fieguth , H. Bie, and K. Bizheva, " Automatic tracking of pupillary dynamics from in-vivo functional optical coherence tomography images ", Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization , Accepted. Details

Cho, D. , A. Wong , D. A. Clausi , J. Callaghan, and J. Yates, " Markov-Chain Monte Carlo based Image Reconstruction for Streak Artifact Reduction on Contrast Enhanced Computed Tomography ", Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization , Accepted. Details

Wong, A. , and J. Scharcanski, " Monte Carlo Despeckling of Transrectal Ultrasound (TRUS) Images of the Prostate ", Digital Signal Processing , Accepted. Details

Glaister, J. , A. Wong , and D. A. Clausi , " Automatic segmentation of skin lesions from dermatological photographs using a joint probabilistic texture distinctiveness approach ", IEEE Transactions on Biomedical Engineering , Accepted. Details

Amelard, R. , C. Scharfenberger , F. Kazemzadeh , K. J. Pfisterer, B. S. Lin, A. Wong , and D. A. Clausi , " Non-contact transmittance photoplethysmographic imaging (PPGI) for long-distance cardiovascular monitoring ", Nature Scientific Reports , vol. 6, no. 5, October, 2015. Details

Shafiee, M. J. , S. Haider , A. Wong , D. Lui , A. Cameron , A. Modhafar, P. Fieguth , and M. A. Haider, " Apparent Ultra-High b-value Diffusion-Weighted Image Reconstruction via Hidden Conditional Random Fields ", TRANSACTIONS ON MEDICAL IMAGING , vol. 34, no. 5: IEEE, 2015. Details

Lui, D. , A. Cameron , A. Modhafar, D. Cho , and A. Wong , " Low-dose computed tomography via Spatially-adaptive Monte Carlo reconstruction ", Computerized Medical Imaging and Graphics , vol. 37, issue 7-8, pp. 438 - 449, October, 2013. Details

Cameron, A. , D. Lui , A. Boroomand , J. Glaister , A. Wong , and K. Bizheva, " Stochastic speckle noise compensation in optical coherence tomography using non-stationary spline-based speckle noise modelling ", Biomedical Optics Express , vol. 4, no. 9, August, 2013. Details

Lui, D. , A. Modhafar, J. Glaister , A. Wong , and M. A. Haider, " Monte Carlo Bias Field Correction in Endorectal Diffusion Imaging ", IEEE Transactions on Biomedical Engineering , vol. 61, issue 2, pp. 368 - 380, August, 2013. Details

Wong, A. , J. Glaister , A. Cameron , and M. A. Haider, " Correlated Diffusion Imaging ", BMC Medical Imaging , 2013. Details

Wong, A. , K. Gallagher, and J. Callaghan, " Computerized System for Measurement of Muscle Thickness Based on Ultrasonography ", Computer Methods in Biomechanics and Biomedical Engineering , 2013. Details

Glaister, J. , R. Amelard , A. Wong , and D. A. Clausi , " MSIM: Multi-Stage Illumination Modeling of Dermatological Photographs for Illumination-Corrected Skin Lesion Analysis ", IEEE Transactions on Biomedical Engineering , vol. 60, issue 7, pp. 1873 - 1883, November, 2013. Details

Schwartz, S. , C. Liu , A. Wong , D. A. Clausi , P. Fieguth , and K. Bizheva, " Energy-guided learning approach to compressive FD-OCT ", Optics Express , vol. 21, issue 1, pp. 329-344, 2013. Details

Liu, C. , A. Wong , K. Bizheva, P. Fieguth , and H. Bie, " Homotopic, non-local sparse reconstruction of optical coherence tomography imagery ", Optics Express , vol. 20, no. 9, pp. 10200-10211, April, 2012. Details

Schwartz, S. , A. Wong , and D. A. Clausi , " Compressive fluorescence microscopy using saliency-guided sparse reconstruction ensemble fusion ", Optics Express , vol. 20, issue 16, pp. 17281–17296, July, 2012. Details

Karimi, A., A. Wong , and K. Bizheva, " Automated detection and cell density assessment of keratocytes in the human corneal stroma from ultrahigh resolution optical coherence tomograms ", Biomedical Optics Express , 2012. Details

Wong, A. , S. Hariri, and K. Bizheva, " Tensor total variation approach to optical coherence tomography reconstruction ", Biomedical Optics Express , 2012. Details

Moayed, A. A., S. Hariri, C. Liu , A. Wong , V. Choh, and K. Bizheva, " Stimulus Specific Pupil Dynamics Measured In-vivo in Birds (Gallus Gallus Domesticus) with Ultrahigh Resolution Optical Coherence Tomography ", Investigative Ophthalmology and Visual Science , 11, vol. 53, issue 6869, 2012. Details

Wong, A. , E. Irving, R. Genest, V. Choh, and N. Chandrashekar, " Automatic three-dimensional reconstruction of the chick eye based on high resolution photographic images ", Computer Methods in Biomechanics and Biomedical Engineering , 2011. Details

Wong, A. , and J. Scharcanski, " Dynamic Fisher-Tippett Region Merging Approach to Transrectal Ultrasound Prostate Lesion Segmentation ", IEEE Transactions on Information Technology in BioMedicine , 2011. Details

Kazemzadeh, F. , A. Wong , B. B. Behr, and A. R. Hajian, " Depth Profilometry via Multiplexed Optical High-Coherence Interferometry ", PLoS ONE : PLoS. Details

Conference papers

Kasiri, K. , P. Fieguth , and D. A. Clausi , " Self-similarity measure for multi-modal image registration ", IEEE International Conference on Image Processing (ICIP) , Accepted. Details

Kasiri, K. , P. Fieguth , and D. A. Clausi , " Sorted self-similarity for multi-modal image registration ", International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) , Accepted. Details

Kasiri, K. , P. Fieguth , and D. A. Clausi , " Cross modality label fusion in multi-atlas segmentation ", IEEE International Conference on Image Processing , 2014. Details

Kasiri, K. , H. Sekkati , and D. A. Clausi , " Quality assessment in digital pathology images ", Imaging Network Ontario-ImNO, 2016. Details

Amelard, R. , C. Scharfenberger , A. Wong , and D. A. Clausi , " Illumination-compensated non-contact imaging photoplethysmography via dual-mode temporally-coded illumination ", SPIE Photonics West, Multimodal Biomedical Imaging X , February, 2015. Details

Amelard, R. , C. Scharfenberger , A. Wong , and D. A. Clausi , " Non-contact assessment of melanin distribution via multispectral temporal illumination coding ", SPIE Photonics West, Multimodal Biomedical Imaging X , February, 2015. Details

Kasiri, K. , P. Fieguth , and D. A. Clausi , " Structural Representations for Multi-modal Image Registration Based on Modified Entropy ", International Conference on Image Analysis and Recognition (ICIAR) , Accepted. Details

Boroomand, A. , B. Tan, A. Wong , and K. Bizheva, " Axial resolution improvement in spectral domain optical coherence tomography using a depth-adaptive maximum-a-posterior framework ", SPIE Photonics West (BiOS) , San Francisco, USA, 2015. Details

Boroomand, A. , M. J. Shafiee , A. Wong , and K. Bizheva, " Lateral resolution enhancement via imbricated spectral domain optical coherence tomography in a maximum-a-posterior reconstruction framework ", SPIE Photonics West (BiOS) , San Francisco, USA, 2015. Details

Cho, D. , S. Haider , R. Amelard , A. Wong , and D. A. Clausi , " Physiological Characterization of Skin Lesion using Non-linear Random Forest Regression Model ", 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society , Sheraton Chicago Hotel and Towers Chicago, IL, USA, pp. 3349-3352, September, 2014. Details

Khalvati, F., A. Wong , G. Bjarnason, and M. Haider, " A Multi-Parametric Diffusion Magnetic Resonance Imaging Texture Feature Model for Prostate Cancer Analysis ", Annual International Conference of the IEEE Engineering in Medicine and Biology Society , 2014. Details

Cameron, A. , A. Modhafar, F. Khalvati, D. Lui , M. J. Shafiee , A. Wong , and M. Haider, " Multiparametric MRI Prostate Cancer Analysis via a Hybrid Morphological-Textural Model ", Annual International Conference of the IEEE Engineering in Medicine and Biology Society , 2014. Details

Kasiri, K. , P. Fieguth , and D. A. Clausi , " Cross modality label fusion in multi-atlas segmentation ", IEEE International Conference on Image Processing , 2014. Details

Khalvati, F., A. Modhafar, A. Cameron , A. Wong , and M. Haider, " A Multi-Parametric Diffusion Magnetic Resonance Imaging Texture Feature Model for Prostate Cancer Analysis ", MICCAI 2014 Workshop on Computational Diffusion MRI , 2014. Details

Carter, K., S. Marschall, A. Gawish , P. Fieguth , L. Sorbara, and K. Bizheva, " Accuracy evaluation of scleral lens thickness and radius of curvature using high-resolution SD- and SS-OCT ", SPIE Photonics West , 2014. Details

Kazemzadeh, F. , S. Haider , A. Wong , C. Scharfenberger , and D. A. Clausi , " Concurrent Multiview Discrete Spectral Imaging Device from the VIS to the NIR ", SPIE: Optics + Photonics , Novel Optical Systems Design and Optimization XVII , vol. 9193, San Diego, USA, SPIE Proceedings, 2014. Details

Liu, C., A. A. Moyed, A. Wong, P. Fieguth , V. Chan, H. Bie, and K. Bizheva, " Automatic algorithm for measuring visually evoked pupil size changes from OCT images ", Ophthalmic Technologies XXIII, SPIE , 2013. Details

Schwartz, S. , A. Wong , and D. A. Clausi , " Multi-scale saliency-guided compressive sensing approach to efficient robotic laser range measurements ", 2012 Ninth Conference on Computer and Robot Vision (CRV) , pp. 1-8, May, 2012. Details Glaister, J. , A. Wong , and D. A. Clausi , " Illumination Correction in Dermatological Photographs using Multi-stage Illumination Modeling for Skin Lesion Analysis ", 34th Annual International Conference of the IEEE Engineering in Medicine and Biology , pp. 102-105, 2012. Details Cameron, A. , J. Glaister , A. Wong , and M. Haider, " Non-parametric Bayesian Estimation of Apparent Diffusion Coefficient from Diffusion-Weighted Magnetic Resonance Imaging Data ", 34th Annual International Conference of the IEEE Engineering in Medicine and Biology , pp. 412-415, 2012. Details Glaister, J. , A. Cameron , A. Wong , and M. Haider, " Quantitative Investigative Analysis of Tumour Separability in the Prostate Gland using Ultra-high b-value Computed Diffusion Imaging ", 34th Annual International Conference of the IEEE Engineering in Medicine and Biology , pp. 420-423, 2012. Details Schwartz, S. , A. Wong , and D. A. Clausi , " Saliency-guided compressive fluorescence microscopy ", 34th Annual International Conference of the IEEE Engineering in Medicine and Biology, , San Diego, USA., pp. 4365 - 4368 , 2012. Details

Karimi, A., A. Wong , and K. Bizheva, " Automated detection and counting of keratocytes in human corneal stroma from ultrahigh-resolution optical coherence tomograms ", SPIE Photonics West (BiOS) , 2012. Details

Wong, A. , " Constrained Bayesian streak artifact reduction approach for contrast enhanced computed tomography imaging of the intervertebral disc ", Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE EMBC) , 2011. Details

Eichel, J. A. , D. Lee, A. Wong , P. Fieguth , D. A. Clausi , and K. Bizheva, " Quantitative comparison of despeckling and frame averaging approaches to processing retinal OCT tomograms ", SPIE Photonics West (BiOS) , 2011. Details

Kachouie, N. Nezamoddin , P. Tan, D. Gamble, K. McNagnyl, L. Kelly, M. Hughes, J. Bains, P. Fieguth , J. Wong, and E. Jervis, " Imaging informatics and cell shape analysis: examining the role of NHERF-1 and podocalyxin in uropod formation ", Stem Cell Network Annual General Meeting , Toronto, 2007. Details

Booth, S., and D. A. Clausi , " Segmentation and three-dimensional reconstruction using MRI vertebral slices ", Canadian Conference on Electrical and Computer Engineering , Toronto, ON, Canada, pp. 1303 - 1307, January, 2001. Details

Booth, S., and D. A. Clausi , " Automated segmentation and 3-d reconstruction from spinal MRI ", World Congress on Medical Physics and Biomedical Engineering , Chicago, IL, July 28, 2000. Details

Book chapters

Kazemzadeh, F. , and A. Wong , A System, Method and Apparatus for Ultra-resolved Ultra-wide Field-of-view Multispectral and Hyperspectral Holographic Microscopy , , vol. 62155416, USA, April 30, 2015. Details

Kazemzadeh, F. , A. Wong , and S. Haider , Imaging System and Method for Concurrent Multiview Multispectral Polarimetric Light-field High Dynamic Range Imaging , , USA, 2014. Details

Hajian, A. R., F. Kazemzadeh , B. B. Behr, T. M. Haylock, and L. M. Chifman, A Device and Methods for Generating a Fringe Pattern Based on the Depth of a Surface of a Substance , , USA, 2011. Details

Eichel, J. A. , Statistical Model-Based Corneal Reconstruction , , Waterloo, ON, Canada, University of Waterloo, 2013. Details

Clausi, D. A. , " Finite Element Simulation of Early Embryonic Development ", Department of Systems Design Engineering, Waterloo, Ontario, Canada, University of Waterloo, 1992. Details

Contact Waterloo
Maps & Directions
Accessibility

The University of Waterloo acknowledges that much of our work takes place on the traditional territory of the Neutral, Anishinaabeg, and Haudenosaunee peoples. Our main campus is situated on the Haldimand Tract, the land granted to the Six Nations that includes six miles on each side of the Grand River. Our active work toward reconciliation takes place across our campuses through research, learning, teaching, and community building, and is co-ordinated within the Office of Indigenous Relations .

Browse Course Material

Course info, instructors.

Dr. Gari Clifford
Dr. John Fisher
Dr. Julie Greenberg
Dr. William Wells

Departments

Health Sciences and Technology
Electrical Engineering and Computer Science
Aeronautics and Astronautics

As Taught In

Signal Processing
Biomedical Instrumentation
Biomedical Signal and Image Processing
Biomedicine
Medical Imaging

Learning Resource Types

Course description.

You are leaving MIT OpenCourseWare

U.S. Department of Health & Human Services
National Institutes of Health

En Español | Site Map | Staff Directory | Contact Us

Labs @ NIBIB

Advanced Image Processing

Image analysis is a powerful tool in cell biology to collect quantitative measurements in time and space. Because microscopy imaging can easily produce terabytes of research data, accurate and automated analysis methods are key to successfully quantifying relevant information in such large image collections.

High-Performance Image Computation

Cell nuclei segmentation.

Cell nuclei segmentation is typically the first critical step for microscopy image analysis. With accurate cell nuclei segmentation, multiple biological analyses can be subsequently performed, including cell-type classification, cell counting, and cell tracking, which provides valuable information for researchers.

We developed a Mask Regional Convolutional Neural Networks (RCNN) -based method for nuclei segmentation. Mask RCNN [1] is a state-of-the-art object segmentation framework that can identify not only the location of any object, but also its segmented mask .

schematic of key components of computational technique

Example 1: Nuclei segmentation of an adult worm

{"preview_thumbnail":"/sites/default/files/styles/video_embed_wysiwyg_preview/public/video_thumbnails/VchE7sP1zqo.jpg?itok=dgGA1qt8","video_url":" https://youtu.be/VchE7sP1zqo ","settings":{"responsive":1,"width":"854","height":"480","autoplay":0},"settings_summary":["Embedded Video (Responsive)."]}

Triple-view line confocal imaging of an adult worm. Sample size is ~870 mm x 53 mm x 48 mm. Manual segmentation of all nuclei (n=2136) took several days/weeks.
With our Mask RCNN-based nuclei segmentation model, segmentation of all nuclei took < 1 hour on single NAVDIA Quadro P6000 GPU.
Compared with manually segmented nuclei, the accuracy of the Mask RCNN-based segmentation model is 94.42%.

Example 2: Nuclei segmentation of C. elegans embryos

Mask RCNN-based nuclei segmentation can be utilized for cell counting throughout the entire period of embryogenesis. Here, we integrated nuclei segmentation into a cell-tracking system to map the growth and migration of every cell in a live, developing worm embryo from fertilization to maturity.

Example 3: Evaluation of image quality of imaging systems

To quantify three imaging systems, we used Mask RCNN to segment and count the number of nuclei from 15 worm embryos. These three imaging systems included single-view light-sheet imaging (raw), single view light-sheet imaging followed by a one-step deep learning (DL) prediction (one-step DL), and single view light-sheet imaging followed by a two-step deep learning prediction (two-step DL). For the C. elegans embryonic system, the exact number of nuclei is known, as their positions and divisions were previously manually observed and scored by John Sulston with differential interference contrast (DIC) microscopy. Against the Sulston ground truth, the raw single confocal view found fewer than half of all nuclei. The two-step DL prediction fared much better, capturing the majority of the nuclei and outperforming the one-step DL prediction.

Image Stitching

We developed an image-stitching package that allows simple and efficient alignment of multi-tile, multi-view, and multi-channel image datasets, which are acquired by light sheet microscopes. This package supports images from megabyte-sized images up to terabyte-sized images, which are produced when acquiring cleared tissue samples with light sheet microscopy.

Rapid Image Deconvolution and Multiview Fusion

The contrast and resolution of images obtained with optical microscopes can be improved by deconvolution and computational fusion of multiple views of the same sample [2]. Because these methods are computationally expensive for large datasets, we have designed several software pipelines for different applications, including for rapid image deconvolution and/or multiview fusion.

Pipeline 1: Joint-view Deconvolution on Cleared-tissue Datasets

schematic of joint view deconvolution of image

Table 1: Computation time on a pair of 3800 x 3400 x 1200 data (28G)

Processing Type	Single Workstation Time (hr)	Biowulf Cluster Time (min)
Stitching Tiles	0.5	15
Deskew + Interpolation + Rotation	4	20
Subvolume Registration + Deconvolution	7	30
Stitching Subvolumes	5	15
Combined Processing Time	~17	90

Pipeline 2: Single-view Deconvolution on Cleared-tissue Dataset

schematic of single view deconvolution of image

Table 2: Computation time on a pair of 3800 x 3400 x 1200 data (28G)

Processing Type	Single Workstation Time (hr)	Biowulf Cluster Time (min)
Stitching Tiles	0.5	15
Deskew + Interpolation + Rotation	2	20
Subvolume Registration + Deconvolution	6	20
Stitching Subvolumes	5	15
Combined Processing Time	~16	65

Machine Learning for Image Denoising, Resolution Enhancement, and Segmentation

Pipeline 3: joint-view deconvolution on small time serial data.

We also developed a registration and joint-view deconvoltuion package for small data (no stitching/splitting required) with multiple time points.

Table 3: Computation time of 1020 x 2048 x 100 (400M), 300 time points, 2 colors

Processing Type	Single Workstation Time	Biowulf Cluster Time (min)
Subvolume Registration + Deconvolution (at each time point)	4 min	4
Combined Processing Time	300 x 4 x 2 = 40 hr	120*

*With multiple available GPUs, all deconcovultion jobs could be finished within 2 hours.

Image Denoising and Resolution Enhancement

For super-resolution microscopy applications, we use 3D residual channel attention networks (RCAN) [3]. We first extended the original RCAN to handle 3D images; this method matches or exceeds the performance of previous networks in denoising fluorescence microscopy data. We can apply this capability for super-resolution imaging over thousands of image volumes (tens of thousands of images). This method allows for RCAN and other networks to extend resolution, providing better resolution enhancement than alternatives, especially along the axial dimension. Finally, when we use stimulated emission depletion microscopy (STED) and expansion-microscopy ground truth to train RCAN models using multiple fixed- and live-cell samples, we demonstrate four-to five-fold improvement in volumetric resolution.

{"preview_thumbnail":"/sites/default/files/styles/video_embed_wysiwyg_preview/public/video_thumbnails/058lZs62J-8.jpg?itok=vpKd2D2z","video_url":" https://youtu.be/058lZs62J-8 ","settings":{"responsive":1,"width":"854","height":"480","autoplay":0},"settings_summary":["Embedded Video (Responsive)."]}

3D-RCAN denoising dual-color imaging of mitochondria (magenta) and lysosomes (cyan) in live U2OS cells. Left: Raw dual-color image (noisy). Right: 3D-RCAN output. The deep learning denoised image allows for the quantification and tracking of mitochondrial and mitochondria–lysosome interactions, which is not possible in the raw images.

{"preview_thumbnail":"/sites/default/files/styles/video_embed_wysiwyg_preview/public/video_thumbnails/CqyvdH5qOAk.jpg?itok=cINKNEPR","video_url":" https://youtu.be/CqyvdH5qOAk ","settings":{"responsive":1,"width":"854","height":"480","autoplay":0},"settings_summary":["Embedded Video (Responsive)."]}

3D-RCAN enables transformation of confocal images to STED images. A model was trained with pairs of confocal and STED images. In the video, the raw resonant confocal data (left) has poorly defined nuclei and chromosomes; these structures were clearly resolved in the RCAN predictions (right).

{"preview_thumbnail":"/sites/default/files/styles/video_embed_wysiwyg_preview/public/video_thumbnails/SFa45nAoMjU.jpg?itok=HjC9O7V7","video_url":" https://youtu.be/SFa45nAoMjU ","settings":{"responsive":1,"width":"854","height":"480","autoplay":0},"settings_summary":["Embedded Video (Responsive)."]}

3D-RCAN enables transformation of Instant Structured Illumination Microscope (iSIM) to expansion images. Dynamics and organization of the actin and microtubule cytoskeleton in Jurkat T cells are much better resolved with RCAN-predicted expansion images.

Wu, Y., Han, X., Su, Y. et al. Multiview confocal super-resolution microscopy. Nature 600, 279–284 (2021). https://doi.org/10.1038/s41586-021-04110-0
Guo, M., Li, Y., Su, Y. et al. Rapid image deconvolution and multiview fusion for optical microscopy. Nat Biotechnol 38, 1337–1346 (2020). https://doi.org/10.1038/s41587-020-0560-x
Chen, J., Sasaki, H., Lai, H. et al. Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes. Nat Methods 18, 678–687 (2021). https://doi.org/10.1038/s41592-021-01155-

Biomedical Image Processing Projects

Biomedical image processing projects deals with analyzing of captured internal human body images for clinical treatment and diagnosis. The information of physiological and physiology processes are collected through advanced sensors and processed by suitable computing technology . There are several methodologies to study the present state and disorder of specific human organs/tissue.

For this, it utilizes the following different types of biomedical images. And, they are:

Ultrasound (sound)
MRI (magnetism)
CT scans (x-rays)
OCT and Endoscopy (light)
SPECT and PET (nuclear medicine: radioactive pharmaceuticals)

The handling of medical imaging is performed by the computerized system through efficient techniques and algorithms . As well, these techniques are incorporated with many advantages such as scalability, reliability, adaptability, privacy and etc. In general, imaging processes consist of image acquisition, computing, communication, storage, and visualization .

For your information, we shared the summary of the fundamentals of biomedical image processing. All these data make you understand the important terminologies, recent imaging modalities, image processing procedure, formation of medical image, and other futuristic image processing methodologies . Here, we have itemized the expected future developments by our expert’s suggestion.

Let’s have a quick glance over the recent research developments in medical image processing . These areas gain the attention of current active scholars who are pursuing their research careers in the bio-medical image processing research field.

Future Trends in Medical Imaging

Real Implication of Retinal Diseases
Wireless Tiny Single-Chip Ultrasound Sensing
Rise of Artificial Intelligence (AI) in Radiology
Heart or Cardiovascular Imaging Technology
Future of Ultrasound Device and System Portability
Large Superconducting Magnet System Applications
4D Printing Bio-medical Models and Applications
Development of MRIs between Slow and Fast Fuzzy
3D Mathematical and Anatomical Models
Migration and Advancement of X-Rays Films into Digital Files
Skull Stripping Brain Tumor Segmentation

Based on our recent research on image processing over current journals, we found that most of the scholars are interested in using image datasets like spatial scales, which range from cellular/molecular imaging to organ/tissue imaging for their proposed topics. Other than this, scholars are familiar with the dataset from the followings,

Colon Cancer
Nuclear Medicine
Computerized Tomography
Magnetic Resonance
Confocal and Optical Microscopy
Diabetic Retinopathy
Range Image and Video Data sets
Computed Tomography (CT)
Nuclear Magnetic Resonance (NMR)

For your knowledge, we have enumerated some datasets that are popularly used in developing Biomedical Image Processing Projects . Further, we have grouped the following based on category, modalities, network type with their corresponding datasets for your ease.

Datasets for Medical Image Modalities

Modalities: MRI
Network Type: LU-Net
Data Set: ACDC Stacom 2017
Modalities : CT, CXR and MRI
Network Type : FCN, SCAN, dense-FCN and U-Net
Data set : JSRT, Montgomery, JSRT, 172 sparsely annotated CT scan data set, TCIA, NSCLC
Modalities: Microscopic
Network Type: SegNet
Data Set: ALL-IDB1 database
Modalities : MRI
Network Type : 3D U-Net, FCN, cGAN, V-Net, 3d FCN, SegAN-CAT and GAN
Data Set : MRBrainS13, 62 Healthy Brain Images, BRATS2015, BRATS2017, BRATS2018, BRATS2019, Infant Brain Images, ANDI and NITRC data set
Modalities : CT
Network Type : 3D U-Net and CFUN
Data Set : MICCAI 2017 whole heart and MM-WHS2017
Modalities : Funduscopy
Network Type : GAN, PixelBNN, FCN, Res-UNet and U-Net
Data Set : STARE, DRIVE, RIM-ONE, CHASEDB1 and Drishti-GS data set
Modalities: CT
Network Type : Kid-Net
Data Set – 236 Subjects
Network Type : SSNet
Data Set : 60 abdominal MRI Scans
Modalities: 3D echocardiography
Network Type: VoxelAtlasGAN
Data Set: 60 Subjects over 3D echocardiography
Network Type : Attention U-Net
Data Set : TCIA
Network Type : FCN
Data Set : LVSC, RVSC and SCD
Modalities : Dermoscopy
Network Type : FCN and GAN
Data Set : ISBI 2017 and DermoFit
Network Type: 3D U-Net
Data Set: LASC2018
Modalities : MRI and CT
Network Type : DI2IN-AN, FCN and DCNN
Data Set : 1000 CT volumes, LiTS, 3DIRCADb and others
Data Set: MICCAI Challenge Data Set
Modalities : Histopathology
Network Type : GAN
Data Set : IPMCH
Network Type : FCN, V-Net, USE-Net,
Data Set : 152 MRI images, 3 T2-weighted MRI data sets and PROMISE2012
Modalities: MRI and CT
Network Type: Spine-GAN and Btrfly Net
Data Set: 253 multi-center medical patients and 302 CT Scan

For instance: In order to identify the tumor patterns/features, several improved ML algorithms are used for spatial and temporal analysis. Also, it enables to find of the organ/tumor (vasculature, volume, and diameter), fluid/blood flow parameters, and microscopic changes based on the selected appropriate imaging approaches used in wireless body area network projects .

Biomedical Image Processing Concepts Explained

This article is specifically discussed research trends in Biomedical Image Processing Projects along with their technological advancements!!!

Biomedical Image Processing Toolboxes

Image Acquisition Toolbox
Mapping Toolbox
Image Processing Toolbox
Medical Image Processing Toolbox
Deep Learning Toolbox
Computer / Machine Vision Toolbox
Vision HDL Toolbox

Next, we can see the significant development tools and software for implementing any kind of Biomedical Image Processing projects . More than this, there exist many technologies. So, one should choose the optimal tool based on the requirements of the handpicked project.

Development Tools and Software’s for Biomedical Image Processing

In recent research, our experts have recognized image informatics and cloud computing as the top-demanding research fields by scholars. Also, we just want you to know the other important research areas from the followings,

Cloud based Topics in Medical Image Processing

Cloud assisted Medical Imaging Services for Social Medias
Virtual Setting in Mammogram Testing and Interpretation
MIIP: Web Solution for Medical Image Analysis and Understanding
Crowdsourcing based Virtual Colonoscopy Video Analysis (For instances: polyp identification and annotation)

In addition, we have also given you the latest image informatics application in the healthcare sector for illustration purposes. Beyond this, it spreads its fame in all many research fields.

Image Informatics Healthcare Applications

Applying Local Global Classifier for Screening Chest X-Rays
Digital Pathology Image Analysis using Advanced DNN classification
3D-CNN based Malignant Breast Cancer Identification in MR images
Frequency assisted Human Brain Network Similarity Detection
DAX Next Generation is near to 1 million processes over commodity hardware
Automated Thyroid Segmentation in 3D Ultrasound Images
CBIR based Automatic Multi-Label Annotation in Abdominal CT images
Blood Vessels Inpainting based on Optic Disc / Cup Segmentation Methodologies

On the whole, we are glad to give our end-to-end research guidance on Biomedical Image Processing Projects topic under the supervision of our technical experts. So, make use of this opportunity and hold your hand with us to create a masterpiece of research.

Services we offer.

Mathematical proof

Pseudo code

Conference Paper

Research Proposal

System Design

Literature Survey

Data Collection

Thesis Writing

Data Analysis

Rough Draft

Paper Collection

Code and Programs

Paper Writing

Course Work

Faculty & Research

Departments
Faculty Directory
Research Areas
Facilities, Cores, & Centers
Faculty Openings
Faculty Resources
Roberts Innovation Fund

Biomedical Imaging

Research in Biomedical Imaging brings together engineers, physicists, biologists and chemists engaged in the development of methodology for the examination of biological structure and function through imaging. The work encompasses efforts in magnetic resonance imaging, magnetic resonance spectroscopy, nuclear medicine, optical light microscopy and image processing and analysis.

In general, Biomedical Imaging research is focused on the applied mathematics, engineering, physics and chemistry of developing methods that are useful for deriving quantitative information from biomedical images that range in scale from molecular/cellular images to mouse imaging to large animal/human imaging. Examples of recent projects include: the development of MRI pulse sequences and distortion correction strategies for use in acquiring high quality functional MRI (fMRI) images of epilepsy patients; investigation of the changes in neuronal inhibition and excitation in the human brain when different neurological disorders are present using MR Spectroscopgy; and development of strategies to correct for brain shift or prostate motion during image-guided surgical or radiotherapy procedures. In fiscal year 2007, this and other work was funded by over $7M of peer-reviewed external grant funding. Most of this came from the NIH . Almost all faculty working in this area have joint appointments with the Division of Bioimaging Sciences , within the Department of Diagnostic Radiology at the Yale School of Medicine . In 2007, these faculty ranked 11th nationally out of over 70 such programs in NIH funding.

Biomedical Imaging faculty teach a significant number of courses to Yale undergraduate students and graduate students within the Department of Biomedical Engineering . These include courses in the Physics of Medical Imaging, Biophotonics, the Physical and Chemical Basis of Biosensing and Biomedical Image Processing and Analysis. Furthermore, the faculty currently supervise about 20 graduate students in Biomedical Engineering and Electrical Engineering and over 20 postdoctoral Fellows.

Finally, it is important to note that Biomedical Imaging faculty in SEAS direct or play a key role in two major Centers that house a large portion of the imaging equipment used for research by the faculty noted above: The Yale Magnetic Resonance Research Center (MRRC) was founded in 1986 as a result of the recognition that NMR applications, as pioneered by Yale scientists, have enormous potential in biomedical research. The MRRC is now an interdepartmental and interdisciplinary research laboratory that provides state-of-the-art MR equipment, infrastructure and expertise for the development and application of MRI and MRS methodology in biomedical research. Research is focused on the study of intact biological systems by developing methods for obtaining structural, functional, physiological and biochemical information by MRI, MRS and other techniques. Applications include fMRI for neurosurgery and neuroscience, brain, muscle and liver energy metabolism, diabetes, adult and juvenile epilepsy and psychiatric disorders. This Center is directed by Professors Rothman and Constable.

The PET Center , houses CTI HRRT and a CTI HR+ PET scanners along with a cyclotron, a radiochemistry laboratory and a physics/modeling laboratory. The research focus is on the development of new radiotracers for use in a variety of applications, including tracking drug delivery systems and the mathematics and physics of delivering accurate information about metabolism and function. This Center is directed by Professor Richard Carson.

Faculty involved with research:

Joerg Bewersdorf – Cell Biology – BME

Richard Carson – BME – Diagnostic Radiology

James Duncan – BME – EE – Diagnostic Radiology

Tarek Fahmy – BME – ChE & EnvE Fahmeed Hyder – BME – Diagnostic Radiology

Chi Liu – BME – Diagnostic Radiology

Evan Morris – BME – Diagnostic Radiology – Psychiatry Xenophon Papademetris – BME – Diagnostic Radiology

Larry Staib – BME – EE – Diagnostic Radiology

Hemant Tagare – BME – Diagnostic Radiology

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings
My Bibliography
Collections
Citation manager

Save citation to file

Email citation, add to collections.

Create a new collection
Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

Search in PubMed
Search in NLM Catalog
Add to Search

Biomedical image processing

PMID: 7023828

Biomedical image processing is a very broad field; it covers biomedical signal gathering, image forming, picture processing, and image display to medical diagnosis based on features extracted from images. This article reviews this topic in both its fundamentals and applications. In its fundamentals, some basic image processing techniques including outlining, deblurring, noise cleaning, filtering, search, classical analysis and texture analysis have been reviewed together with examples. The state-of-the-art image processing systems have been introduced and discussed in two categories: general purpose image processing systems and image analyzers. In order for these systems to be effective for biomedical applications, special biomedical image processing languages have to be developed. The combination of both hardware and software leads to clinical imaging devices. Two different types of clinical imaging devices have been discussed. There are radiological imagings which include radiography, thermography, ultrasound, nuclear medicine and CT. Among these, thermography is the most noninvasive but is limited in application due to the low energy of its source. X-ray CT is excellent for static anatomical images and is moving toward the measurement of dynamic function, whereas nuclear imaging is moving toward organ metabolism and ultrasound is toward tissue physical characteristics. Heart imaging is one of the most interesting and challenging research topics in biomedical image processing; current methods including the invasive-technique cineangiography, and noninvasive ultrasound, nuclear medicine, transmission, and emission CT methodologies have been reviewed. Two current federally funded research projects in heart imaging, the dynamic spatial reconstructor and the dynamic cardiac three-dimensional densitometer, should bring some fruitful results in the near future. Miscrosopic imaging technique is very different from the radiological imaging technique in the sense that interaction between the operator and the imaging device is very essential. The white blood cell analyzer has been developed to the point that it becomes a daily clinical imaging device. An interactive chromosome karyotyper is being clinical evaluated and its preliminary indication is very encouraging. Tremendous efforts have been devoted to automation of cancer cytology; it is hoped that some prototypes will be available for clinical trials very soon. Automation of histology is still in its infancy; much work still needs to be done in this area. The 1970s have been very fruitful in utilizing the imaging technique in biomedical application; the computerized tomographic scanner and the white blood cell analyzer being the most successful imaging devices...

PubMed Disclaimer

Publication types

Search in MeSH
Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Open access
Published: 31 August 2024

Deep learning for the harmonization of structural MRI scans: a survey

Soolmaz Abbasi 1 ,
Haoyu Lan 2 ,
Jeiran Choupan 2 ,
Nasim Sheikh-Bahaei 3 ,
Gaurav Pandey 4 &
Bino Varghese 3

BioMedical Engineering OnLine volume 23 , Article number: 90 ( 2024 ) Cite this article

2 Altmetric

Metrics details

Medical imaging datasets for research are frequently collected from multiple imaging centers using different scanners, protocols, and settings. These variations affect data consistency and compatibility across different sources. Image harmonization is a critical step to mitigate the effects of factors like inherent differences between various vendors, hardware upgrades, protocol changes, and scanner calibration drift, as well as to ensure consistent data for medical image processing techniques. Given the critical importance and widespread relevance of this issue, a vast array of image harmonization methodologies have emerged, with deep learning-based approaches driving substantial advancements in recent times. The goal of this review paper is to examine the latest deep learning techniques employed for image harmonization by analyzing cutting-edge architectural approaches in the field of medical image harmonization, evaluating both their strengths and limitations. This paper begins by providing a comprehensive fundamental overview of image harmonization strategies, covering three critical aspects: established imaging datasets, commonly used evaluation metrics, and characteristics of different scanners. Subsequently, this paper analyzes recent structural MRI (Magnetic Resonance Imaging) harmonization techniques based on network architecture, network learning algorithm, network supervision strategy, and network output. The underlying architectures include U-Net, Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), flow-based generative models, transformer-based approaches, as well as custom-designed network architectures. This paper investigates the effectiveness of Disentangled Representation Learning (DRL) as a pivotal learning algorithm in harmonization. Lastly, the review highlights the primary limitations in harmonization techniques, specifically the lack of comprehensive quantitative comparisons across different methods. The overall aim of this review is to serve as a guide for researchers and practitioners to select appropriate architectures based on their specific conditions and requirements. It also aims to foster discussions around ongoing challenges in the field and shed light on promising future research directions with the potential for significant advancements.

Introduction

Neuroimaging techniques like magnetic resonance imaging (MRI), positron emission tomography (PET) and computerized tomography (CT) play a vital role in studying the brain's structure and function [ 1 , 2 , 3 ]. These medical imaging modalities provide invaluable insight for diagnosing neurological disorders, understanding brain development, and investigating neurodegenerative processes. In comparison to other neuroimaging modalities, MRI is known for its ability to generate excellent soft tissue contrast, making it instrumental for studying subtle tissue details including the brain or spinal cord [ 4 ]. Additionally, it is vital for many differential diagnoses of neurological disorders, including tumors [ 5 ], inflammatory conditions [ 6 ], and degenerative disorders [ 7 ]. MRI utilizes various imaging options and pulse sequences according to clinical needs. These sequences generate images with different contrasts, such as ${T}_{1}$ -weighted, ${T}_{2}$ -weighted, and PD-weighted (Fig. 1 ).

Example images acquired from Guy’s Hospital using a Philips 1.5T system, sourced from the IXI dataset [ 8 ]

Variations in scanner hardware, imaging parameters, and acquisition protocols can lead to systematic differences in the appearance and quantitative measures derived from neuroimages. These divergences, if unaddressed, can reduce the statistical power of neuroimaging studies, limit the generalizability of findings across sites, and impede efforts to pool and analyze multi-site datasets. Consequently, there is an increasing demand for harmonizing neuroimaging data to mitigate unwanted inter-site and inter-scanner effects.

In recent years, data-driven harmonization techniques leveraging machine learning have gained significant traction. Deep learning models have demonstrated remarkable ability to capture complex data representations and transformations, making them well-suited for tackling neuroimaging harmonization challenges. By learning from data acquired across multiple sites and scanners, these models can disentangle biologically relevant signals from technical artifacts, enabling the generation of harmonized neuroimages or derived measures. However, compared to other neuroimaging harmonization efforts, one of the factors that make MRI harmonization more complex is that grayscale-based signal intensity in MRI lacks a standardized measure, unlike semi-quantitative measures such as standardized uptake value (SUV) of PET or quantitative measures such as Hounsfield units (HU) of CT [ 3 ]. This accentuates the variability in MRI, resulting in differences in contrast-to-noise ratio, temporal resolution, and spatial resolution. These variations have been found to affect the reliability of radiomic analysis [ 9 ].

With larger and more varied datasets available in open-source databases, researchers can conduct more in-depth analyses and leverage subtle details within the data. By involving more participants and tracking them over an extended period, researchers gain deeper insights into how the study impacts the subjects, such as the effects of a disease or the aging process [ 10 ]. This longitudinal approach also enables a better understanding of the underlying causes of certain diseases, ultimately helping identify optimal treatments, diagnostic methods, and care plans. While larger datasets lead to more accurate results, they also introduce increased variability in how the data are acquired. This acquisition variability issue is almost always present for various reasons. Even at a single site using one scanner, patients may require repeat scans on different MRI scanners if they need additional medical care. Furthermore, the imaging environment itself is prone to changes due to scanner upgrades or replacements occurring over the course of a study [ 11 ]. When dealing with images from multiple sites, the inconsistency becomes more pronounced and can lead to domain shift problems.

In the realm of medical imaging, addressing domain shift is crucial for ensuring the reliability and consistency of AI models across different imaging sites and scanners. Domain adaptation (DA) involves fine-tuning a model to perform well on data from a specific target domain, thereby enhancing its accuracy and applicability within that domain. On the other hand, domain generalization (DG) aims to develop models that generalize effectively to unseen domains, without specific access to target domain data during training [ 12 ]. An emerging technique, image harmonization, focuses on reducing inter-site variation to facilitate meaningful comparisons and analyses of images across diverse imaging environments.

Domain adaptation enhances model accuracy by injecting domain-specific knowledge into a general AI model. This process typically requires access to target domain data during training, allowing the model to better capture domain-specific features and nuances. However, challenges include the need for additional training data [ 13 ], inter-modality heterogeneity [ 14 ], and rigorous model evaluation. Despite these challenges, domain adaptation significantly improves model performance within specific domains.

In contrast, domain generalization tackles the broader challenge of developing models that can generalize well across unseen domains. This approach is particularly beneficial in scenarios where acquiring labeled data from every possible domain is impractical. By learning from multiple related domains during training without direct exposure to the target domain, domain generalization reduces the need for extensive labeling efforts and enhances the model's adaptability to new tasks. However, the lack of direct access to target domain data makes domain generalization more challenging than domain adaptation in practical applications [ 15 ].

Image harmonization focuses on minimizing inter-site variation in medical imaging, enabling consistent analysis and comparison across different imaging sites and scanners [ 16 ]. This technique has shown promising results in standardizing imaging data, thereby improving the reliability of downstream analysis and clinical decision-making. Unlike domain adaptation and generalization, which primarily focus on model training strategies, harmonization directly addresses data variability at the preprocessing stage. This approach simplifies model deployment across diverse clinical settings but may require specialized algorithms tailored to specific imaging modalities.

In the context of MRI harmonization, a "traveling subject" refers to a person or phantom (an object designed to mimic certain properties of human tissues) that is scanned on multiple MRI scanners at different sites. The traveling subject provides a common reference point when being scanned on MRI scanners located at different sites or institutions, which may have different scanner models, field strengths, or acquisition protocols. The data from scanning the traveling subject across sites are used to characterize and correct for scanner-specific variations in the MRI data. The use of traveling subjects is a crucial step in many MRI harmonization pipelines, especially for large multi-site neuroimaging studies, as it provides a way to quantify and mitigate scanner-related effects that could otherwise confound the analysis and interpretation of the pooled dataset.

On a related note, in some cases paired MRI data are used for image harmonization. While the two concepts are related, they are fundamentally different in their application. The purpose of the traveling subject is to directly measure and characterize the scanner-specific variations that need to be harmonized. Paired MRI data, however, refer to having two sets of MRI scans acquired from the same subject, which can be done either within a single scanner (intra-site) or across different scanners (inter-site). Intra-site paired data typically include different imaging modalities (e.g., T1-weighted and T2-weighted images) acquired from the same subject on the same scanner. This intra-site pairing facilitates training by providing complementary information from different modalities. Inter-site paired data, which involve scanning the same subject on both a "source" and a "target" scanner, help in harmonizing data between different scanners by offering corresponding data points under different scanner conditions.

While a traveling subject provides a common reference scanned across all scanners, paired data have separate source and target scans. Also, while traveling subjects are fewer in number but scanned widely across different sites, paired data can involve the full set of study subjects scanned at two sites. Lastly, while traveling subjects directly measure scanner effects, paired data rely on the corresponding subject scans to estimate the scanner-specific transformations required for harmonization. Therefore, while a traveling subject provides a direct measurement of scanner effects, paired data provide a way to estimate and apply those effects to each subject's data during the harmonization process. To provide an example of image harmonization, Fig. 2 demonstrates the traveling subject from six different scanners of the SRPBS Multi-disorder MRI Dataset [ 17 ]. As observed, the image contrast varies across scanners, which is known to affect downstream tasks such as tissue segmentation and disease classification.

Traveling subject from different scanners of the SRPBS Multi-disorder MRI Dataset [ 12 ] with detailed acquisition parameters. The contrast has been changed across scanners

In 2022, a comprehensive overview was presented on the various approaches for radiomics harmonization [ 18 ]. This review study classified these methods into two categories: image-based harmonization and feature-based harmonization. Image-based harmonization techniques are applied directly to the images before extracting radiomics features, while feature-based harmonization aims to reduce the differences between the extracted features themselves. The choice of these techniques can be constrained by the number of samples available for analysis. The review concluded that, up to that point, none of these harmonization methods had been definitively established as the most effective approach within the analysis process.

Statistical methods have been used for feature-based harmonization [ 19 ]. These methods offer fine-grained adjustments, enabling researchers to target specific features for precise correction while maintaining interpretability [ 20 ]. The primary advantages of statistical methods include the following:

Interpretability: Researchers can understand which specific features are being adjusted.

Targeted Adjustments: Precise correction of predefined features is possible, making these methods ideal for datasets with well-understood relevant features.

However, these methods also have notable limitations:

Dependence on Predefined Features: They rely on predefined features for correction, potentially limiting their effectiveness in complex datasets where relevant features are poorly defined [ 21 ].

Potential for Overfitting: In scenarios with high variability and noise, statistical methods might overfit to the training data.

Image-based approaches particularly leverage deep learning models for accurate mapping between source and target MRI images [ 21 ]. Deep learning has revolutionized the field of medical imaging by enabling automated analysis, diagnosis, and prognosis through the extraction of meaningful features from medical images. Image classification, segmentation [ 22 ], image reconstruction [ 23 ], and image registration [ 24 ] are just a few successful applications of deep learning in medicine. Along these lines, recent development of various neural network architectures that have been proposed specifically for the task of medical image harmonization serve as the primary focus of this review article. The main advantages of image-based approaches are as follows:

Automation and Scalability: They can automatically learn complex mappings without requiring predefined features.

Handling Complex Variability: They excel in scenarios with complex and high-dimensional data, effectively capturing intricate patterns and relationships.

However, these methods also face challenges:

Black-Box Nature: Deep learning models are often criticized for their lack of interpretability, making it difficult to understand how corrections are being made.

Data Requirements: They typically require large amounts of training data and computational resources, which may not be available in all settings.

These methods can be selected according to the specific characteristics of their datasets and research goals. Statistical methods are preferable when interpretability and targeted feature correction are paramount, especially in well-understood datasets. In contrast, image-based approaches are better suited for complex datasets with poorly defined features and where automated, scalable solutions are needed.

Combining statistical and image-based approaches could potentially yield better performance by leveraging the strengths of both methods. For instance, a hybrid model could use statistical methods to correct well-defined features while employing deep learning to handle more complex, undefined variability. This combination could enhance both the accuracy and interpretability of harmonization efforts. Future research could explore integrated frameworks that synergistically use both approaches, potentially leading to more robust and generalizable harmonization techniques.

Wen et al. provided an examination of image harmonization methods for brain MRI data, with a particular focus on machine learning (ML)-based approaches used in both explicit and implicit ways [ 25 ]. They reported that a uniform imaging dataset can be achieved by implementing explicit methods that harmonize intensity values and image-derived metrics, or by using implicit methods to improve the performance of a downstream task. They also noted that traveling subject datasets are crucial for the effective implementation of explicit harmonization, as these enable the machine learning models to avoid learning biased information from the population. However, contemporary traveling subject datasets have limitations in terms of size and issues related to scan–rescan reliability, which can hinder the performance of the ML models. In contrast, implicit methods do not require a traveling subject dataset. Researchers only need to determine the source and reference domains to develop the machine learning algorithm for harmonizing the MRI scans.

Different harmonization solutions, including the image domain and feature domain, have been discussed in another survey [ 26 ]. The image domain harmonization encompasses acquisition protocols and data augmentation, while the feature domain category includes statistical normalization, Combat [ 27 ], and deep learning. GAN, Neural Style Transfer (NST), and their combination were discussed as deep learning-based harmonization techniques.

Hu et al. investigated both statistical and deep learning methods for harmonization [ 20 ]. The study noted that statistical techniques could provide robustness and effectiveness, especially in scenarios with smaller sample sizes or when dealing with confounding factors. Conversely, deep learning models may be better suited to handle the complex nature of image-level data, which poses significant challenges for conventional statistical approaches. A recent book chapter by Zuo et al. [ 28 ] reviewed disentangled representation learning methods for MR image harmonization, demonstrating how disentangled representations can be learned through both supervised and unsupervised image-to-image translation techniques.

Literature search

To narrow down our primary focus, we conducted a comprehensive literature search that met specific inclusion criteria including applying harmonization technique on structural MRI, employing deep learning methods, and in some papers harmonizing images for downstream tasks. The search was organized across the PubMed database using the terms "MRI harmonization" AND "deep learning," "MRI harmonization" AND "structural MRI," and "MRI harmonization" AND "deep learning" AND "structural MRI" through February 2024. This search returned 285 papers and the duplicate papers were excluded in the first phase. The same keywords "disentanglement representation learning for MRI" and "MRI harmonization using transformers" were used for searches on Google Scholar. Studies that were excluded encompass those that (1) were applied on the PET, fMRI, dMRI, or anything other than structural MRI, (2) did not employ a deep learning architecture, and (3) did not belong to the aim and scope of this review according to titles and abstracts. After applying criteria to the literature search and conducting screening, we incorporated a total of 38 papers into our study.

All the papers included in this study were published between 2019 and 2024, with approximately 26% from 2022 and 34% from 2023. The identified studies were analyzed in terms of their network architecture, learning algorithm, network framework, and network output. Based on the deep network architecture, the approaches can be classified into U-Net, GANs, VAEs, flow-based generative models, transformers, and custom Networks. Some networks learn via disentangled representation learning, which involves extracting and separating meaningful features or factors from the MRI data associated with different imaging characteristics. Among the studies, 29% utilized custom networks with or without disentanglement representation, 24% employed GANs, 13% used U-Net, and 13% were based on VAEs. Additionally, two of them relied on transformers, and three utilized flow-based generative models, while the rest employed a combination of two networks.

The majority of articles, about 74%, relied on publicly available datasets; however, 26% used local datasets. In addition to that about 53% of the articles proposed a 3D model or worked with 3D images. It is noteworthy that 66% of the papers employed harmonization solely on T1-weighted images, while the rest also utilized other contrasts. Furthermore, 63% of the studies conducted tests on the harmonized images for downstream tasks, whereas the remainder focused merely on image harmonization. Additionally, 15 of the articles have publicly available source code.

Motivations

Based on the survey of prior efforts, the motivation behind this review is outlined as follows:

A thorough, wide-ranging evaluation of deep learning-based methods for harmonizing structural MRI data across various benchmarks is lacking. Such a comprehensive investigation could shed light on the advantages and limitations of existing approaches in this domain.

While previous review articles have examined various harmonization techniques, they have not covered harmonization strategies that leverage transformers, flow-based generative models, or custom-designed neural network architectures.

A comprehensive comparison of large-scale brain imaging datasets for training and evaluating harmonization methods has not been conducted.

There has been a lack of detailed discussion surrounding the evaluation metrics used for assessing MRI harmonization methods, including considerations for scenarios with or without the presence of traveling subject data.

Driven by the aforementioned motivations, the primary goals of this study are to address existing gaps, elucidate the current limitations for a comprehensive comparison of harmonization techniques, and analyze the pros and cons of deep learning architectures for MRI harmonization. As such, our focus is solely on deep learning-based harmonization techniques. The design of networks for MRI harmonization can draw inspiration from networks used for domain adaptation [ 29 ] and image-to-image translation [ 30 ] in various computer vision applications. However, due to the complex structure of the brain and the intensity levels of MRI images, MRI harmonization networks have fundamental differences from networks in other applications. MRI harmonization aims to preserve the anatomical (content) information of the source image while transforming the contrast (style) to the target domain.

Contributions

SA performed literature search and wrote first draft of the manuscript. BV reviewed the first draft and conceived the project. SA, HA, JC, NSB, GP, and BV reviewed all subsequent drafts and approved the final manuscript.

This study provides a categorization of state-of-the-art deep learning-based MRI harmonization techniques based on their architectural design (e.g., GAN, U-Net, VAE), learning algorithms, network frameworks, and network outputs. This systematic classification offers valuable insights into the appropriate design considerations for developing effective harmonization networks.

This study conducts a comprehensive comparison of well-established large-scale datasets based on their fundamental characteristics, such as the number of participants, age range, target challenges, scanner types, and image modalities. Additionally, it discusses the lack of a dedicated harmonization dataset that addresses specific challenges faced by the medical imaging community.

This study examines the commonly used evaluation metrics in explicit image harmonization techniques, such as the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). It highlights the limitations of these evaluation criteria, pointing out their potential shortcomings in enabling accurate comparisons across different harmonization methods.

The rest of the paper is structured as follows. Sect. " The Overview of Harmonization " introduces harmonization. Sect. " Deep Learning-based Harmonization Taxonomy " categorizes different harmonization techniques. While acknowledging that certain papers may fit into more than one category, they are classified based on their predominant focus. Sect. " Applicability and Limitations of Harmonization " discusses the applicability and limitations of harmonization in MRI neuroimaging in a general context. Sect. " Discussion " analyzes the strengths and weaknesses of the various image harmonization methods. Sect. " Conclusion and Future Direction " provides concluding remarks.

The overview of harmonization

The MRI harmonization is formulated as a transformation that maps images from a source domain ${X}_{s}$ to a target domain ${X}_{t}$ . Let ${x}_{s}$ denote an MRI image within the source domain, and ${x}_{t}$ signify its counterpart within the target domain. The objective of MRI harmonization is to learn a mapping function $f$ :

The function $f$ should preserve the anatomical content of the source image while aligning its contrast to match that of the target domain. This can be expressed mathematically as follows:

where $\epsilon$ represents the residual error introduced during the harmonization process. To learn the mapping function $f$ , a deep learning model using adversarial and auxiliary loss functions can be employed to minimize the discrepancy between the distribution of synthesized images ${x}_{t}$ and real images from the target domain ${X}_{t}$ .

There are several key factors to consider when evaluating image harmonization techniques. This section will discuss these factors, including the types of datasets used for training, the scanners employed for image acquisition (if relevant), and the metrics used to assess the success of harmonization approaches. Each of these aspects can influence the effectiveness and suitability of a particular method and can present unique challenges (Fig. 3 ).

Overview of Harmonization

Pooling MRI data from diverse sources is crucial for high-powered and large-scale brain imaging studies. These sources include different sites, scanners, and acquisition protocols. Fortunately, several large neuroimaging datasets already exist, such as the UK Biobank [ 31 ], ABIDE [ 32 ], and ADNI [ 33 ]. Apart from heterogeneities in data due to scanners and modalities, the following complexities further complicate the harmonization process:

Disease complexity: While some harmonization methods use healthy brain scans, brain diseases pose additional challenges. Subtle changes in diseased brains are crucial for studies like Alzheimer's disease progression, and harmonization techniques must not obscure this information.

Segmentation and classification complexity: When diverse images from different scanners are harmonized, it must allow for precise and accurate segmentation (identification of specific brain structures) and classification (grouping images based on disease state).

Biological covariate balance complexity: Ensuring a balanced distribution of biological covariates such as gender and age in training datasets remains important. This is critical if the sample size of the training data is limited. A balanced representation of biological covariates and accounting for their difference helps achieve more generalizable and realistic results.

While Table 1 provides an overview of common datasets used in harmonization research, these datasets present challenges due to their inherent diversity. These large-scale datasets are often collected from multiple sites across different countries, and harmonization studies frequently utilize only a subset of the data or even employ entirely different datasets. This variation in data origin and usage hinders standardization and makes direct statistical comparisons between studies difficult.

The global estimation for the number of MRI machines was around 36,000 with 2500 machines being manufactured annually [ 39 ]. While MRI is a vital tool in medical diagnosis, their inherent limitations can affect image quality and accuracy. These limitations arise from two main sources:

Natural effects

The MRI process itself and the equipment involved are susceptible to natural phenomena that can cause issues. Examples include magnetic field inhomogeneity, gradient non-linearities, and variations in radiofrequency (RF) coil sensitivity. These effects manifest as variations in image intensity, distortions, and artifacts, ultimately impacting image quality and potentially leading to misdiagnosis. Fortunately, manufacturers are constantly working on mitigating these natural effects to improve the reliability and quality of MRI systems.

Acquisition settings

The specific settings used during an MRI scan significantly impact the resulting images. Factors like pulse sequence type, repetition time (TR), echo time (TE), and flip angle contribute to the appearance and accuracy of the scan. Table 2 provides a detailed breakdown of these factors. Selecting and optimizing these settings is crucial to minimize inherent limitations like noise, artifacts, and inconsistencies in image quality. By tailoring these settings to specific diagnostic needs, healthcare professionals can ensure high-quality imaging outcomes.

Accurately assessing the effectiveness of harmonization algorithms is critical. This subsection delves into several essential metrics used for this purpose:

Peak signal-to-noise ratio (PSNR)

This metric measures the ratio between the maximum possible signal (image intensity) and the corrupting noise that affects image quality. Higher PSNR values generally indicate better harmonization and is essential for accurate diagnosis and assessment in clinical settings [ 46 ]. Improved PSNR aims to preserve important anatomical details, aiding radiologists in making precise evaluations. Thus, the PSNR ratio is a highly effective quality indicator in evaluating the effectiveness of harmonization architecture.

This ratio is derived from the difference between the original image and the harmonized version. The PSNR is calculated using the following equation:

where R demonstrates the maximum fluctuation in the input image data type.

Mean absolute error (MAE)

MAE calculates the average of the absolute differences in pixel intensity values between the original and harmonized images. Lower MAE values indicate that the harmonized image closely resembles the original, preserving essential diagnostic information. This accuracy in intensity values is crucial for tasks in image harmonization, where the goal is to closely approximate the original data. This fidelity is important for various applications, including diagnostic assessments and other clinical evaluations where accurate representation is paramount.

where M and N represent the row and column of the input image, respectively.

PSNR and MAE both quantify the difference between the original and harmonized images, but they do so in different ways. PSNR is a logarithmic measure that emphasizes larger differences, making it useful for detecting significant deviations in image quality. MAE, on the other hand, provides a linear measure of average differences, offering a straightforward assessment of overall image fidelity. Together, these metrics provide a comprehensive view of image quality by highlighting both large and small discrepancies.

Structural similarity index measure (SSIM)

SSIM goes beyond just measuring noise levels. It compares the overall structural similarity between two images, considering luminance, contrast, and structure. Clinically, higher SSIM values suggest that the harmonized image retains the structural integrity of the original, which is critical for identifying subtle anatomical changes [ 47 ]. This metric helps ensure that the harmonization process does not distort important clinical features, thereby supporting accurate diagnosis and treatment planning.

The SSIM is based on illumination, contrast, and structural terms (Eqs. 6 – 8 ). In these equations, $\mu$ and $\sigma$ are local mean and standard deviation, respectively.

SSIM complements PSNR and MAE by focusing on structural information rather than just pixel-wise differences. While PSNR and MAE quantify numerical discrepancies, SSIM evaluates how well the harmonized image preserves the structural integrity of the original, considering luminance, contrast, and structural similarity. This combination ensures that both numerical accuracy and structural fidelity are assessed.

In addition to these core metrics, evaluation techniques specific to generative models might also be employed for certain harmonization approaches.

Recent research [ 48 ] has highlighted a crucial disconnect between how well harmonization algorithms perform according to traditional metrics and their actual impact on downstream applications. This suggests that common image similarity metrics like PSNR and SSIM might not fully capture the effectiveness of harmonization in improving compatibility across different datasets (cross-domain consistency). As a result, there is a growing need to re-evaluate current metrics to ensure they accurately assess the success of harmonization techniques.

To address this limitation, new assessment methods have been proposed [ 49 ]. These methods focus on two key aspects of harmonization:

Intensity Harmonization: The Wasserstein Distance (WD) is used to measure how well intensity levels are harmonized. It achieves this by calculating the movement of histograms between images, essentially quantifying how similar the intensity distributions become. Clinically, ensuring consistent intensity levels across images from different scanners or protocols can improve the reliability of quantitative measurements, such as volumetric analysis.

Anatomy Preservation: To evaluate how well anatomical structures are preserved during harmonization, segmentation is performed on both the original and harmonized images. The relative Absolute Volume Difference (rAVD) is then calculated to compare the segmentation results. This provides a measure of how closely the harmonized image retains the anatomical information from the original image. Accurate anatomical preservation ensures that clinical assessments, such as tracking disease progression or planning interventions, are based on reliable data.

Summarizing, WD measures the similarity of intensity distributions, ensuring that the overall intensity levels are harmonized across images. rAVD, on the other hand, evaluates how well anatomical structures are preserved during harmonization. By combining these metrics, we can assess both the consistency of intensity harmonization and the preservation of anatomical details, offering a dual perspective on harmonization effectiveness.

By incorporating these more specific metrics alongside traditional ones, researchers can gain a more comprehensive understanding of how different harmonization algorithms perform and their suitability for real-world applications.

In many real-world applications, "ground truth" data, which represents the perfect or ideal outcome, may not be available. This is especially common in situations involving unpaired datasets, where there is no direct correspondence between elements. Image generation tasks frequently encounter this scenario.

To address this challenge, researchers have developed metrics that assess image quality without relying on ground truth. These metrics offer valuable insights for objectively evaluating model performance:

Inception Score (IS): A popular metric for assessing the quality and diversity of images generated by models like Generative Adversarial Networks (GANs) [ 50 ]. IS considers both the distinctiveness of individual images and the overall variety within the generated set.

Fréchet Inception Distance (FID): This metric compares the similarity between the distribution of real images and the distribution of generated images in a feature space extracted by a deep learning model [ 51 ] (often the Inception network). Lower FID values indicate better alignment between the real and generated data distributions.

Kernel Inception Distance (KID): Similar to FID, KID measures the discrepancy between the feature representations of real and generated images. However, KID utilizes kernel methods for the comparison. It is primarily used to assess the overall quality of generated images [ 52 ].

Learned Perceptual Image Patch Similarity (LPIPS): This metric goes beyond basic image statistics and leverages a pre-trained deep learning model to assess the perceptual similarity between images. LPIPS considers factors like human visual perception and aims to quantify how similar two images appear to the human eye [ 53 ].

By employing these metrics alongside traditional methods, researchers gain a more comprehensive understanding of how models perform in scenarios lacking ground truth data.

While Inception Score (IS), Fréchet Inception Distance (FID), Kernel Inception Distance (KID), and Learned Perceptual Image Patch Similarity (LPIPS) are valuable tools for assessing the quality and diversity of generated images, they may not directly translate to evaluating image harmonization techniques because of the following:

Differing Goals: Image generation aims to create entirely new, realistic images, whereas harmonization focuses on aligning existing images while preserving their anatomical content. Metrics like IS and FID prioritize diversity and novelty, which might not be desirable in harmonization.

Focus on Anatomy: Preserving anatomical accuracy is paramount in harmonization. These metrics, however, do not explicitly assess how well anatomical structures are maintained during the process.

However, there might be situations where these metrics could be incorporated into a broader evaluation framework for harmonization, for example, target domain matching. In this scenario, if the harmonization process involves generating synthetic images to match a specific target domain (e.g., MRI scans from a particular scanner model), these metrics could be used to measure the discrepancy between real images from the target domain and the synthetic images produced during harmonization. This could provide insights into how well the harmonized images capture the characteristics of the target domain but would not necessarily address anatomical preservation.

In conclusion, while the mentioned metrics offer valuable insights for image generation, alternative methods are needed to comprehensively assess the success of image harmonization techniques, particularly regarding anatomical fidelity.

Lastly, LPIPS measures the distance between feature representations extracted from pre-trained deep neural networks, like VGG or ResNet. These feature representations capture high-level perceptual qualities of the images, such as textures, shapes, and structures. Unlike metrics like IS or FID, LPIPS considers these learned features, potentially making it a valuable tool for image harmonization. While LPIPS was not specifically designed for harmonization, it offers a unique advantage: assessing the perceptual quality and fidelity of harmonized images. Specifically, it reports on the similarity between the harmonized images and the images from the target domain in terms of human perception. Further research is needed to determine its full effectiveness for harmonization evaluation in various contexts.

These metrics offer insights into the quality and diversity of harmonized images. Clinically, they can be useful in scenarios where harmonization involves generating synthetic images to match a target domain. For instance, when harmonizing images to a specific scanner's characteristics, these metrics help ensure that the generated images align well with the clinical standards of the target domain, thus facilitating consistency in diagnostic practices. IS measures the quality and diversity of generated images, while FID and KID compare the distribution of real and generated images in a feature space. LPIPS evaluates perceptual similarity based on high-level features. When used in harmonization tasks, these metrics can help assess how well the harmonized images match the target domain characteristics, particularly in terms of perceptual and feature-level fidelity.

In addition to the mentioned metrics, it is crucial to evaluate not only the technical aspects of the harmonization process but also its practical utility in real-world applications. It is important to consider how well the harmonized MRI data perform in tasks such as disease classification, age estimation, and ROI segmentation. For instance, ImUnity model [ 54 ] assessed the classification ability to identify individuals with ASD (Autism Spectrum Disorder) within the ABIDE database, both before and after the harmonization process. In [ 55 ] the improvement of Alzheimer's disease classification was reported after applying a harmonization strategy. In another study [ 56 ], the segmentation of the thalamus from various MR image modalities was performed, and the impact of harmonization on the segmentation algorithm was investigated.

Common metrics used to evaluate MRI harmonization methods for downstream tasks like segmentation and classification include Dice Similarity Coefficient, Jaccard Index, Accuracy, Precision, and Recall. Other metrics such as Hausdorff Distance, F1-score, AUC-ROC, and Sensitivity are also employed [ 57 ]. These metrics offer quantitative measures of the performance of harmonization methods in tasks like anatomical segmentation and disease classification.

Deep learning-based harmonization taxonomy

Designing a successful harmonization network hinges on four critical elements:

Data Availability: The type of data available for training, whether paired (corresponding images from source and target domains) or unpaired (images from each domain without direct matches), significantly impacts the design choices

Loss Functions: These functions mathematically quantify the errors made by the network during training. The specific loss function chosen guides the network toward achieving the desired harmonization goals.

Backbone Architecture: The underlying architecture of the neural network serves as the foundation for learning image representations. Different architectures offer varying capabilities for feature extraction and image transformation.

Learning Procedure: The optimization algorithm used to train the network plays a crucial role in its effectiveness. This includes techniques for adjusting network weights and parameters to minimize errors.

The following sections delve into a systematic categorization of deep learning-based harmonization methods. This categorization is based on these four key aspects:

Network Architecture: The underlying structure of the neural network.

Network Learning Algorithm: The specific optimization technique used for training.

Network Supervision Strategy: The approach used to guide the training process of the neural network.

Network Output: The form of the output generated by the network (e.g., harmonized image, segmentation map).

Figure 4 provides a visual representation of this proposed classification scheme, highlighting the different aspects considered.

Taxonomy of deep learning-based MRI harmonization approaches

Network architecture

The choice of network architecture is critical for MRI harmonization because it dictates the model's ability to learn and represent the complex relationships between images acquired from different sources or scanners. Different architectures offer varying degrees of complexity and flexibility, which ultimately influence their performance in harmonizing MRI data. Figure 5 provides a historical timeline illustrating the evolution of deep learning networks for MRI harmonization in recent years.

Timeline of deep harmonization methods

To better understand these methods, we can categorize them based on the underlying network architecture they employ. Some of the commonly used architectures (Fig. 6 ) in MRI harmonization include U-Net, GANs, VAEs, Flow-based Generative Models, Transformers, and Custom Networks (approaches are not exclusively in one category and are a combination of several networks).

General diagram for deep learning-based medical image harmonization. A U-Net-based methods architecture. B GAN-based method architecture. C CycleGAN-based method architecture. D VAE-based method architecture. E. Flow-based generative model. F Transformers-based architecture

The U-Net architecture has been shown to be successful in segmenting [ 58 ] and synthesizing medical images [ 59 ]. The U-Net architecture has provided good results in dealing with large and diverse datasets in medical imaging. Due to the skip connections, it effectively retains the finer details from the initial images and has demonstrated strong performance in image-to-image translation [ 60 ] (Fig. 6 A). One notable advantage of employing U-Net is its ability to enhance data with elastic deformation. It also can extract a large number of feature channels in upsampling. However, an inherent limitation is its comprehensive downsampling, which may result in the loss of spatial information [ 61 ].

On the basis of U-Net architecture, a supervised contrast harmonization has been introduced which is called DeepHarmony [ 62 ]. An overlap cohort was provided through two different protocols in order to get training data. This technique transforms images from protocols to generate harmonized ones that imitate the contrast of the target protocol. DeepHarmony is trained in two ways: one-to-one and many-to-one. Both of them require four separate networks. The former uses a single contrast from the source protocol and produces an image with the corresponding contrast from the target protocol. The latter employs four of the input contrasts including T1-weighted, FLAIR, PD-weighted, and T2-weighted from the source protocol to generate an output contrast from the target protocol. According to the result, this approach improves the result compared with one-to-one, but it needs further parameters. This approach needs training data of paired traveling subjects. In large-scale studies, it is hard to acquire.

Bottani et al. [ 63 ] utilized three architectures based on 3D U-Net to synthesize T1w non-contrast enhancement (T1w-nce) from T1w contrast enhancement (T1w-ce). These modifications included a version with added residual connections referred to as Res-U-Net, a version with incorporated attention mechanisms called Att-U-Net, and a version incorporating both transformer and convolutional layers known as Trans-U-Net. These models were employed as standalone generators and also incorporated into a conditional GAN setup, along with the addition of a patch-based discriminator. Although the models offered a degree of interpretability and provided promising results in brain image segmentation, there is a limitation to creating paired T1w-nce and T1w-ce due to the time and cost constraints. In another study [ 64 ], a U-Net model was developed to learn the non-linear transformation from the contrast of a source image to that of a target image across three MRI contrasts. The training and validation have been accomplished using 2D paired MR images.

The U-Net architecture synthesizes images at the pixel level using paired data, necessitating precise image coregistration for effective model training. Consequently, inadequate alignment of paired MR images could result in the loss of certain brain structure information in the generated images.

Generative adversarial network (GAN)

Generative adversarial networks are an approach to generative modeling using deep learning methods. Such architecture can be considered as image-to-image translation, which generates an image in an unsupervised learning task. GANs have attracted enormous interest in image translation, typically to generate new images from existing ones. The incorporation of a generator and a discriminator creates a GAN model (Fig. 6 B).

A CycleGAN [ 65 ], which is widely used in MRI harmonization, stands as a robust deep learning framework facilitating image-to-image translation without the necessity of paired training data. It is made up of two GANs, including two discriminators and two generators (Fig. 6 C). The objective of the model is to comprehend the attributes of the target domain and produce novel images from the source domain that exhibit these attributes. CycleGAN offers several key advantages over other image-to-image translation models. It excels in terms of accuracy by utilizing unpaired data, thus delivering superior results without the need for an extensive collection of paired training images. Its robustness to domain shifts in the data makes it versatile, allowing it to perform well with input images from various domains, enabling a wide range of translation tasks. Moreover, CycleGAN can generate high-quality images with smaller datasets, making it particularly valuable for tasks with limited training data, such as medical image-to-image harmonization [ 66 , 67 ].

In [ 68 ], an unsupervised image-to-image canonical mapping based on CycleGAN was learned from a diverse dataset to a reference domain. This approach was evaluated on brain age prediction and schizophrenia classification to show how can mitigate confounding data variation while retaining semantic information. A Maximum Classifier Discrepancy Generative Adversarial Network (MCD-GAN) was introduced [ 69 ], leveraging the benefits of both generative models and maximum discrepancy theory. Komandur et al. [ 66 ] employed a 3D CycleGAN to harmonize brain MRI data from diverse sources. They concluded that GAN-harmonized data yield higher accuracy compared to raw data for the age prediction task.

Training the CycleGAN can be time consuming, especially when dealing with a large number of training images. Overfitting is a concern with CycleGAN, possibly resulting in inferior outcomes on unseen data. Interpretability can be a hurdle with CycleGAN, making it challenging to comprehend the rationale behind the model's generated results. To address infant neuroimaging datasets harmonization, S2SGAN (Surface-to-Surface GAN) was introduced [ 70 ]. This method combines the spherical U-Net and the CycleGAN. The presented cycle-consistent adversarial networks are based on a spherical cortical surface for harmonizing cortical thickness maps between different scanners.

The majority of unsupervised approaches are unable to differentiate between variability caused by image acquisition and that originating from population differences across different sites. As a result, these methods necessitate datasets to include subjects or patient groups with comparable clinical or demographic profiles. Deep learning frameworks have shown success in dealing with image translation by breaking it down into basic content (e.g., line contours and orientation) and complex style (primarily color and texture) [ 71 ]. This framework is known by the “Disentangled Representation” term (more details are provided in part 3–2). All images with the same domain have the same content space but the style can vary. This allows the framework to make changes to the style while maintaining the original content. The aim is to maintain consistency in content but adjust the image style. This approach has been used to produce highly realistic translation results.

In [ 72 ], cross-site MRI image harmonization has been considered a style transfer problem instead of a domain transfer problem to overcome the need for the datasets to include patient groups or subjects with homogeneous demographic information. The proposed approach tries to overcome the limitation of the statistical method [ 73 ]. Some statistical methods need certain clinical or demographic characteristics of subjects within the dataset to control the acquisition-based variance. This method is capable of harmonizing MRI images without prior knowledge of their scan/site labels and harmonized by infusing style information derived from a single reference image.

The StarGAN is a version of GAN that enables Image-to-Image GANs to perform mapping across more than two domains using a single generator and acquire shared features applicable to all domains [ 74 ], whereas conventional models need multiple generators. However, it has limited ability to capture minor feature variations. The StarGAN v2 was utilized to process various datasets using canonical mapping from different sites to a reference domain [ 75 ]. By doing so, they reduced the impact of site-based variance while preserving the meaning provided by the input data.

A 3D model named Image Generation with Unified Adversarial Networks (IGUANe) was introduced [ 76 ] that benefits domain translation and style transfer methods to harmonize multicenter brain MR images. It extends the CycleGAN architecture by integrating multiple domains for training using a many-to-one approach.

Addressing hallucinations in GANs

Hallucinations refer to the generation of artificial structures in the output images that do not correspond to real anatomical features. This is particularly problematic in medical applications, where the authenticity of every detail is crucial for accurate diagnosis and treatment planning.

The causes of hallucinations in GANs are multifaceted [ 77 ]. One primary cause is the imbalance between the generator and the discriminator during training, where the generator may learn to produce plausible but incorrect details to fool the discriminator. Another cause is the lack of sufficient and diverse training data, which can lead to overfitting and the generation of unrealistic artifacts. Additionally, the inherent randomness in GANs can introduce noise that manifests as hallucinations.

To mitigate hallucinations, several strategies have been suggested in the literature. One approach is to enhance the quality and quantity of training data [ 78 ]. Techniques such as data augmentation can also help in this regard. Another method is to improve the training process through techniques like progressive growing of GANs [ 79 ], where the model is trained on low-resolution images initially and progressively moves to higher resolutions, allowing for more stable training and better-quality outputs.

Regularization techniques, such as spectral normalization and gradient penalty [ 80 ], can also help by stabilizing the training dynamics and reducing the likelihood of hallucinations. Additionally, incorporating domain-specific knowledge through the use of hybrid models that combine GANs with traditional image processing techniques or other deep learning models can provide more reliable outputs. By integrating these strategies, future work can aim to reduce the incidence of hallucinations in GAN-generated images, thereby enhancing their clinical applicability and reliability.

Variational autoencoders (VAEs)

The GAN networks and VAE are two of the most popular AI image generators. The VAEs consist of two main architectures: encoders and decoders (Fig. 6 D). The encoder learns and encodes the representation of input data and maps it to the latent space. The decoder converts the latent space to get back the original data [ 81 ]. According to the comparative study regarding anomaly segmentation on brain MRI images, GAN-based models are recognized for their ability to generate ultra-realistic and sharp images [ 82 ]. Meanwhile, AutoEncoders are known for their propensity to produce blurry reconstructions.

Torbati et al. proposed a multi-scanner harmonization framework [ 83 ]. This encoder–decoder architecture maps the MRIs from multi-scanners to the latent space and then maps the latent embedding to the harmonized image space. It considers two training steps to preserving the anatomical structure: (1) the harmonized images and input image should be as similar as possible and the variance of embeddings across the scanners should be minimized; (2) ensuring that the output images remain similar across scanners. This step helps maintain uniformity and consistency between various scans.

One-shot learning learns from limited data and has shown significant results in many tasks of medical imaging [ 84 , 85 , 86 ]. Based on VAEs, a one-shot learning method was proposed for harmonization across imaging locations [ 49 ]. During testing, the architecture utilizes an image from a clinical site to create an image that aligns with the intensity scale of the cooperating sites. In another study, a zero-shot learning framework using style-blind autoencoders was introduced [ 87 ]. The network was trained to recognize and extract essential content information exclusively. Consequently, the trained network demonstrated the capability for zero-shot harmonization by discarding unknown scanner-dependent contrast information.

The architectures based on the combination of GAN networks and VAEs have been presented in multiple studies. Cackowski et al. introduced ImUnity [ 54 ]. To decrease the effect of the scanner or site identity on the training results, the generator (VAE) was equipped with a bias learning module connected to the bottleneck. Additionally, a biological preservation module was proposed to maintain pertinent biological information within the latent space representation.

Flow-based generative model

A flow-based generative model is a type of generative model that transforms a simple input distribution into a more complex data distribution using a series of invertible transformations called flows (Fig. 6 E). These models offer exact likelihood evaluation, making them suitable for tasks like density estimation. They are flexible, scalable, and can handle high-dimensional data, making them applicable to various tasks such as image generation and denoising.

Recently, BlindHarmony [ 88 ] was introduced as a solution for blind harmonization, where a flow-based blind MR image harmonization framework was developed. BlindHarmony utilized only the target domain dataset during training. The objective is to discover a harmonized image that retains the anatomical structure and contrast of the input source domain image while ensuring a high likelihood in the flow model, thus facilitating harmonization for the target domain by leveraging the invertibility of flow models. Beizaee et al. proposed an unsupervised MR harmonization method based on normalizing flow [ 89 ]. Within this framework, a shallow harmonizer network was trained to restore images of the source domain from their augmented counterparts. Subsequently, a normalizing flow network was trained to understand the distribution of the source domain. Ultimately, during testing, modifications were made to the harmonizer network so that the resulting images aligned with the distribution learned by the normalizing flow model of the source domain. In another study, a causal flow-based approach was proposed to address the issue of varying feature distributions in multi-site data utilized for Parkinson's disease classification [ 90 ].

Flow-based models are inherently invertible, allowing for bidirectional mapping between domains without loss of information and the transformation process is interpretable so facilitates a better understanding of the harmonization process. However, compared to other generative models, flow-based methods are relatively newer in the field of MRI harmonization, leading to fewer established techniques and benchmarks.

Transformers

Recently, transformers with attention mechanisms (Fig. 6 F) have gained promising performance in medical image processing [ 91 ] and image-to-image translation [ 92 ]. Yao et al. [ 93 ] employed two attention-based image-to-image translation frameworks, Morph-UGATIT and QS-Attn [ 94 ] for MRI harmonization. The effectiveness of these harmonization strategies was evaluated and compared to the conventional CycleGAN by performing a subcortical segmentation task on a heterogeneous dataset acquired at 1.5T and 3T. Among the frameworks assessed, QS-Attn stands out with the most optimal performance. Morph-UGATIT shows comparable performance to QS-Attn and exhibits enhancements in most subcortical regions compared to the CycleGAN model. They concluded that attention-based harmonization techniques demonstrate notable improvements over the baseline frameworks, especially when combined with diverse downstream tasks like segmentation. In [ 95 ] two transformer encoders were introduced to extract both style and content information from MR images, and two decoders were utilized to generate harmonized image patches. Additionally, the impact of changes in image resolution on position encoding was addressed. To capture semantic information in images of varying scales, a content-aware positional encoding scheme method was employed, effectively accommodating images of different sizes.

Custom networks

The custom-designed networks are characterized by architectures that, while sharing similarities with U-Net, GANs, VAEs, flow-based generative models, and transformer-based approaches, include distinct components and configurations tailored to address specific challenges in this research domain. These architectures utilize elements that diverge from typical implementations of the mentioned architectures. They incorporate unique configurations of convolutional layers, pooling layers, domain classifiers, and specially designed blocks that may not fit into these established categories.

Inspired by the adversarial framework and domain adaptation techniques, a harmonization approach was introduced that can be effective for classification, regression, and segmentation tasks while employing two diverse network architectures [ 96 ]. Image harmonization can be considered a multi-source joint domain adaptation problem. This approach tries to produce shared feature representations that are invariant to the acquisition scanner while still completing the main task of interest across scanners and acquisition protocols with minimum performance compromise.

An attention-guided domain adaptation was introduced for multi-site MRI harmonization and was applied to automated brain disorder identification [ 97 ]. In this framework, the attention discovery and domain transfer modules were defined to automatically pinpoint discriminative dementia-related regions in each whole-brain MRI scan and facilitate knowledge transfer between the source and target domains, respectively. Wolleb et al. [ 98 ] introduced a constraint in the latent space of an encoder–classifier network to ignore scanner-related characteristics.

Network learning algorithm

The learning strategies for MRI harmonization has been categorized into two main groups: disentangled learning (DL) methods and non-disentangled learning methods (non-DLs).

Disentangled Learning (DL) Methods: Disentangled learning methods refer to approaches where the neural network or algorithm is explicitly designed to learn separate and interpretable factors or features from the input data. In the context of MRI harmonization,

DL methods aim to disentangle latent factors such as imaging artifacts, variations in acquisition protocols, tissue types, and other confounding factors that contribute to variability in MRI scans.

These methods typically employ architectures such as VAEs, adversarial training techniques, or other models with explicit mechanisms to learn invariant representations across different datasets.

The goal of DL methods is to improve the robustness and generalization of MRI harmonization by separating out and modeling the underlying factors of variability.

Non-Disentangled Learning Methods (non-DLs): Non-disentangled learning methods, in contrast, do not prioritize the disentanglement of underlying factors in the input data:

These methods may include traditional neural networks, regression-based models, or simpler machine learning algorithms.

They focus on direct mapping from input (MRI scans with variability) to output (harmonized MRI scans) without explicitly modeling or separating out the distinct factors contributing to variability.

While effective in certain scenarios, non-DL methods may be less robust to dataset variations and might not generalize as well across different MRI datasets with varying acquisition conditions.

Studies have shown that inverting the magnetic resonance imaging signal equation produces an encoded image with disentangled contrast data and contrast-invariant anatomical data. This means the image can be separated into two distinct parts that are denoted by $\beta$ and $\theta$ for anatomical map and contrast, respectively. Figure 7 demonstrates the disentanglement representation learning diagram which uses encoder–decoder architecture.

Disentanglement representation learning diagram

Dewey et al. proposed an approach that includes two sub-networks [ 99 ]. The first one is the encoder and the second one is the decoder. These two sub-networks are connected by the latent space which contains disentangled contrast data and contrast-invariant anatomical data. This encoder and decoder are based on U-Net architecture. For training the network, the T1-weighted and T2-weighted images of the same anatomy using different scanning protocols. It is expected that the network will produce the same $\beta$ value for the same anatomies and equal $\theta$ for images generated from scanners with the same protocol. This approach utilizes the multiple contrast magnetic resonance (MR) images acquired within each site. These intra-site paired data can be found in the same session. But, relying on this technique alone will not provide a globally disentangled latent space. Zuo et al. introduced a similar architecture which is called CALAMITI [ 100 ] based on information bottleneck theory. The algorithm learns a global latent space of anatomical and contrast information and it can be adapted to a new testing site using only the data collected at the new site. This architecture requires paired images from the same site during training which might limit certain applications, especially in cases where obtaining multi-contrast images is not feasible. Zuo et al. introduced a method that uses a single MR modality [ 101 ]. The inputs of the encoder–decoder are two slices from different orientations of the same 3D volume instead of paired images. Additionally, they defined a new information-based metric for evaluating disentanglement.

According to the inspiration of advancements in multi-domain image translation, Multiple-site Unsupervised Representation Disentanglement (MURD) was introduced [ 102 ]. The harmonized images were produced using combining the content of the original image with styles from a specific site or a generator. The style generator enables the generating of multiple appearances concerning natural style variations associated with each site. In [ 103 ], another work based on disentangled representation was introduced that disengaged the image into content and scanner-specific space. This method was evaluated on healthy controls and multiple sclerosis (MS) cohorts. Zhao et al. developed a deep learning model to harmonize the multi-site cortical data using a surface-based autoencoder [ 104 ]. The encoded cortical features were subsequently decomposed into components related to site-specific characteristics and those unrelated to site effects. An adversarial strategy was employed to promote the disentanglement of these components. Subsequently, the decoding of the site-unrelated features, combined with other site-related features, facilitates the generation of mappings across different sites.

A disentangled latent energy-based style translation (DLEST) framework was introduced [ 105 ] in order to harmonize image-level structural MRI. The proposed model disentangles site-specific style translation and site-invariant image generation through the utilization of an energy-based model and a latent autoencoder.

Harmonization with Attention-based Contrast, Anatomy, and Artifact Awareness (HACA3) was suggested to overcome some drawbacks of synthetic-based disentanglement [ 106 ]. The previous methods to harmonize MR images are limited by their reliance on assumptions that contrast images from the same subject share the same anatomy. These assumptions are doubtful as different contrasts are aimed at highlighting distinct anatomical features. Moreover, these methods require a fixed set of images for training, which is often limited to T1-weighted and T2-weighted data. Finally, the existing methods are sensitive to artifact images and other image artifacts, making them less useful in practical applications.

Utilizing DRL's advantages, it becomes possible to learn and align independent factors within generation objectives with the latent representation through disentanglement. Consequently, this enables effective control over the generation process [ 107 ].

In contrast, non-disentangled representations encapsulate multiple factors of variation in a more intertwined manner, making it challenging to isolate individual factors and understand their influence on the harmonization process. While non-disentangled approaches may offer simplicity and computational efficiency, they often lack the interpretability and robustness necessary for reliable MRI harmonization across diverse datasets.

Network supervision strategy

Broadly, harmonization techniques can be classified into major categories, including supervised, unsupervised, semi-supervised, and self-supervised approaches. Supervised harmonization methods [ 62 , 108 ] are employed to harmonize images from different scanners/sites using a cross-domain dataset. These methods require a group of subjects to be scanned in both domains. This arrangement provides the training and validation data that the model requires. Due to the logistics and costs of acquiring data, gathering cross-domain data is uncommon in practice. Additionally, cross-domain data are limited. Typically, data from multiple domains are available without cross-domain data. This necessitates an unsupervised harmonization method [ 66 , 89 , 109 ], which requires a training method without data from the same subjects from multiple domains.

Semi-supervised approaches [ 110 ], on the other hand, are trained using a dataset containing under-sampled acquisitions of both source and target contrasts from MRI scans. In contrast, self-supervised approaches in the realm of harmonization enable models to learn from the inherent structure of the data itself, eliminating the need for external labels. However, it is important to note that many methods introduced in the field of harmonization are categorized under unsupervised techniques.

Network output

In MRI harmonization, methods can be categorized into two categories based on their network output: direct and indirect. In the direct network output category, methods focus on predicting the target image directly from the reference image. Deep learning models within this category are specifically trained for harmonizing data, allowing for straightforward evaluation by a radiologist. Conversely, methods in the indirect network output category involve training models on a downstream task, such as classification, registration, segmentation, or age prediction. In this category, the harmonization process occurs implicitly through optimization during training, resulting in harmonized data that remain concealed from direct observation. Figure 8 illustrates the general diagram of direct and indirect harmonization.

Classifying Harmonization according to Network Output

There are some papers in the literature that consider harmonization for downstream tasks. Grigorescu et al. [ 111 ] explored two unsupervised domain adaptation techniques, seeking the optimal solution for tissue segmentation maps using T2-weighted magnetic resonance imaging data from an unseen neonatal population born preterm. In [ 56 ], a 3D U-Net architecture was presented to segment the thalamus from multiple MR image modalities, and the effect of harmonization on the segmentation algorithm was investigated. Tor-Diez et al. [ 112 ] used an unpaired image-to-image translation strategy based on adversarial networks and supervised segmentation for the anterior visual pathway. They concluded that harmonization can improve the segmentation results significantly. In another study, with the aim of boosting Alzheimer's disease classification, the Attention-Guided Generative Adversarial Network (AG-GAN) was used for data harmonization [ 55 ]. Komandur et al. [ 66 ] proposed a CycleGAN-based harmonization for improving the results of age estimation. According to the assumption that neglecting downstream applications during harmonization can hinder overall performance, the goal-specific harmonization framework was proposed [ 113 ]. This VAE-based architecture utilizes downstream application performance to regulate the harmonization procedure. They concluded that while this approach enhances downstream performance, it may also limit generalization to new downstream applications, potentially necessitating repetition of the training procedure for each one.

In indirect approaches, according to downstream tasks, different objective functions for clustering, classification, or regression can be defined, so they need a diverse range of learning procedures, parameters, and optimization algorithms. Therefore, when addressing a different task, the harmonization technique needs to be initiated anew.

The details regarding the papers' information are presented in Table 3 . Since the datasets and evaluation metrics are different the comprehensive comparison is limited. The variation in scanners and number of participants, healthy and patient cases, and the investigated contrasts can affect the result and applicability.

Applicability and limitations of harmonization

Harmonization techniques in MRI neuroimaging are essential for mitigating scanner-related variability and site effects, allowing for more reliable and comparable results across different studies and cohorts. These methods enhance the statistical power of studies by increasing the effective sample size and facilitating meta-analyses and multi-site collaborations. Harmonization is particularly beneficial in large-scale studies involving data collected from different scanners, protocols, and populations.

Despite its advantages, harmonization is not without limitations. One major concern is that harmonization algorithms may introduce bias if not properly validated across different datasets. The effectiveness of harmonization can vary depending on the specific characteristics of the datasets, including differences in scanner types, imaging protocols, and the populations being studied. Additionally, harmonization processes might inadvertently remove or obscure biologically relevant variations that are not related to scanner differences.

Another limitation is the complexity and computational cost associated with advanced harmonization techniques. Implementing these methods often requires significant expertise and resources, which may not be readily available in all research settings. Furthermore, the choice of harmonization method can impact the results, necessitating careful consideration and validation of the chosen approach.

Neuroimaging analysis should not always use harmonization, especially in scenarios where the primary goal is to investigate scanner-specific effects or when studying the inherent variability between different imaging systems. In such cases, harmonization could mask the very differences that are of interest. Additionally, in single-site studies with consistent imaging protocols, the need for harmonization may be minimal or unnecessary.

Harmonization might negatively impact results if applied inappropriately. For instance, over-harmonization can lead to the loss of important biological signals, resulting in reduced sensitivity to detect true effects. It is crucial to balance the removal of unwanted scanner-related variance with the preservation of genuine biological variability. Researchers should perform extensive validation to ensure that harmonization does not distort the data in a way that affects the study outcomes.

Inconsistent contrast across MRI scans presents a significant hurdle for modern medical image analysis techniques. This becomes particularly evident when using deep learning models trained on specific image types. For example, a segmentation model designed for CT scans might perform poorly on MR images due to the fundamental differences in how these imaging techniques capture the body. While research efforts like cross-domain synthesis [ 114 , 115 , 116 , 117 ] have aimed to address these challenges, inconsistencies in contrast remain a persistent issue, even within the realm of MRI scans themselves [ 28 ].

It is important to distinguish between image harmonization and cross-domain synthesis, although both techniques address challenges with image variability. While image harmonization aims to align images from different sources (e.g., scanners) while preserving anatomical details and spatial relationships, cross-domain synthesis focuses on generating images in a target domain based on images from a different source domain. While the former is particularly useful when combining datasets from different sources for analysis and it helps ensure consistency and reduces variability, enabling more reliable comparisons, the goal of the latter is to preserve key features like structures or textures, while also creating visually realistic images in the target domain. This technique can be used for data augmentation, domain adaptation, and image enhancement. In essence, harmonization aims to make existing images from different sources more compatible, while cross-domain synthesis aims to create entirely new images within a specific domain. Given this distinction, and our focus on harmonization techniques, we have not explored cross-domain synthesis techniques within this paper.

While most harmonization techniques reviewed here leverage 2D images, there is growing recognition that 3D models offer significant advantages. 3D models hold greater potential for capturing the full complexity of medical image features, potentially leading to improved learning performance. However, 3D models come with significant computational limitations such as increased memory requirements to store and process 3D data and longer training times for deep learning models due to the larger amount of data. To address these challenges, some harmonization approaches employ a hybrid strategy for example, 2D Slices with Multi-Orientation. In this approach, models are trained using 2D axial, coronal, and sagittal slices extracted from each 3D MR volume. Subsequently, these multi-directional 2D slices are then combined into a harmonized 3D volume [ 106 ] .

A majority of current harmonization methods reported in the literature evaluate their performance primarily on T1-weighted MRI scans. While this is a common starting point, it is important to acknowledge the limitation of limited generalizability, i.e., these methods might not achieve the same level of success with other MRI contrasts, such as T2-weighted, PD-weighted, and T2-FLAIR images.

The preprocessing steps before harmonization approach can affect the harmonization outcome. In many cases, the harmonizing native MRI is an essential step. Subsequent investigation should clarify the influence of harmonization on native MRI and explore how the quality of harmonization can be conditional on the preprocessing procedure employed [ 73 ]. By understanding this interplay between preprocessing and harmonization, researchers can develop more robust and effective pipelines for MRI data analysis.

Evaluating and comparing different harmonization techniques presents several obstacles. Firstly, there is a current lack of standardized and comprehensive datasets encompassing a wide variety of MRI contrasts, scanner types, and patient demographics (healthy vs. patient groups). Many studies rely on subsets of data, focusing on scanners with minimal differences or specific patient groups. Secondly, given the importance of harmonization in medical image processing, establishing a reference dataset would be highly beneficial. This dataset should ideally include multiple MRI contrasts, involve data from various patient cohorts, and encompass a diversity of challenges commonly encountered in real-world scenarios. Such a benchmark would facilitate more comprehensive evaluation and comparison of harmonization techniques for downstream tasks. Thirdly, conventional image similarity metrics might not fully capture the effectiveness of harmonization. They may prioritize overall similarity without adequately considering factors like cross-domain consistency (compatibility between data from different sources) and preservation of crucial anatomical details. Re-evaluating harmonization success requires metrics that comprehensively assess these key aspects.

While supervised learning approaches using U-Net convolutional neural networks have shown promise in MRI harmonization, they have limitations. Firstly, supervised methods require paired data, meaning the same patients need to be scanned on multiple scanners. This can be expensive and time consuming to acquire, limiting the applicability of these approaches. Secondly, supervised methods often work best for brain imaging due to the relative homogeneity of the MR signal and the feasibility of performing rigid image registration (aligning images based on anatomical landmarks). Addressing these challenges will be crucial to advance the development and evaluation of effective harmonization techniques for broader applications in medical image analysis.

Accurately assessing the effectiveness of different image harmonization techniques remains a challenge due to two key limitations. Firstly, there is an absence of standardized benchmark datasets encompassing a wide variety of factors hinders comprehensive evaluation and comparison. The current studies often rely on the following:

Subsets of data: Some studies use a limited portion of a larger dataset, potentially missing valuable information.

Homogeneous data: Some studies focus on data acquired from scanners with minimal differences, limiting the generalizability of findings.

Data from specific patient groups: Some datasets might be restricted to healthy or diseased individuals, neglecting the real-world scenario where datasets may include both.

Given the crucial role of harmonization in medical image processing, a robust reference dataset is urgently needed. This dataset should ideally include multiple MRI contrasts (T1-weighted, T2-weighted, etc.), data from diverse patient cohorts (healthy and diseased) and a variety of challenges commonly encountered in real-world settings (e.g., scanner variations, acquisition protocols). Such a comprehensive benchmark would enable researchers to thoroughly evaluate and compare harmonization techniques, ultimately improving their performance in downstream tasks.

Conventional image similarity metrics (like PSNR or SSIM) primarily focus on overall image similarity. While important, they may not fully capture the cross-domain consistency, i.e., how well does the harmonized image align with data from a different source (e.g., another scanner)? They may not also fully provide anatomical preservation, i.e., does the harmonized image retain the crucial anatomical details present in the original image? To address these limitations, a re-evaluation of success metrics is necessary. New metrics should be developed, or existing ones adapted to comprehensively assess these essential aspects of harmonization.

Addressing these limitations in datasets and evaluation methods represents a crucial step toward the development and implementation of next-generation harmonization techniques for broader use in medical image analysis.

While U-Net convolutional neural networks have shown promise in supervised learning approaches for MRI harmonization, they face some limitations such as paired data dependency, applicability constraints such as MR signal homogeneity and rigid image registration. These limitations restrict the broader applicability of supervised U-Net-based approaches for harmonization in medical image analysis.

Generative Adversarial Networks (GANs) have been employed to address harmonization by synthesizing images with a specified contrast, where the “content” from the input image is retained while adjusting the contrast to match that of a target scanner. The CycleGAN utilizes unpaired training data and unsupervised learning, showing promise in harmonization tasks and leading to developments in the prediction of brain age and classification. However, an inherent limitation of GANs is their inability to inherently distinguish content from contrast, potentially resulting in alterations to anatomical details to align more with the target scanner dataset, causing “geometry shifts.” Preserving patient anatomy is crucial for precise diagnosis and treatment. In the absence of structural uniformity, the generated images might lack clinically significant specifics. Additionally, GANs are well known for producing artificial structures that are not present in the initial training data, a phenomenon commonly referred to as “hallucination.”

Variational Autoencoders (VAEs) offer an alternative approach to MRI harmonization that addresses a key limitation of supervised learning: the need for paired data. VAEs can potentially harmonize data across multiple sites without requiring scans from the same subjects at each location. This is achieved by learning a latent representation of the data, which essentially captures the underlying characteristics of the images in a compressed form. The VAE then transforms data from one site into another using this latent space. However, some of the limitations of VAEs blurry reconstructions and challenging latent space interpretability. While VAEs hold promise for multi-site harmonization without paired data, further research is needed to address these limitations and improve the accuracy and detail preservation in the harmonized images.

Disentangled representation learning aims to separate an image's style (contrast) and content (anatomy) into distinct representations. This allows for modifications to the style while preserving the underlying anatomical details. However, in complex MRI data, factors like contrast and anatomical details can be intertwined and challenging to perfectly separate. This can lead to ambiguities in the disentanglement process, resulting in overlapping or mixed representations of the intended factors. Additionally, extending disentangled representation learning to 3D or higher dimensions presents additional challenges. The increased complexity of higher-dimensional spaces makes it more difficult to disentangle the features within them.

Vision Transformers (ViTs) have emerged as a powerful tool in computer vision, demonstrating effectiveness across diverse tasks like segmentation, classification, and image-to-image translation. This versatility stems from their core mechanism, self-attention. Unlike traditional convolutional neural networks, ViTs can directly analyze relationships between any two parts of an image, allowing them to capture long-range dependencies and gain a deeper understanding of the global context. However, ViTs also face some limitations due to need for high-resolution input images for optimal performance and substantial computational memory and processing power for training and inference. These factors can limit the applicability of ViTs in scenarios with limited computational resources or where processing speed is critical.

In reviewing the advancements in deep learning models for MRI harmonization, it is evident that even marginal improvements in image quality metrics can be of significant clinical value. However, these improvements often appear minor when comparing new network architectures. The statistical analysis of these improvements is crucial to determine their true significance. For instance [ 118 ], highlights the importance of using rigorous statistical methods such as Analysis of Variance (ANOVA) and Mixed Effects Models (MEM).

Similarly [ 119 ], investigated three U-Nets (dense, robust, and anisotropic) for upscaling low-quality MRI images. Despite non-statistically significant differences in basic evaluation metrics, mixed effects statistics illustrated significant differences. This suggests that while the detailed architecture of these U-Nets may not drastically alter the outcomes, the use of robust statistical techniques can reveal critical differences and interactions. These findings underscore the importance of employing comprehensive statistical methods to fully understand and validate the performance of different network configurations.

Furthermore, the application of robust statistical techniques, including cross-validation, paired t-tests [ 120 ], Wilcoxon signed-rank tests [ 121 ], and bootstrap methods, can enhance the reliability, generalizability, and rigor of findings in deep learning model evaluations. These approaches collectively provide a comprehensive framework for assessing model performance beyond subjective evaluation metrics alone. Thus, future research should prioritize not only advancing novel architectures but also ensuring meticulous statistical validation of performance improvements to substantiate their clinical efficacy.

In parallel with these advancements, the integration of foundation models into the harmonization process holds the potential to further refine image quality and consistency across diverse datasets. Foundation models, which are large-scale, pre-trained deep learning models, have recently attracted significant attention across various deep learning challenges. These models are trained on extensive datasets to enhance generalization, contextual reasoning, and adaptability across different modalities. They can be fine-tuned for new tasks using task-specific prompts without the need for extensive retraining or labeled data. The field of medical imaging is increasingly exploring these models to leverage their advanced capabilities and improve outcomes [ 122 ].

While the application of foundation models in image harmonization is still an emerging field, these models offer substantial potential for improving consistency and compatibility across diverse medical imaging datasets. Future research should focus on exploring and optimizing the use of these models, conducting comprehensive quantitative comparisons, and addressing the specific challenges associated with harmonization in medical imaging.

Conclusion and future direction

This review provides a comprehensive overview of state-of-the-art deep learning-based methods for harmonizing Magnetic Resonance Imaging (MRI) scans. We categorized harmonization approaches based on their underlying network architecture, including U-Net, Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), flow-based models, and transformers. We surveyed current literature on MRI harmonization and report significant progress in harmonization techniques, with improvements observed in downstream tasks that rely on harmonized images.

Despite these advancements, several challenges remain:

Data Standardization: The wide variety of acquisition parameters across scanners, disease states, and patient demographics (e.g., gender) can affect brain size and pose challenges for harmonization robustness. Developing standardized datasets encompassing these diverse scenarios and incorporating comprehensive evaluations for each challenge would be valuable.

Evaluation Metrics: Current metrics primarily focus on contrast similarity. Novel metrics are needed to assess how well harmonization techniques address the "shift problem" (differences in image intensity distributions) while preserving crucial anatomical information.

Multi-Site Harmonization: Current methods often focus on harmonization between two specific sites. Exploring techniques that can handle data from multiple sites would be beneficial.

Architectural Innovation: Combining the strengths of different network architectures (e.g., U-Net for segmentation and GANs for image generation) could lead to more robust harmonization solutions. Additionally, computational efficiency should be considered, as faster models are more practical for real-world applications.

Generalizability: Extending harmonization frameworks beyond specific MRI contrasts (T1-weighted, PD-weighted, T2-FLAIR) and even exploring other modalities like PET or CT could be a promising research direction.

By addressing these challenges and exploring new avenues, deep learning has the potential to further revolutionize MRI harmonization, ultimately leading to improved medical diagnosis and treatment planning.

Availability of data and materials

No datasets were generated or analyzed during the current study.

Bangerter NK, Morrell G, Grech-Sollars M. Magnetic resonance imaging. In: bioengineering innovative solutions for cancer. Cambridge: Academic Press; 2019. p. 163–94.

Google Scholar

Khanduri S. Magnetic resonance imaging physics. In: textbook of radiology for CT and MRI technicians with MCQs. 2018;109–109. https://doi.org/10.5005/jp/books/14192.

Carré A, Battistella E, Niyoteka S, Sun R, Deutsch E, Robert C. AutoComBat: a generic method for harmonizing MRI-based radiomic features. Sci Reports. 2022;12(1):12762. https://doi.org/10.1038/s41598-022-16609-1 .

Article Google Scholar

Fratini M, Abdollahzadeh A, DiNuzzo M, Salo RA, Maugeri L, Cedola A, et al. Multiscale imaging approach for studying the central nervous system: methodology and perspective. Front Neurosci. 2020;14:72. https://doi.org/10.3389/fnins.2020.00072 .

Lakshmi MJ, Nagaraja RS. Brain tumor magnetic resonance image classification: a deep learning approach. Soft Comput. 2022;26(13):6245–53. https://doi.org/10.3390/cancers15164172 .

Han KM, Ham BJ. How inflammation affects the brain in depression: a review of functional and structural MRI studies. J Clin Neurol. 2021;17(4):503. https://doi.org/10.3988/jcn.2021.17.4.503 .

Noor MB, Zenia NZ, Kaiser MS, Mamun SA, Mahmud M. Application of deep learning in detecting neurological disorders from magnetic resonance images: a survey on the detection of Alzheimer’s disease, Parkinson’s disease and schizophrenia. Brain Inform. 2020;7:1–21.

IXI Brain Development Dataset. https://brain-development.org/ixi-dataset/ .

Tixier F, Jaouen V, Hognon C, Gallinato O, Colin T, Visvikis D. Evaluation of conventional and deep learning based image harmonization methods in radiomics studies. Phys Med Biol. 2021;66(24):245009. https://doi.org/10.1088/1361-6560/ac39e5 .

Lawrence E, Vegvari C, Ower A, Hadjichrysanthou C, De Wolf F, Anderson RM. A systematic review of longitudinal studies which measure Alzheimer’s disease biomarkers. J Alzheimer’s Dis. 2017. https://doi.org/10.3233/JAD-170261 .

Pomponio R, Erus G, Habes M, Doshi J, Srinivasan D, Mamourian E, et al. Harmonization of large MRI datasets for the analysis of brain imaging patterns throughout the lifespan. Neuroimage. 2020;208:116450. https://doi.org/10.1016/j.neuroimage.2019.116450 .

Wang J, Lan C, Liu C, Ouyang Y, Qin T, Lu W, Chen Y, Zeng W, Philip SY. Generalizing to unseen domains: a survey on domain generalization. IEEE Trans Knowl Data Eng. 2022;35(8):8052–72.

Guan H, Liu M. Domain adaptation for medical image analysis: a survey. IEEE Trans Biomed Eng. 2021;69(3):1173–85.

Chen C, Dou Q, Chen H, Qin J, Heng PA. Unsupervised bidirectional cross-modality adaptation via deeply synergistic image and feature alignment for medical image segmentation. IEEE Trans Med Imaging. 2020;39(7):2494–505.

Zhou K, Liu Z, Qiao Y, Xiang T, Loy CC. Domain generalization: a survey. IEEE Trans Pattern Anal Mach Intell. 2022;45(4):4396–415.

Zhang J, Zuo L, Dewey BE, Remedios SW, Hays SP, Pham DL, Prince JL, Carass A. Harmonization-enriched domain adaptation with light fine-tuning for multiple sclerosis lesion segmentation. InMed Imaging 2024 Clin Biomed Imaging. 2024;12930:635–41.

Tanaka SC, Yamashita A, Yahata N, Itahashi T, Lisi G, Yamada T, et al. A multi-site, multi-disorder resting-state magnetic resonance image database. Sci Data. 2021;8(1):227. https://doi.org/10.6084/m9.figshare.14716329 .

Stamoulou E, Spanakis C, Manikis GC, Karanasiou G, Grigoriadis G, Foukakis T, et al. Harmonization strategies in multicenter MRI-based radiomics. J Imaging. 2022;8(11):303. https://doi.org/10.3390/jimaging8110303 .

Orlhac F, Eertink JJ, Cottereau AS, Zijlstra JM, Thieblemont C, Meignan M, et al. A guide to ComBat harmonization of imaging biomarkers in multicenter studies. J Nuclear Med. 2022;63(2):172–9.

Hu F, Chen AA, Horng H, Bashyam V, Davatzikos C, Alexander-Bloch A, et al. Image harmonization: a review of statistical and deep learning methods for removing batch effects and evaluation metrics for effective harmonization. Neuroimage. 2023;20:120125.

Roca V, Kuchcinski G, Pruvo JP, Manouvriez D, Leclerc X, Lopes R. A three-dimensional deep learning model for inter-site harmonization of structural MR images of the brain: extensive validation with a multicenter dataset. Heliyon. 2023. https://doi.org/10.1016/j.heliyon.2023.e22647 .

Ayaz A, Al KY, Amirrajab S, Lorenz C, Weese J, Pluim J, et al. Brain MR image simulation for deep learning based medical image analysis networks. Comput Methods Programs Biomed. 2024;248:108115.

Klemenz AC, Albrecht L, Manzke M, Dalmer A, Böttcher B, Surov A, et al. Improved image quality in CT pulmonary angiography using deep learning-based image reconstruction. Sci Reports. 2024;14(1):2494.

Deng L, Lan Q, Zhi Q, Huang S, Wang J, Yang X. Deep learning-based 3D brain multimodal medical image registration. Med Biol Eng Comput. 2024;62(2):505–19.

Wen G, Shim V, Holdsworth SJ, Fernandez J, Qiao M, Kasabov N, et al. Machine learning for brain MRI data harmonisation: a systematic review. Bioengineering. 2023;10(4):397. https://doi.org/10.3390/bioengineering10040397 .

Mali SA, Ibrahim A, Woodruff HC, Andrearczyk V, Müller H, Primakov S, et al. Making radiomics more reproducible across scanner and imaging protocol variations: a review of harmonization methods. J Personal Med. 2021;11(9):842. https://doi.org/10.3390/jpm11090842 .

Bayer JMM, Thompson PM, Ching CRK, Liu M, Chen A, Panzenhagen AC, et al. Site effects how-to and when: an overview of retrospective techniques to accommodate site effects in multi-site neuroimaging analyses. Front Neurol. 2022;13:923988.

Zuo L, Liu Y, Prince JL, Carass A. An overview of disentangled representation learning for MR image harmonization. Deep Learn Med Image Anal. 2024;1:135–52.

Yang J, Liu J, Xu N, Huang J. TVT: transferable vision transformer for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2023; 520–530.

Ko K, Yeom T, Lee M. SuperstarGAN: generative adversarial networks for image-to-image translation in large-scale domains. Neural Netw. 2023;162:330–9. https://doi.org/10.1016/j.neunet.2023.02.042 .

Sudlow C, Gallacher J, Allen N, Beral V, Burton P, Danesh J, et al. UK Biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med. 2015;12(3):e1001779. https://doi.org/10.1371/journal.pmed.1001779 .

Di Martino A, Yan CG, Li Q, Denio E, Castellanos FX, Alaerts K, et al. The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism. Mol Psychiatry. 2014;19(6):659–67. https://doi.org/10.1038/mp.2013.78 .

Weiner MW, Veitch DP, Aisen PS, Beckett LA, Cairns NJ, Green RC, et al. The Alzheimer’s disease neuroimaging initiative 3: continued innovation for clinical trial improvement. Alzheimer’s Dementia. 2017;13(5):561–71. https://doi.org/10.1016/j.jalz.2016.10.006 .

Casey BJ, Cannonier T, Conley MI, Cohen AO, Barch DM, Heitzeg MM, et al. The Adolescent brain cognitive development (ABCD) study: imaging acquisition across 21 sites. Dev Cogn Neurosci. 2018;32:43–54. https://doi.org/10.1016/j.dcn.2018.03.001 .

Ellis KA, Bush AI, Darby D, De Fazio D, Foster J, Hudson P, et al. The Australian Imaging, Biomarkers and Lifestyle (AIBL) study of aging: methodology and baseline characteristics of 1112 individuals recruited for a longitudinal study of Alzheimer’s disease. Int Psychogeriatrics. 2009;21(4):672–87. https://doi.org/10.1017/S1041610209009405 .

Marcus DS, Wang TH, Parker J, Csernansky JG, Morris JC, Buckner RL. Open Access Series of Imaging Studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. J Cognit Neurosci. 2007;19(9):1498–507. https://doi.org/10.1162/jocn.2007.19.9.1498 .

Marcus DS, Fotenos AF, Csernansky JG, Morris JC, Buckner RL. Open access series of imaging studies: longitudinal MRI data in nondemented and demented older adults. J Cognit Neurosci. 2010. https://doi.org/10.1162/jocn.2009.21407 .

LaMontagne PJ, Keefe S, Lauren W, Xiong C, Grant EA, Moulder KL, et al. OASIS-3: longitudinal neuroimaging, clinical, and cognitive dataset for normal aging and Alzheimer disease. MedRxiv. 2019. https://doi.org/10.1101/2019.12.13.19014902 .

Ogbole GI, Adeyomoye AO, Badu-Peprah A, Mensah Y, Nzeh DA. Survey of magnetic resonance imaging availability in West Africa. Pan Afr Med J. 2018. https://doi.org/10.11604/pamj.2018.30.240.14000 .

Rutt BK, Lee DH. The impact of field strength on image quality in MRI. J Magnet Reson Imaging. 1996;6(1):57–62. https://doi.org/10.1002/jmri.1880060111 .

Ahmed SY, Hassan FF. Optimizing imaging resolution in brain MRI: understanding the impact of technical factors. J Med Life. 2023;16(6):920. https://doi.org/10.25122/jml-2022-0212 .

Thrower SL, Al Feghali KA, Luo D, Paddick I, Hou P, Briere T, Li J, McAleer MF, McGovern SL, Woodhouse KD, Yeboa DN. The effect of slice thickness on contours of brain metastases for stereotactic radiosurgery. Adv Radiat Oncol. 2021;6(4):100708. https://doi.org/10.1016/j.adro.2021.100708 .

Ma YJ, Jang H, Chang EY, Hiniker A, Head BP, Lee RR, Corey-Bloom J, Bydder GM, Du J. Ultrashort echo time (UTE) magnetic resonance imaging of myelin: technical developments and challenges. Quant Imaging Med Surg. 2020;10(6):1186.

Nazarpoor M. The effect of inversion times on the minimum signal intensity of the contrast agent concentration using inversion recovery t1-weighted fast imaging sequence. Med J Islamic Republic Iran. 2014;28:128.

Morrell GR, Schabel MC. An analysis of the accuracy of magnetic resonance flip angle measurement methods. Phys Med Biol. 2010;55(20):6157.

Sijbers J, Scheunders P, Bonnet N, Van Dyck D, Raman E. Quantification and improvement of the signal-to-noise ratio in a magnetic resonance image acquisition procedure. Magnet Reson Imaging. 1996;14(10):1157–63.

Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process. 2004;13(4):600–12. https://doi.org/10.1109/TIP.2003.819861 .

Ravano V, Démonet JF, Damian D, Meuli R, Piredda GF, Huelnhagen T, Maréchal B, Thiran JP, Kober T, Richiardi J. Neuroimaging harmonization using cGANs: image similarity metrics poorly predict cross-protocol volumetric consistency. Int Workshop Mach Learn Clin Neuroimag. 2022;18:83–92.

Parida A, Jiang Z, Anwar SM, Foreman N, Stence N, Fisher MJ, Packer RJ, Avery RA, Linguraru MG. Harmonization across imaging locations (HAIL): one-shot learning for brain MRI. arXiv. 2023. https://doi.org/10.48550/arXiv.2308.11047 .

Treder MS, Codrai R, Tsvetanov KA. Quality assessment of anatomical MRI images from generative adversarial networks: human assessment and image quality metrics. J Neurosci Methods. 2022;374:109579.

Tronchin L, Sicilia R, Cordelli E, Ramella S, Soda P. Evaluating GANs in medical imaging. In: Engelhardt S, Oksuz I, Zhu D, Yuan Y, Mukhopadhyay A, Heller N, Huang SX, Nguyen H, Sznitman R, Xue Y, editors. Deep generative models, and data augmentation, labelling, and imperfections. MICCAI. Cham: Springer International Publishing; 2021. p. 112–21.

Chapter Google Scholar

Bińkowski M, Sutherland DJ, Arbel M, Gretton A. Demystifying mmd gans. arXiv. 2018. https://doi.org/10.48550/arXiv.1801.01401 .

Zhang R, Isola P, Efros AA, Shechtman E, Wang O. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2018; 586–595.

Cackowski S, Barbier EL, Dojat M, Christen T. Imunity: a generalizable VAE-GAN solution for multicenter MR image harmonization. Med Image Anal. 2023;88:102799. https://doi.org/10.1016/j.media.2023.102799 .

Sinha S, Thomopoulos SI, Lam P, Muir A, Thompson PM. Alzheimer’s disease classification accuracy is improved by MRI harmonization based on attention-guided generative adversarial networks. Int Sympos Med Inform Proc Anal. 2021;12088:180–9. https://doi.org/10.1117/12.2606155 .

Shao M, Zuo L, Carass A, Zhuo J, Gullapalli RP, Prince JL. Evaluating the impact of MR image harmonization on thalamus deep network segmentation. Med Imaging 2022 Image Proc. 2022;12032:115–21. https://doi.org/10.1117/12.2613159 .

Taha AA, Hanbury A. Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med Imaging. 2015;15:1–28.

Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF, editors. Medical Image computing and computer-assisted intervention–MICCAI. Springer International Publishing: Cham; 2015. p. 234–41.

Dewey BE, Zhao C, Carass A, Oh J, Calabresi PA, van Zijl PCM, et al. Deep harmonization of inconsistent MR data for consistent volume segmentation. In: Gooya A, Goksel O, Oguz I, Burgos N, editors., et al., Simulation and synthesis in medical imaging: third international workshop, SASHIMI 2018, held in conjunction with MICCAI. Springer International Publishing: Cham; 2018. p. 20–30.

Shiri I, Ghafarian P, Geramifar P, Leung KHY, Ghelichoghli M, Oveisi M, et al. Direct attenuation correction of brain PET images using only emission data via a deep convolutional encoder-decoder (Deep-DAC). Eur Radiol. 2019;29:6867–79.

Zhang S, Niu Y. LcmUNet: a lightweight network combining CNN and MLP for real-time medical image segmentation. Bioengineering. 2023;10(6):712. https://doi.org/10.3390/bioengineering10060712 .

Dewey BE, Zhao C, Reinhold JC, Carass A, Fitzgerald KC, Sotirchos ES, et al. DeepHarmony: a deep learning approach to contrast harmonization across scanner changes. Magn Reson Imaging. 2019;64:160–70. https://doi.org/10.1016/j.mri.2019.05.041 .

Bottani S, Thibeau-Sutre E, Maire A, Stroër S, Dormont D, Colliot O, et al. Homogenization of brain MRI from a clinical data warehouse using contrast-enhanced to non-contrast-enhanced image translation with U-Net derived models. Med Imaging. 2022;12032:576–82. https://doi.org/10.1117/12.2608565 .

Osman AF, Tamam NM. Deep learning-based convolutional neural network for intramodality brain MRI synthesis. J Appl Clin Med Phys. 2022;23(4):e13530. https://doi.org/10.1002/acm2.13530 .

Zhu JY, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. Proc IEEE Int Con Comput Vision. 2017. https://doi.org/10.1109/ICCV.2017.244 .

Komandur D, Gupta U, Chattopadhyay T, Dhinagar NJ, Sophia I, California S, et al. Unsupervised harmonization of brain MRI using 3D CycleGANs and its effect on brain age prediction. Int Sympos Med Inform Proc Anal (SIPAIM). 2023. https://doi.org/10.1109/SIPAIM56729.2023.10373501 .

Modanwal G, Vellal A, Buda M, Mazurowski MA. MRI image harmonization using cycle-consistent generative adversarial network. Med Imaging. 2020;11314:259–64. https://doi.org/10.1117/12.2551301 .

Bashyam VM, Doshi J, Erus G, Srinivasan D, Abdulkadir A, Habes M, et al. Medical image harmonization using deep learning based canonical mapping: toward robust and generalizable learning in imaging. arXiv. 2020. https://doi.org/10.48550/arXiv.2010.05355 .

Yan W, Fu Z, Sui J, Calhoun VD. ‘Harmless’ adversarial network harmonization approach for removing site effects and improving reproducibility in neuroimaging studies. Ann Int Conf IEEE Eng Med Biol Soc (EMBC). 2022. https://doi.org/10.1109/EMBC48229.2022.9871061 .

Zhao F, Wu Z, Wang L, Lin W, Xia S, Shen D, Li G. UNC/UMN baby connectome project consortium. Harmonization of infant cortical thickness using surface-to-surface cycle-consistent adversarial networks. In: Shen D, Liu T, Peters TM, Staib LH, Essert C, Zhou S, Yap P-T, Khan A, editors. International conference on medical image computing and computer-assisted intervention. Cham: Springer International Publishing; 2019. p. 475–83.

Huang X, Liu MY, Belongie S, Kautz J. Multimodal unsupervised image-to-image translation. In: Proceedings of the European conference on computer vision (ECCV). 2018. p. 172–89.

Liu M, Maiti P, Thomopoulos S, Zhu A, Chai Y, Kim H, et al. Style transfer using generative adversarial networks for multi-site MRI harmonization. Med Image Comput Comput Assist Intervent MICCAI. 2021. https://doi.org/10.1002/hbm.26422 .

Liu M, Zhu AH, Maiti P, Thomopoulos SI, Gadewar S, Chai Y, et al. Style transfer generative adversarial networks to harmonize multisite MRI to a single reference image to avoid overcorrection. Hum Brain Mapp. 2023;44(14):4875–92. https://doi.org/10.1002/hbm.26422 .

Saxena S, Teli MN. Comparison and analysis of image-to-image generative adversarial networks: a survey. arXiv. 2021. https://doi.org/10.48550/arXiv.2112.12625 .

Bashyam VM, Doshi J, Erus G, Srinivasan D, Abdulkadir A, Singh A, et al. Deep generative medical image harmonization for improving cross-site generalization in deep learning predictors. J Magnet Reson Imaging. 2022;55(3):908–16. https://doi.org/10.1002/jmri.27908 .

Roca V, Kuchcinski G, Pruvo JP, Manouvriez D, Lopes R. IGUANe: a 3D generalizable CycleGAN for multicenter harmonization of brain MR images. arXiv. 2024. https://doi.org/10.48550/arXiv.2402.03227 .

Bai Z, Wang P, Xiao T, He T, Han Z, Zhang Z, Shou MZ. Hallucination of multimodal large language models: a survey. arXiv. 2024. https://doi.org/10.48550/arXiv.2404.18930 .

Sarkar P, Ebrahimi S, Etemad A, Beirami A, Arık SÖ, Pfister T. Mitigating object hallucination via data augmented contrastive tuning. arXiv. 2024. https://doi.org/10.48550/arXiv.2405.18654 .

Liang J, Yang X, Huang Y, Li H, He S, Hu X, Chen Z, Xue W, Cheng J, Ni D. Sketch guided and progressive growing GAN for realistic and editable ultrasound image synthesis. Med Image Anal. 2022;79:102461.

Miyato T, Kataoka T, Koyama M, Yoshida Y. Spectral normalization for generative adversarial networks. ArXiv. 2018. https://doi.org/10.48550/arXiv.1802.05957 .

Kingma DP, Welling M. Auto-encoding variational bayes. ArXiv. 2013. https://doi.org/10.48550/arXiv.1312.6114 .

Baur C, Denner S, Wiestler B, Navab N, Albarqouni S. Autoencoders for unsupervised anomaly segmentation in brain MR images: a comparative study. Med Image Anal. 2021;69:101952. https://doi.org/10.1016/j.media.2020.101952 .

Torbati ME, Tudorascu DL, Minhas DS, Maillard P, Decarli CS, Jae Hwang S. Multi-scanner harmonization of paired neuroimaging data via structure preserving embedding learning. Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021; 3284–3293.

Jadon S. An overview of deep learning architectures in few-shot learning domain. arXiv. 2020. https://doi.org/10.48550/arXiv.2008.06365 .

Pachetti E, Colantonio S. A systematic review of few-shot learning in medical imaging. ArXiv. 2023. https://doi.org/10.48550/arXiv.2309.11433 .

Kotia J, Kotwal A, Bharti R, Mangrulkar R. Few shot learning for medical imaging. Mach Learn Algorithms Indust Appl. 2021.

Fatania K, Clark A, Frood R, Scarsbrook A, Al-Qaisieh B, Currie S, et al. Harmonisation of scanner-dependent contrast variations in magnetic resonance imaging for radiation oncology, using style-blind auto-encoders. Phys Imaging Radiat Oncol. 2022;22:115–22. https://doi.org/10.1016/j.phro.2022.05.005 .

Jeong H, Byun H, Kang DU, Lee J. BlindHarmony." Blind" Harmonization for MR Images via Flow model. Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023; 21129–21139.

Beizaee F, Desrosiers C, Lodygensky GA, Dolz J. Harmonizing flows: unsupervised MR harmonization based on normalizing flows. International Conference on Information Processing in Medical Imaging. 2023; 347–359.

Vigneshwaran V, Wilms M, Camacho MI, Souza R, Forkert N. Improved multi-site Parkinson’s disease classification using neuroimaging data with counterfactual inference. In: Medical Imaging with Deep Learning. PMLR; 2024. p. 1304–17.

He K, Gan C, Li Z, Rekik I, Yin Z, Ji W, et al. Transformers in medical image analysis. Intell Med. 2023;3(1):59–78. https://doi.org/10.1016/j.imed.2022.07.002 .

Torbunov D, Huang Y, Yu H, Huang J, Yoo S, Lin M, et al. Uvcgan: UNET vision transformer cycle-consistent GAN for unpaired image-to-image translation. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2023; 702–712.

Yao X, Lou A, Li H, Hu D, Lu D, Liu H, Wang J, Stoebner Z, Johnson H, Long JD, Paulsen JS. Novel application of the attention mechanism on medical image harmonization. Med Imaging. 2023;12464:184–94. https://doi.org/10.1117/12.2654392 .

Hu X, Zhou X, Huang Q, Shi Z, Sun L, Li Q. Qs-attn: Query-selected attention for contrastive learning in i2i translation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022; 18291–18300.

Han D, Yu R, Li S, Wang J, Yang Y, Zhao Z, Wei Y, Cong S. MR Image harmonization with transformer. In 2023 IEEE International Conference on Mechatronics and Automation (ICMA). 2023; 2448–2453.

Dinsdale NK, Jenkinson M, Namburete AIL. Deep learning-based unlearning of dataset bias for MRI harmonisation and confound removal. Neuroimage. 2021. https://doi.org/10.1016/j.neuroimage.2020.117689 .

Guan H, Liu Y, Yang E, Yap PT, Shen D, Liu M. Multi-site MRI harmonization via attention-guided deep domain adaptation for brain disorder identification. Med Image Anal. 2021;71:102076. https://doi.org/10.1016/j.media.2021.102076 .

Wolleb J, Sandkühler R, Bieder F, Barakovic M, Hadjikhani N, Papadopoulou A, et al. Learn to ignore: domain adaptation for multi-site MRI analysis. International Conference on Medical Image Computing and Computer-Assisted Intervention. 2022; 725–735.

Dewey BE, Zuo L, Carass A, He Y, Liu Y, Mowry EM, et al. A disentangled latent space for cross-site MRI harmonization. International Conference on Medical Image Computing and Computer-Assisted Intervention. 2020; 720–729.

Zuo L, Dewey BE, Carass A, Liu Y, He Y, Calabresi PA, et al. Information-based disentangled representation learning for unsupervised MR harmonization. International Conference on Information Processing in Medical Imaging. 2021; 346–359.

Zuo L, Liu Y, Xue Y, Han S, Bilgel M, Resnick SM, et al. Disentangling a single MR modality. MICCAI Workshop on Data Augmentation, Labelling, and Imperfections. Cham: Springer Nature Switzerland; 2022. p. 54–63.

Book Google Scholar

Liu S, Yap P-T. Learning multi-site harmonization of magnetic resonance images without traveling human phantoms. Commun Eng. 2024;3(1):6.

Li H, Gopal S, Sekuboyina A, Zhang J, Niu C, Pirkl C, et al. Unpaired MR image homogenisation by disentangled representations and its uncertainty. Uncertainty for safe utilization of machine learning in medical imaging, and perinatal imaging, placental and preterm image analysis: 3rd international workshop. Cham: Springer International Publishing; 2021. p. 44–53.

Zhao F, Wu Z, Zhu D, Liu T, Gilmore J, Lin W, Wang L, Li G. Disentangling Site Effects with Cycle-Consistent Adversarial Autoencoder for Multi-site Cortical Data Harmonization. International Conference on Medical Image Computing and Computer-Assisted Intervention. 2023; 369–379

Wu M, Zhang L, Yap PT, Lin W, Zhu H, Liu M. Structural MRI harmonization via disentangled latent energy-based style translation. In: Cao X, Xuanang X, Rekik I, Cui Z, Ouyang X, editors. International Workshop on machine learning in medical imaging. Cham: Springer Nature Switzerland; 2023. p. 1–11.

Zuo L, Liu Y, Xue Y, Dewey BE, Bilgel M, Mowry EM, et al. HACA3: a unified approach for multi-site MR image harmonization. Comput Med Imaging Graphics. 2023;109:102285. https://doi.org/10.1016/j.compmedimag.2023.102285 .

Wang X, Chen H, Tang SA, Wu Z, Zhu W. Disentangled representation learning. ArXiv. 2022. https://doi.org/10.48550/arXiv.2211.11695 .

Jog A, Carass A, Roy S, Pham DL, Prince JL. Random forest regression for magnetic resonance image synthesis. Med Image Anal. 2017;35:475–88. https://doi.org/10.1016/j.media.2016.08.009 .

Chang X, Cai X, Dan Y, Song Y, Lu Q, Yang G, et al. Self-supervised learning for multi-center magnetic resonance imaging harmonization without traveling phantoms. Phys Med Biol. 2022;67(14):145004. https://doi.org/10.1088/1361-6560/ac7b66 .

Yurt M, Dalmaz O, Dar S, Ozbey M, Tınaz B, Oguz K, Çukur T. Semi-supervised learning of MRI synthesis without fully-sampled ground truths. IEEE Trans Med Imaging. 2022;41(12):3895–906.

Grigorescu I, Vanes L, Uus A, Batalle D, Cordero-Grande L, Nosarti C, et al. Harmonized segmentation of neonatal brain MRI. Front Neurosci. 2021;15:662005. https://doi.org/10.3389/fnins.2021.662005 .

Tor-Diez C, Porras AR, Packer RJ, Avery RA, Linguraru MG. Unsupervised MRI homogenization: application to pediatric anterior visual pathway segmentation. Machine learning in medical imaging: MICCAI. 2020. 180–188.

An L, Chen J, Chen P, Zhang C, He T, Chen C, Zhou JH, Yeo BT. Of Aging LS, Alzheimer’s disease neuroimaging initiative. Goal-specific brain MRI harmonization. Neuroimage. 2022;263:119570. https://doi.org/10.1016/j.neuroimage.2022.119570 .

Jin C-B, Kim H, Liu M, Jung W, Joo S, Park E, et al. Deep CT to MR synthesis using paired and unpaired data. Sensors. 2019;19(10):2361. https://doi.org/10.3390/s19102361 .

Wang T, Lei Y, Fu Y, Wynne JF, Curran WJ, Liu T, et al. A review on medical imaging synthesis using deep learning and its clinical applications. J Appl Clin Med Phys. 2021;22(1):11–36. https://doi.org/10.1002/acm2.13121 .

Pan Y, Liu M, Lian C, Zhou T, Xia Y, Shen D. Synthesizing missing PET from MRI with cycle-consistent generative adversarial networks for Alzheimer’s disease diagnosis. Med Image Comput Comput Assist Intervent MICCAI. 2018. https://doi.org/10.1007/978-3-030-00931-1_52 .

Zhang J, Cui Z, Jiang C, Zhang J, Gao F, Shen D. Mapping in cycles: dual-domain PET-CT synthesis framework with cycle-consistent constraints. International Conference on Medical Image Computing and Computer-Assisted Intervention. 2022; 758–767.

Sharma R, Tsiamyrtzis P, Webb AG, Leiss EL, Tsekos NV. Learning to deep learning: statistics and a paradigm test in selecting a UNet architecture to enhance MRI. Magn Reson Mater Phys Biol Med. 2023;21:1–22.

Sharma R, Tsiamyrtzis P, Webb AG, Seimenis I, Loukas C, Leiss E, Tsekos NV. A deep learning approach to upscaling “low-quality” MR Images: an in silico comparison study based on the UNet framework. Appl Sci. 2022;12(22):11758.

Atanda OG, Ismaila W, Afolabi AO, Awodoye OA, Falohun AS, Oguntoye JP. Statistical Analysis of a deep learning based trimodal biometric system using paired sampling T-Test. International Conference on Science, Engineering and Business for Sustainable Development Goals (SEB-SDG). IEEE. 2023;1:1–10.

Akter S, Shamrat FJ, Chakraborty S, Karim A, Azam S. COVID-19 detection using deep learning algorithm on chest X-ray images. Biology. 2021;10(11):1174.

Azad B, Azad R, Eskandari S, Bozorgpour A, Kazerouni A, Rekik I, Merhof D. Foundational models in medical imaging: a comprehensive survey and future vision. ArXiv. 2023. https://doi.org/10.48550/arXiv.2310.18689 .

Download references

Acknowledgements

This work was funded by the National Institutes of Health (5R01NS128486-03).

Author information

Authors and affiliations.

Department of Computer Engineering, Yazd University, Yazd, Iran

Soolmaz Abbasi

Department of Neurology, University of Southern California, Los Angeles, CA, USA

Haoyu Lan & Jeiran Choupan

Department of Radiology, University of Southern California, Los Angeles, CA, USA

Nasim Sheikh-Bahaei & Bino Varghese

Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY, USA

Gaurav Pandey

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bino Varghese .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Abbasi, S., Lan, H., Choupan, J. et al. Deep learning for the harmonization of structural MRI scans: a survey. BioMed Eng OnLine 23 , 90 (2024). https://doi.org/10.1186/s12938-024-01280-6

Download citation

Received : 10 May 2024

Accepted : 06 August 2024

Published : 31 August 2024

DOI : https://doi.org/10.1186/s12938-024-01280-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Harmonization
Structural MRI
Generative adversarial networks
Variational autoencoders
Disentangled representation learning

BioMedical Engineering OnLine

ISSN: 1475-925X

Submission enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
Front Public Health
PMC10662291

Medical image analysis using deep learning algorithms

Mengfang li.

1 The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China

Yuanyuan Jiang

2 Department of Cardiovascular Medicine, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China

Yanzhou Zhang

Haisheng zhu.

3 Department of Cardiovascular Medicine, Wencheng People’s Hospital, Wencheng, China

Associated Data

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

In the field of medical image analysis within deep learning (DL), the importance of employing advanced DL techniques cannot be overstated. DL has achieved impressive results in various areas, making it particularly noteworthy for medical image analysis in healthcare. The integration of DL with medical image analysis enables real-time analysis of vast and intricate datasets, yielding insights that significantly enhance healthcare outcomes and operational efficiency in the industry. This extensive review of existing literature conducts a thorough examination of the most recent deep learning (DL) approaches designed to address the difficulties faced in medical healthcare, particularly focusing on the use of deep learning algorithms in medical image analysis. Falling all the investigated papers into five different categories in terms of their techniques, we have assessed them according to some critical parameters. Through a systematic categorization of state-of-the-art DL techniques, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), Long Short-term Memory (LSTM) models, and hybrid models, this study explores their underlying principles, advantages, limitations, methodologies, simulation environments, and datasets. Based on our results, Python was the most frequent programming language used for implementing the proposed methods in the investigated papers. Notably, the majority of the scrutinized papers were published in 2021, underscoring the contemporaneous nature of the research. Moreover, this review accentuates the forefront advancements in DL techniques and their practical applications within the realm of medical image analysis, while simultaneously addressing the challenges that hinder the widespread implementation of DL in image analysis within the medical healthcare domains. These discerned insights serve as compelling impetuses for future studies aimed at the progressive advancement of image analysis in medical healthcare research. The evaluation metrics employed across the reviewed articles encompass a broad spectrum of features, encompassing accuracy, sensitivity, specificity, F-score, robustness, computational complexity, and generalizability.

1. Introduction

Deep learning is a branch of machine learning that employs artificial neural networks comprising multiple layers to acquire and discern intricate patterns from extensive datasets ( 1 , 2 ). It has brought about a revolution in various domains, including computer vision, natural language processing, and speech recognition, among other areas ( 3 ). One of the primary advantages of deep learning is its capacity to automatically learn features from raw data, thereby eliminating the necessity for manual feature engineering ( 4 ). This makes it especially powerful in domains with large, complex datasets, where traditional machine learning methods may struggle to capture the underlying patterns ( 5 ). Deep learning has also facilitated significant advancements in various tasks, including but not limited to image and speech recognition, comprehension of natural language, and the development of autonomous driving capabilities ( 6 ). For instance, deep learning has enabled the creation of exceptionally precise computer vision systems capable of identifying objects in images and videos with unparalleled precision. Likewise, deep learning has brought about substantial enhancements in natural language processing, leading to the development of models capable of comprehending and generating language that resembles human-like expression ( 7 ). Overall, deep learning has opened up new opportunities for solving complex problems and has the potential to transform many industries, including healthcare, finance, transportation, and more.

Medical image analysis is a field of study that involves the processing, interpretation, and analysis of medical images ( 8 ). The emergence of deep learning algorithms has prompted a notable transformation in the field of medical image analysis, as they have increasingly been employed to enhance the diagnosis, treatment, and monitoring of diverse medical conditions in recent years ( 9 ). Deep learning, as a branch of machine learning, encompasses the training of algorithms to acquire knowledge from vast quantities of data. When applied to medical image analysis, deep learning algorithms possess the capability to automatically identify and categorize anomalies in various medical images, including X-rays, MRI scans, CT scans, and ultrasound images ( 10 ). These algorithms can undergo training using extensive datasets consisting of annotated medical images, where each image is accompanied by labels indicating the corresponding medical condition or abnormality ( 11 ). Once trained, the algorithm can analyze new medical images and provide diagnostic insights to healthcare professionals. The application of deep learning algorithms in medical image analysis has exhibited promising outcomes, as evidenced by studies showcasing high levels of accuracy in detecting and diagnosing a wide range of medical conditions ( 12 ). This has led to the development of various commercial and open-source software tools that leverage deep learning algorithms for medical image analysis ( 13 ). Overall, the utilization of deep learning algorithms in medical image analysis has the capability to bring about substantial enhancements in healthcare results and transform the utilization of medical imaging in diagnosis and treatment.

Medical image processing is an area of research that encompasses the creation and application of algorithms and methods to analyze and decipher medical images ( 14 ). The primary objective of medical image processing is to extract meaningful information from medical images to aid in diagnosis, treatment planning, and therapeutic interventions ( 15 ). Medical image processing involves various tasks such as image segmentation, image registration, feature extraction, classification, and visualization. The primary aim of medical image processing is to extract pertinent information from medical images, facilitating the tasks of diagnosis, treatment planning, and therapeutic interventions. Each modality has its unique strengths and limitations, and the images produced by different modalities may require specific processing techniques to extract useful information ( 16 ). Medical image processing techniques have revolutionized the field of medicine by providing a non-invasive means to visualize and analyze the internal structures and functions of the body. It has enabled early detection and diagnosis of diseases, accurate treatment planning, and monitoring of treatment response. The use of medical image processing has significantly improved patient outcomes, reduced treatment costs, and enhanced the quality of care provided to patients. Visual depictions of CNNs in the context of medical image analysis using DL algorithms portray a layered architecture, where initial layers capture rudimentary features like edges and textures, while subsequent layers progressively discern more intricate and abstract characteristics, allowing the network to autonomously extract pertinent information from medical images for tasks like detection, segmentation, and classification. Additionally, the visual representations of RNNs in medical image analysis involving DL algorithms illustrate a network structure adept at grasping temporal relationships and sequential patterns within images, rendering them well-suited for tasks such as video analysis or the processing of time-series medical image data. Furthermore, visual representations of GANs in medical image analysis employing DL algorithms exemplify a dual-network framework: one network, the generator, fabricates synthetic medical images, while the other, the discriminator, assesses their authenticity, facilitating the generation of lifelike images closely resembling actual medical data. Moreover, visual depictions of LSTM networks in medical image analysis with DL algorithms delineate a specialized form of recurrent neural network proficient in processing sequential medical image data by preserving long-term dependencies and learning temporal patterns crucial for tasks like video analysis and time-series image processing. Finally, visual representations of hybrid methods in medical image analysis using DL algorithms portray a combination of diverse neural network architectures, often integrating CNNs with RNNs or other specialized modules, enabling the model to harness both spatial and temporal information for a comprehensive analysis of medical images.

Case studies and real-world examples provide tangible evidence of the effectiveness and applicability of DL algorithms in various medical image analysis tasks. They underscore the potential of this technology to revolutionize healthcare by improving diagnostic accuracy, reducing manual labor, and enabling earlier interventions for patients. Here are several examples of case studies and real-worlds applications:

Case Study: In Vijayalakshmi ( 17 ), a DL algorithm was trained to identify skin cancer from images of skin lesions. The algorithm demonstrated accuracy comparable to that of dermatologists, highlighting its potential as a tool for early skin cancer detection.

Case Study: also, De Fauw et al. ( 18 ) in Moorfields Eye Hospital, developed a DL system capable of identifying diabetic retinopathy from retinal images. The system was trained on a dataset of over 128,000 images and achieved a level of accuracy comparable to expert ophthalmologists.

Case Study: A study conducted by Guo et al. ( 8 ) at Massachusetts General Hospital utilized DL techniques to automate the segmentation of brain tumors from MRI scans. The algorithm significantly reduced the time required for tumor delineation, enabling quicker treatment planning for patients.

Case Study: The National Institutes of Health (NIH) released a dataset of chest X-ray images for the detection of tuberculosis. Researchers have successfully applied deep learning algorithms to this dataset, achieving high accuracy in identifying TB-related abnormalities.

Case Study: Meena and Roy ( 19 ) at Stanford University developed a deep learning model capable of detecting bone fractures in X-ray images. The model demonstrated high accuracy and outperformed traditional rule-based systems in fracture detection.

Within the realm of medical image analysis utilizing DL algorithms, ML algorithms are extensively utilized for precise and efficient segmentation tasks. DL approaches, especially convolutional neural networks (CNNs) and recurrent neural networks (RNNs), have demonstrated exceptional proficiency in capturing and leveraging spatial dependencies and symmetrical properties inherent in medical images. These algorithms enable the analyzing medical image of symmetric structures, such as organs or limbs, by leveraging their inherent symmetrical patterns. The utilization of DL mechanisms in medical image analysis encompasses various practical approaches, including generative adversarial networks (GANs), hybrid models, and combinations of CNNs and RNNs. The objective of this research is to offer a thorough examination of the uses of DL techniques in the domain of deep symmetry-based image analysis within medical healthcare, providing a comprehensive overview. By conducting an in-depth systematic literature review (SLR), analyzing multiple studies, and exploring the properties, advantages, limitations, datasets, and simulation environments associated with different DL mechanisms, this study enhances comprehension regarding the present state and future pathways for advancing and refining deep symmetry-based image analysis methodologies in the field of medical healthcare. The article is structured in the following manner: The key principles and terminology of ML/DL in medical image analysis are covered in the first part, followed by an investigation of relevant papers in part 3. Part 4 discusses the studied mechanisms and tools for paper selection, while part 5 illustrates the classification that was selected. Section 6 presents the results and comparisons, and the remaining concerns and conclusion are explored in the last section.

2. Fundamental concepts and terminology

The concepts and terms related to medical image analysis using DL algorithms that are covered in this section are essential for understanding the underlying principles and techniques used in medical image analysis.

2.1. The role of image analysis in medical healthcare

The utilization of deep learning algorithms for image analysis has brought about a revolution in medical healthcare by facilitating advanced and automated analysis of medical images ( 20 ). Deep learning methods, including Convolutional Neural Networks (CNNs), have showcased outstanding proficiency in tasks like image segmentation, feature extraction, and classification, exhibiting remarkable performance ( 21 ). By leveraging large amounts of annotated data, deep learning models can learn intricate patterns and relationships within medical images, facilitating accurate detection, localization, and diagnosis of diseases and abnormalities. Deep learning-based image analysis allows for faster and more precise interpretation of medical images, leading to improved patient outcomes, personalized treatment planning, and efficient healthcare workflows ( 22 ). Furthermore, these algorithms have the potential to assist in early disease detection, assist radiologists in decision-making, and enhance medical research through the analysis of large-scale image datasets. Overall, deep learning-based image analysis is transforming medical healthcare by providing powerful tools for image interpretation, augmenting the capabilities of healthcare professionals, and enhancing patient care ( 23 ).

2.2. Medical image analysis application

The utilization of deep learning algorithms in medical image analysis has discovered numerous applications within the healthcare sector. Deep learning techniques, notably Convolutional Neural Networks (CNNs), have been widely employed for tasks encompassing image segmentation, object detection, disease classification, and image reconstruction ( 24 ). In medical image analysis, these algorithms can assist in the detection and diagnosis of various conditions, such as tumors, lesions, anatomical abnormalities, and pathological changes. They can also aid in the evaluation of disease progression, treatment response, and prognosis. Deep learning models can automatically extract meaningful features from medical images, enabling efficient and accurate interpretation ( 25 ). The application of this technology holds promise for elevating clinical decision-making, ameliorating patient outcomes, and optimizing resource allocation in healthcare settings. Moreover, deep learning algorithms can be employed for data augmentation, image registration, and multimodal fusion, facilitating a comprehensive and integrated analysis of medical images obtained from various modalities. With continuous advancements in deep learning algorithms, medical image analysis is witnessing significant progress, opening up new possibilities for precision medicine, personalized treatment planning, and advanced healthcare solutions ( 26 ).

2.3. Various aspects of medical image analysis for the healthcare section

Medical image analysis encompasses various crucial aspects in the healthcare sector, enabling in-depth examination and diagnosis based on medical imaging data ( 27 ). Image preprocessing constitutes a crucial element, encompassing techniques like noise reduction, image enhancement, and normalization, aimed at enhancing the quality and uniformity of the images. Another essential aspect is image registration, which aligns multiple images of the same patient or acquired through different imaging modalities, enabling precise comparison and fusion of information ( 28 ). Feature extraction is another crucial step, where relevant characteristics and patterns are extracted from the images, aiding in the detection and classification of abnormalities or specific anatomical structures. Segmentation plays a vital role in delineating regions of interest, enabling precise localization and measurement of anatomical structures, tumors, or lesions ( 29 ). Finally, classification and recognition techniques are applied to differentiate normal and abnormal regions, aiding in disease diagnosis and treatment planning. Deep learning algorithms, notably Convolutional Neural Networks (CNNs), have exhibited extraordinary achievements in diverse facets of medical image analysis by acquiring complex patterns and representations from extensive datasets of medical imaging ( 30 ). However, challenges such as data variability, interpretability, and generalization across different patient populations and imaging modalities need to be addressed to ensure reliable and effective medical image analysis in healthcare applications.

3. Relevant reviews

We are going to look into some recent research on medical image analysis using DL algorithms in this part. The initial purpose is to properly make a distinction between the current study’s significant results in comparison with what is discussed in this paper. Due to advancements in AI technology, there is a growing adoption of AI mechanisms in medical image analysis. Simultaneously, academia has shown a heightened interest in addressing challenges related to medical image analysis. Furthermore, medical image analysis is a hierarchical network management framework modeled to direct analysis availability to aim medical healthcare. In this regard, Gupta and Katarya ( 31 ) provided a comprehensive review of the literature on social media-based surveillance systems for healthcare using machine learning. The authors analyzed 50 studies published between 2011 and 2021, covering a wide range of topics related to social media monitoring for healthcare, including disease outbreaks, adverse drug reactions, mental health, and vaccine hesitancy. The review highlighted the potential of machine learning algorithms for analyzing vast amounts of social media data and identifying relevant health-related information. The authors also identified several challenges associated with the use of social media data, such as data quality and privacy concerns, and discuss potential solutions to address these challenges. The authors noted that social media-based surveillance systems can complement traditional surveillance methods by providing real-time data on health-related events and trends. They also suggested that machine learning algorithms can improve the accuracy and efficiency of social media monitoring by automatically filtering out irrelevant information and identifying patterns and trends in the data. The review highlighted the importance of data pre-processing and feature selection in developing effective machine learning models for social media analysis.

As well, Kourou et al. ( 32 ) reviewed machine learning (ML) applications for cancer prognosis and prediction. The authors started by describing the challenges of cancer treatment, highlighting the importance of personalized medicine and the role of ML algorithms in enabling it. The paper then provided an overview of different types of ML algorithms, including supervised and unsupervised learning, and discussed their potential applications in cancer prognosis and prediction. The authors presented examples of studies that have used ML algorithms for diagnosis, treatment response prediction, and survival prediction across different types of cancer. They also discussed the use of multiple data sources for ML algorithms, such as genetic data, imaging data, and clinical data. The paper concluded by addressing the challenges and limitations encountered in using ML algorithms for cancer prognosis and prediction, which include concerns regarding data quality, overfitting, and interpretability. The authors proposed that ML algorithms hold significant potential for enhancing cancer treatment outcomes. However, they emphasized the necessity for further research to optimize their application and tackle the associated challenges in this domain.

Moreover, Razzak et al. ( 33 ) provided a comprehensive overview of the use of deep learning in medical image processing. The authors deliberated on the potential of deep learning algorithms in diverse medical imaging tasks, encompassing image classification, segmentation, registration, and synthesis. They emphasized the challenges encountered when employing deep learning, such as the requirement for extensive annotated datasets, interpretability of deep models, and computational demands. Additionally, the paper delved into prospective avenues in the field, including the integration of multi-modal data, transfer learning, and the utilization of generative models. In summary, the paper offered valuable perspectives on the present status, challenges, and potential advancements of deep learning in the domain of medical image processing.

In addition, Litjens et al. ( 34 ) provided a comprehensive survey of the applications of deep learning in medical image analysis. A thorough introduction of the deep learning approaches used in each of these areas is provided by the authors as they look at a variety of tasks in medical imaging, including picture classification, segmentation, detection, registration, and creation. Additionally, they look at the difficulties and restrictions of using deep learning algorithms for medical image analysis, such as the need for sizable datasets with annotations and the interpretability of deep models. The growth of explainable and interpretable deep learning models is highlighted in the paper’s conclusion along with other potential future possibilities in the area, such as the integration of multimodal data. In summary, this survey serves as a valuable resource for researchers and practitioners, offering insights into the current state and future prospects of deep learning in the context of medical image analysis.

Additionally, Bzdok and Ioannidis ( 35 ) discussed the importance of exploration, inference, and prediction in the fields of neuroscience and biomedicine. The author highlighted the importance of integrating diverse data types, such as neuroimaging, genetics, and behavioral data, in order to achieve a comprehensive comprehension of intricate systems. Bzdok also delved into the role of machine learning in facilitating the identification of patterns and making predictions based on extensive datasets. The author provided an account of several specific applications of machine learning in neuroscience and biomedicine, including forecasting disease progression and treatment response, analyzing brain connectivity networks, and identifying biomarkers for disease diagnosis. The paper concluded by discussing the challenges and limitations encountered when employing machine learning in these domains, while emphasizing the essentiality of carefully considering the ethical and social implications of these technologies. Moreover, the paper underscored the potential of machine learning to transform our understanding of complex biological systems and enhance medical outcomes. Table 1 depicts summary of related works.

Summary of related works.

Author	Main idea	Advantage	Disadvantage
Gupta and Katarya ( )	Providing a comprehensive review of the literature on social media-based surveillance systems for healthcare using machine learning
Kourou, et al. ( )	Reviewing ML applications for cancer prognosis and prediction
Razzak, et al. ( )	Providing a comprehensive overview of the use of deep learning in medical image processing
Litjens, et al. ( )	Providing a comprehensive survey of the applications of deep learning in medical image analysis
Bzdok and Ioannidis ( )	Discussing the importance of exploration, inference, and prediction in the fields of neuroscience and biomedicine
Our work	Introducing a new taxonomy of DL methods in medical image analysis

4. Methodology of research

We thoroughly examined pertinent documents that partially explored the utilization of DL methods in medical image analysis. By utilizing the Systematic Literature Review (SLR) methodology, this section comprehensively encompasses the field of medical image analysis. The SLR technique encompasses a thorough evaluation of all research conducted on a significant topic. This section concludes with an extensive investigation of ML techniques in the realm of medical image analysis. Furthermore, the reliability of the research selection methods is scrutinized. In the subsequent subsections, we have provided supplementary information concerning research techniques, encompassing the selection metrics and research inquiries.

4.1. Formalization of question

The primary aims of the research are to identify, assess, and differentiate all key papers within the realm of using DL methods medical image analysis. A systematic literature review (SLR) can be utilized to scrutinize the constituents and characteristics of methods for accomplishing the aforementioned objectives. Furthermore, an SLR facilitates the acquisition of profound comprehension of the pivotal challenges and difficulties in this domain. The following paragraph outlines several research inquiries:

Research Question 1: In what manners can DL techniques in the field of medical image analysis be categorized? The answer to this question can be found in Part 5.

Research Question 2: What types of techniques do scholars employ to execute their investigation? Parts 5.1 to 5.7 elucidate this query.

Research Question 3: Which parameters attracted the most attention in the papers? What are the most popular DL applications utilized in medical image analysis? The answer to this question is included in Part 6.

Research Question 4: What unexplored prospects exist in this area? Part 7 proffers the answer to this question.

4.2. The procedure of paper exploration

The present investigation’s pursuit and selection methodologies are classified into four distinct phases, as depicted in Figure 1 . In the initial phase, a comprehensive list of keywords and phrases was utilized to scour various sources, as demonstrated in Table 2 . An electronic database was employed to retrieve relevant documents, including Chapters, Journals, technical studies, conference papers, notes, and special issues, resulting in a total of 616 papers as is shown if Figure 2 . These papers were then subjected to an exhaustive analysis based on a set of predetermined standards, and only those meeting the stipulated criteria, illustrated in Figure 3 , were selected for further evaluation. The distribution of publishers in this initial phase is shown in Figure 4 , and the number of articles left after the first phase was 481.

An external file that holds a picture, illustration, etc.
Object name is fpubh-11-1273253-g001.jpg

The phases of the article searching and selection process.

Keywords and search criteria.

S#	Keywords and search criteria	S#	Keywords and search criteria
S1	“DL” and “Medical”	S6	“AI” and “Healthcare”
S2	“ML” and “Healthcare”	S7	“Healthcare” and “DL algorithms”
S3	“DL” and “Image Analysis”	S8	“DL methods” and “Medical Images”
S4	“ML” and “Medical Healthcare”	S9	“Image Analysis” and “Medical Healthcare”
S5	“AI” and “Medical Healthcare”	S10	“AI methods” and “Medical Images”

An external file that holds a picture, illustration, etc.
Object name is fpubh-11-1273253-g002.jpg

Frequency of publications of studied paper in first stage of paper selection.

An external file that holds a picture, illustration, etc.
Object name is fpubh-11-1273253-g003.jpg

Criteria for inclusion in the paper selection process.

An external file that holds a picture, illustration, etc.
Object name is fpubh-11-1273253-g004.jpg

Frequency of publications of studied paper in second stage of paper selection.

In the subsequent phase, a thorough review of the selected papers’ titles and abstracts was conducted, focusing on the papers’ discussion, methodology, analysis, and conclusion to ensure their relevance to the study. As demonstrated in Figure 5 , only 227 papers were retained after this step and 105 papers were further.

An external file that holds a picture, illustration, etc.
Object name is fpubh-11-1273253-g005.jpg

Frequency of publications of studied paper in third stage of paper selection.

chosen for a more comprehensive review, as illustrated in Figure 6 , with the ultimate aim of selecting papers that adhered to the study’s predetermined metrics. Finally, after careful consideration, 25 articles were hand-picked to investigate other publications.

An external file that holds a picture, illustration, etc.
Object name is fpubh-11-1273253-g006.jpg

Frequency of publications of studied paper in forth stage of paper selection.

5. ML/DL techniques for medical image analysis

In this section, we delve into the implementation of DL methods in the medical healthcare image analysis field. A total of 25 articles satisfying our selection criteria will be presented herein. Initially, we categorize the techniques into 5 primary groups comprising CNNs, RNNs, GANs, LSTMs, and hybrid methodologies encompassing diverse methods. The proposed taxonomy of DL-associated medical image analysis in medical healthcare is depicted in Figure 7 .

An external file that holds a picture, illustration, etc.
Object name is fpubh-11-1273253-g007.jpg

The proposed taxonomy of Bioinformatics.

5.1. Convolutional neural network techniques for medical image analysis

When using deep learning approaches for medical image processing, convolutional neural networks (CNNs) play a significant role. They perform well in tasks like object localization, segmentation, and classification due to their capacity to automatically extract pertinent characteristics from intricate medical pictures. CNNs are able to accurately identify anomalies, diagnose tumors, and segment organs in medical pictures by capturing complex patterns and structures. Important characteristics may be learnt at various levels by utilizing the hierarchical structure of CNNs, which improves analysis and diagnosis. Employing CNNs in medical image analysis has notably improved the precision, effectiveness, and automation of diagnostic procedures, ultimately leading to advantageous patient care and treatment results.

In this regard, Singh et al. ( 36 ) highlighted the role of artificial intelligence (AI) and machine learning (ML) techniques in advancing biomedical material design and predicting their toxicity. The authors emphasized the need for efficient and safe materials for medical applications and how computational methods can aid in this process. The paper explored diverse categories of AI and ML algorithms, including random forests, decision trees, and support vector machines, which can be employed for predicting toxicity. The authors provided a case study wherein they utilized a random forest algorithm to predict the toxicity of carbon nanotubes. They also highlighted the importance of data quality and quantity for accurate predictions, as well as the need for interpretability and transparency of AI/ML models. The paper concluded by discussing future research directions in this area, including the integration of multi-omics data, network analysis, and deep learning techniques. This paper demonstrated the potential of AI/ML in advancing biomedical material design and reducing the need for animal testing.

Also, Jena et al. ( 37 ) investigated the impact of parameters on the performance of deep learning models for the classification of diabetic retinopathy (DR) in a smart healthcare system. Using retinal fundus pictures, the scientists developed a convolutional neural network (CNN) architecture with two branches to categorize diabetic retinopathy (DR). A branch for feature extraction and another for classification are both included in the suggested model. A pre-trained model is used in the feature extraction branch to extract pertinent characteristics from the input picture, and the classification branch uses these features to predict the severity of DR. The learning rate, number of epochs, batch size, and optimizer were among the variables that the authors experimented with in order to evaluate the model’s performance. The outcomes showed that the suggested model, when using the ideal parameter configuration, had an accuracy of 98.12%. The authors also suggested a secure IoT-based blockchain-based smart healthcare system for processing and storing medical data. The proposed system could be used for the early diagnosis and treatment of DR, thereby improving patient outcomes.

As well, Thilagam et al. ( 38 ) presented a secure Internet of Things (IoT) healthcare architecture with a deep learning-based access control system. The proposed system is designed to ensure that only authorized personnel can access the sensitive medical information stored in IoT devices. The authors used deep learning algorithms to develop a robust access control system that can identify and authenticate users with high accuracy. The system also included an encryption layer to ensure that all data transmitted between devices is secure. The authors assessed the proposed architecture through a prototype implementation, which revealed that the system can securely access medical data in real-time. Additionally, the authors conducted a comparison with existing solutions and demonstrated that their approach outperforms others in terms of accuracy, security, and scalability. The paper underscored the potential of employing deep learning algorithms in healthcare systems to enhance security and privacy, while facilitating real-time access to medical data.

Besides, Ismail et al. ( 39 ) proposed a CNN-based model for analyzing regular health factors in an IoMT (Internet-of-Medical-Things) environment. The model extracted feature from multiple health data sources, such as blood pressure, pulse rate, and body temperature, using CNN-based algorithms, which are then used to predict the risk of health issues. The proposed model is capable of classifying health data into five categories: normal, pre-hypertension, hypertension, pre-diabetes, and diabetes. The authors utilized a real-world dataset comprising health data from 50 individuals to train and evaluate the model. The findings indicated that the proposed model exhibited a remarkable level of accuracy and surpassed existing machine learning models in terms of both predictive accuracy and computational complexity. The authors expressed their confidence that the proposed model could contribute to the advancement of health monitoring systems, offering real-time monitoring and personalized interventions, thereby preventing health issues and enhancing patient outcomes.

And, More et al. ( 40 ) proposed a security-assured CNN-based model for the reconstruction of medical images on the Internet of Healthcare Things (IoHT) with the goal of ensuring the privacy and security of medical data. The proposed framework comprises two main components: a deep learning-based image reconstruction model and a security-enhanced encryption model. The image reconstruction model relies on a convolutional neural network (CNN) to accurately reconstruct original medical images from compressed versions. To safeguard the transmitted images, the encryption model employs a hybrid encryption scheme that combines symmetric and asymmetric techniques. Through evaluation using a widely recognized medical imaging dataset, the results demonstrated the model’s remarkable reconstruction accuracy and effective security performance. This study underscores the potential of leveraging deep learning models in healthcare, particularly within medical image processing, while emphasizing the crucial need for ensuring the security and privacy of medical data. Table 3 discusses the CNN methods used in medical image analysis and their properties.

The methods, properties, and features of CNN-medical image analysis mechanisms.

Author	Main idea	Simulation environment	Datasets
Singh, et al. ( )	Presenting a case study where they employed a random forest algorithm for toxicity prediction of carbon nanotubes	Python	27 observations
Jena, et al. ( )	Proposing a 2-branch convolutional neural network (CNN) architecture to classify DR in retinal fundus images	Python	Fundus images of 102 diabetic patients
Thilagam, et al. ( )	Presenting a secure Internet of Things (IoT) healthcare architecture with a deep learning-based access control system	Python	100 participants performing 10 different gestures and activities over a duration of 60 s each
Ismail, et al. ( )	Proposing a CNN-based model for analyzing regular health factors in an IoMT (Internet-of-Medical-Things) environment	Python	Real-time health examinations of 10,806 citizens
More, et al. ( )	Proposing a security-assured CNN-based model for the reconstruction of medical images on the Internet of Healthcare Things (IoHT)	Python	2,260 images of Ultrasound, CT scan, and MRI

5.2. Generative adversarial network techniques for medical image analysis

The importance of GAN methods in medical image analysis using deep learning algorithms lies in their ability to generate realistic synthetic images, augment datasets, and improve the accuracy and effectiveness of diagnosis and analysis for various medical conditions. By the same token, in Vaccari et al. ( 41 ) the authors proposed a generative adversarial network (GAN) technique to address the issue of generating synthetic medical data for Internet of Medical Things (IoMT) applications. The authors detailed the application of their proposed method for generating a wide range of medical data samples encompassing both time series and non-time series data. They emphasized the advantages of employing a Generative Adversarial Network (GAN)-based approach, such as the capacity to generate realistic data capable of enhancing the performance of Internet of Medical Things (IoMT) systems. Through experiments utilizing authentic medical datasets like electrocardiogram (ECG) data and healthcare imaging data, the authors validated the efficacy of their proposed technique. The results demonstrated that their GAN-based method successfully produced synthetic medical data that closely resembled real medical data, both visually and statistically, as indicated by various metrics. The authors concluded that their proposed technique has the potential to be a valuable tool for generating synthetic medical data for use in IoMT applications.

Toward accurate prediction of patient length of stay at emergency.

As well, Kadri et al. ( 42 ) presented a framework that utilizes a deep learning model to predict the length of stay of patients at emergency departments. The proposed model employed a GAN to generate synthetic training data and address the problem of insufficient training data. The model used multiple input modalities, including demographic information, chief complaint, triage information, vital signs, and lab results, to predict the length of stay of patients. The authors demonstrated that their proposed framework surpassed multiple baseline models, showcasing its exceptional performance in accurately predicting the length of stay for patients in emergency departments. They recommended the deployment of the proposed framework in real-world settings, anticipating its potential to enhance the efficiency of emergency departments and ultimately improve patient outcomes.

Yang et al. ( 43 ) proposed a novel semi-supervised learning approach using GAN for clinical decision support in Health-IoT platform. The proposed model generated new samples from existing labeled data, creating additional labeled data for training. The GAN-based model undergoes training on a vast unlabeled dataset to generate medical images that exhibit enhanced realism for subsequent training purposes. These generated samples are then employed to fine-tune the pre-trained CNN, resulting in an improved classification accuracy. To assess the effectiveness of the proposed model, three medical datasets are utilized, and the findings demonstrate that the GAN-based semi-supervised learning approach surpasses the supervised learning approach, yielding superior accuracy and reduced loss values. The paper concludes that the proposed model presents the potential to enhance the accuracy of clinical decision support systems by generating supplementary training data. Furthermore, the proposed approach can be extended to diverse healthcare applications, including disease diagnosis and drug discovery.

Huang et al. ( 44 ) proposed a deep learning-based model, DU-GAN, for low-dose computed tomography (CT) denoising in the medical imaging field. The architecture of DU-GAN incorporates dual-domain U-Net-based discriminators and a GAN, aiming to enhance denoising performance and generate high-quality CT images. The proposed approach adopts a dual-domain architecture, effectively utilizing both the image domain and transform domain to differentiate real images from generated ones. DU-GAN is trained on a substantial dataset of CT images to grasp the noise distribution and noise from low-dose CT images. The results indicate that the DU-GAN model surpasses existing methods in terms of both quantitative and qualitative evaluation metrics. Furthermore, the proposed model exhibits robustness across various noise levels and different types of image data. The study showed the potential of the proposed approach for practical application in the clinical diagnosis and treatment of various medical conditions.

Purandhar et al. ( 45 ) proposes the use of Generative Adversarial Networks (GAN) for classifying clustered health care data. This study’s GAN classifier contains both a discriminator network and a generator network. While the discriminator tells the difference between genuine and false samples, the generator learns the underlying data distribution. Utilizing data from Electronic Health Records (EHRs), the MIMIC-III dataset was used by the scientists in their research. The outcomes show that the GAN classifier accurately and successfully categorizes the medical problems of patients. The authors also demonstrated the superiority of their GAN classifier by contrasting it with conventional machine learning techniques. The suggested GAN-based strategy shows promise for illness early detection and diagnosis, with potential for bettering healthcare outcomes and lowering costs. Table 4 discusses the GAN methods used in medical image analysis.

The methods, properties, and features of GAN-medical image analysis mechanisms.

Author	Main idea	Simulation environment	Datasets
Vaccari, et al. ( )	Proposing a generative adversarial network (GAN) technique to address the issue of generating synthetic medical data for Internet of Medical Things (IoMT) applications	–	43 samples
Kadri, et al. ( )	Presenting a framework that utilizes a deep learning model to predict the length of stay of patients at emergency departments	Python	44,676 patients
Yang, et al. ( )	Proposing a novel semi-supervised learning approach using GAN for clinical decision support in Health-IoT platform	Python	11,039 Stroke patient
Huang, et al. ( )	Proposing a deep learning-based model, DU-GAN, for low-dose CT denoising	–	850 CT scans
Purandhar, et al. ( )	Proposing the use of GAN for classifying clustered health care data	–	452 instances

5.3. Recurrent neural network techniques for medical image analysis

Recurrent Neural Networks (RNNs) are essential in medical image analysis using deep learning algorithms due to their ability to capture temporal dependencies and contextual information. RNNs excel in tasks involving sequential or time-series data, such as analyzing medical image sequences or dynamic imaging modalities. Their capability to model long-term dependencies and utilize information from previous time steps enables the detection of patterns, disease progression prediction, and tracking tumor growth. RNN variants like LSTM and GRU further enhance their ability to capture complex temporal dynamics, making them vital in extracting meaningful insights from medical image sequences.

Sridhar et al. ( 46 ) proposed a novel approach for reducing the size of medical images while preserving their diagnostic quality. The authors introduced a two-stage framework that combines a Recurrent Neural Network (RNN) and a Genetic Particle Swarm Optimization with Weighted Vector Quantization (GenPSOWVQ). In the first stage, the RNN is employed to learn the spatial and contextual dependencies within the images, capturing important features for preserving diagnostic information. In the second stage, the GenPSOWVQ algorithm optimized the image compression process by selecting the best encoding parameters. The experimental results demonstrated the effectiveness of the proposed model in achieving significant image size reduction while maintaining high diagnostic accuracy. The combination of RNN and GenPSOWVQ enabled an efficient and reliable approach for medical image compression, which can have practical implications in storage, transmission, and analysis of large-scale medical image datasets.

Pham et al. ( 47 ) discussed the use of DL to predict healthcare trajectories from medical records. The authors argued that deep learning can be used to model the complex relationships between different medical conditions and predict how a patient’s healthcare journey might evolve over time. The study used data from electronic medical records of patients with various conditions, including diabetes, hypertension, and heart disease. The proposed DL model used a CNNs and RNNs to capture both the temporal and spatial relationships in the data. The research discovered that the deep learning model exhibited a remarkable ability to accurately forecast the future healthcare path of patients with a notable level of precision. The authors’ conclusion highlighted the potential of deep learning to transform healthcare delivery through enhanced accuracy in predictions and personalized care. Nevertheless, the authors acknowledged that the integration of deep learning in healthcare is still at an early phase, necessitating further investigation to fully unleash its potential.

Wang et al. ( 48 ) proposed a new approach for dynamic treatment recommendation using supervised reinforcement learning with RNNs. The authors aimed to address the challenge of making treatment decisions for patients with complex and dynamic health conditions by developing an algorithm that can adapt to changes in patient health over time. The proposed approach involved using an RNN to model patient health trajectories and predict the optimal treatment at each step. The training of the model involves a blend of supervised and reinforcement learning techniques, aimed at optimizing treatment decisions for long-term health benefits. The authors assessed the effectiveness of this approach using a dataset comprising actual patients with hypertension and demonstrated its superiority over conventional machine learning methods in terms of predictive accuracy. The suggested method holds promise in enhancing patient outcomes by offering personalized treatment recommendations that can adapt to variations in the patient’s health status.

Jagannatha and Yu ( 49 ) discusses the use of bidirectional recurrent neural networks (RNNs) for medical event detection in electronic health records (EHRs). Electronic Health Records (EHRs) offer valuable insights for medical research, yet analyzing them can be arduous due to the intricate nature and fluctuations in the data. To address this, the authors introduce a bidirectional RNN model capable of capturing the interdependencies in the sequential data of EHRs, encompassing both forward and backward relations. Through training on an EHR dataset and subsequent evaluation, the model’s proficiency in detecting medical events is assessed. The findings reveal that the bidirectional RNN surpasses conventional machine learning methods in terms of medical event detection. The authors also compare different variations of the model, such as using different types of RNNs and adding additional features to the input. Overall, the study demonstrates the potential of using bidirectional RNNs for medical event detection in EHRs, which could have important implications for improving healthcare outcomes and reducing costs.

Cocos et al. ( 50 ) focused on developing a deep learning model for pharmacovigilance to identify adverse drug reactions (ADRs) mentioned on social media platforms such as Twitter. In the study, Adverse Drug Reactions (ADRs) were trained and classified using two unique RNN architectures, namely Bidirectional Long-Short Term Memory (Bi-LSTM) and Gated Recurrent Unit (GRU). Various feature extraction methods were also looked at, and their individual performances were discussed. The outcomes showed that the Bi-LSTM model performed better than the GRU model, obtaining an F1-score of 0.86. A comparison of the deep learning models with conventional machine learning models was also done, confirming the higher performance of the deep learning models. The study focused on the possibilities of utilizing social media platforms for pharmacovigilance and underlined the efficiency of deep learning models in precisely detecting ADRs. Table 5 discusses the RNN methods used in medical image analysis.

The methods, properties, and features of RNN-medical image analysis mechanisms.

Author	Main idea	Simulation environment	Datasets
Sridhar, et al. ( )	Proposing a novel approach for reducing the size of medical images	–	50 instances
Pham, et al. ( )	Proposing DL model used a CNNs and RNNs to capture both the temporal and spatial relationships in the data	Python	7,191 patients
Wang, et al. ( )	Proposing a new approach for dynamic treatment recommendation	–	43 K patients
Jagannatha and Yu ( )	Discussing the use of bidirectional recurrent neural networks (RNNs) for medical event detection	Lasagne	780 English EHR notes
Cocos, et al. ( )	Developing a DL model for pharmacovigilance to identify adverse drug reactions (ADRs)	Keras	844 tweets

5.4. Long short-term memory techniques for medical image analysis

The importance of Long Short-Term Memory (LSTM) method in medical image analysis using deep learning algorithms lies in its ability to capture and model sequential dependencies within the image data. Medical images often contain complex spatial and temporal patterns that require understanding of contextual information. LSTM, as a type of recurrent neural network (RNN), excels in modeling long-range dependencies and capturing temporal dynamics, making it suitable for tasks such as time series analysis, disease progression modeling, and image sequence analysis. By leveraging the memory and gating mechanisms of LSTM, it can effectively learn and retain relevant information over time, enabling more accurate and robust analysis of medical image data and contributing to improved diagnostic accuracy and personalized treatment in healthcare applications.

Butt et al. ( 51 ) presented a ML-based approach for diabetes classification and prediction. They used a dataset of 768 patients and 8 clinical features, including age, BMI, blood pressure, and glucose levels. Three different machine learning techniques–logistic regression, decision tree, and k-nearest neighbors–were applied to the preprocessed data before each of these algorithms was used. Sorting patients into the diabetic or non-diabetic category was the goal. Metrics including accuracy, precision, recall, and F1 score were used to evaluate the effectiveness of each method. In order to forecast the patients’ blood glucose levels, a deep learning system, namely a feedforward neural network, was used. A comparison between the performance of the deep learning algorithm and that of the traditional machine learning algorithms was conducted, revealing that the deep learning algorithm surpassed the other algorithms in terms of prediction accuracy. The authors concluded that their approach can be used for early diagnosis and management of diabetes in healthcare applications.

Awais et al. ( 52 ) proposed an Internet of Things (IoT) framework that utilizes Long Short-Term Memory (LSTM) based emotion detection for healthcare and distance learning during COVID-19. The proposed framework offers the ability to discern individuals’ emotions by leveraging physiological signals such as electrocardiogram (ECG), electrodermal activity (EDA), and photoplethysmogram (PPG). Collected data undergoes preprocessing and feature extraction prior to training an LSTM model. To assess its effectiveness, the framework is tested using the PhysioNet emotion database, where the results demonstrate its accurate emotion detection capabilities, reaching an accuracy level of up to 94.5%. With its potential applications in healthcare and distance learning amid the COVID-19 pandemic, the framework proves invaluable for remotely monitoring individuals’ emotional states and providing necessary support and interventions. The paper highlighted the importance of using IoT and machine learning in healthcare, and how it can help to address some of the challenges posed by the pandemic.

Nancy et al. ( 53 ) proposed an IoT-Cloud-based smart healthcare monitoring system for heart disease prediction using deep learning. The technology uses wearable sensors to gather physiological signs from patients, then delivers those signals to a cloud server for analysis. By training on a sizable dataset of ECG signals, a Convolutional Neural Network (CNN)-based deep learning model is used to predict cardiac illness. Transfer learning techniques, especially fine-tuning, are used to optimize the model. The suggested system’s exceptional accuracy in forecasting cardiac illness has been rigorously tested on a real-world dataset. Additionally, the model exhibits the capability to detect the early onset of heart disease, facilitating timely intervention and treatment. The paper concluded that the proposed system can be an effective tool for real-time heart disease monitoring and prediction, which can help improve patient outcomes and reduce healthcare costs.

Queralta et al. ( 54 ) presents an Edge-AI solution for fall detection in health monitoring using LoRa communication technology, fog computing, and LSTM recurrent neural networks. The proposed system consists of a wearable device, a LoRa gateway, and an edge server that processes and analyzes sensor data locally, reducing the dependence on cloud services and improving real-time fall detection. The system employs a MobileNetV2 convolutional neural network to extract features from accelerometer and gyroscope data, followed by an LSTM network that predicts falls. The authors evaluated the performance of the proposed system using a dataset collected from volunteers and achieved a sensitivity of 93.14% and a specificity of 98.9%. They also compared the proposed system with a cloud-based solution, showing that the proposed system had lower latency and reduced data transmission requirements. Overall, the proposed Edge-AI system can provide a low-cost and efficient solution for fall detection in health monitoring applications.

Gao et al. ( 55 ) introduced a novel approach called Fully Convolutional Structured LSTM Networks (FCSLNs) for joint 4D medical image segmentation. The proposed approach utilized the strengths of fully convolutional networks and structured LSTM networks to overcome the complexities arising from spatial and temporal dependencies in 4D medical image data. By integrating LSTM units into the convolutional layers, the FCSLNs successfully capture temporal information and propagate it throughout the spatial dimensions. Empirical findings strongly indicate the outstanding performance of the FCSLNs when compared to existing methods, achieving precise and resilient segmentation of 4D medical images. The proposed framework demonstrates significant promise in advancing medical image analysis tasks and enhancing clinical decision-making processes. Table 6 discusses the LSTM methods used in medical image analysis.

The methods, properties, and features of LSTM-medical image analysis mechanisms.

Author	Main idea	Simulation environment	Datasets
Butt, et al. ( )	Presenting a ML-based approach for diabetes classification and prediction	–	768 records
Awais, et al. ( )	Proposing an Internet of Things (IoT) framework for healthcare and distance learning during COVID-19	Tensorflow	1,000 samples of data
Nancy, et al. ( )	Proposing an IoT-Cloud-based smart healthcare monitoring system for heart disease prediction using deep learning	Tensorflow	100,000 records
Queralta, et al. ( )	Proposing an IoT-Cloud-based smart healthcare monitoring system for heart disease prediction	Keras/Tensorflow	20 data points
Gao, et al. ( )	Introducing a novel approach called Fully Convolutional Structured LSTM Networks (FCSLNs) for joint 4D medical image segmentation	–	10 samples

5.5. Hybrid techniques for bio and medical informatics

Hybrid methods in medical image analysis, which combine deep learning algorithms with other techniques or data modalities, are of significant importance. Deep learning has demonstrated remarkable success in tasks like image segmentation and classification. However, it may face challenges such as limited training data or interpretability issues. By incorporating hybrid methods, researchers can overcome these limitations and achieve enhanced performance. Hybrid approaches can integrate traditional machine learning techniques, statistical models, or domain-specific knowledge to address data scarcity or improve interpretability. Additionally, combining multiple data modalities, such as medical images with textual reports or physiological signals, enables a more comprehensive understanding of the medical condition and facilitates better decision-making. Ultimately, hybrid methods in medical image analysis empower healthcare professionals with more accurate and reliable tools for diagnosis, treatment planning, and patient care. In this regard, Shahzadi et al. ( 56 ) proposed a novel cascaded framework for accurately classifying brain tumors using a combination of convolutional neural networks (CNNs) and long short-term memory (LSTM) networks. The proposed approach utilized the CNN’s capability to extract significant features from brain tumor images and the LSTM’s capacity to capture temporal dependencies present in the data. The cascaded framework comprised of two stages: firstly, a CNN was utilized to extract features from the tumor images, and subsequently, an LSTM network was employed to model the temporal information within these extracted features. The experimental findings clearly illustrate the exceptional performance of the CNN-LSTM framework when compared to other cutting-edge methods, exhibiting remarkable accuracy in the classification of brain tumors. The proposed method held promise for improving the diagnosis and treatment planning of brain tumors, ultimately benefiting patients and healthcare professionals in the field of neuro-oncology.

Also, Srikantamurthy et al. ( 57 ) proposed a hybrid approach for accurately classifying benign and malignant subtypes of breast cancer using histopathology imaging. Transfer learning was used to combine the strengths of long short-term memory (LSTM) networks with convolutional neural networks (CNNs) in a synergistic manner. The histopathological pictures were initially processed by the CNN to extract relevant characteristics, which were then sent into the LSTM network for sequential analysis and classification. By harnessing transfer learning, the model capitalized on pre-trained CNNs trained on extensive datasets, thereby facilitating efficient representation learning. The proposed hybrid approach showed promising results in accurately distinguishing between benign and malignant breast cancer subtypes, contributing to improved diagnosis and treatment decisions in breast cancer patients.

Besides, Banerjee et al. ( 58 ) presented a hybrid approach combining Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) for the classification of histopathological breast cancer images. Using data augmentation approaches, the classifier’s robustness is increased. ResNet50, InceptionV3, and a CNN that has been pretrained on ImageNet are used to extract deep convolutional features. An LSTM Recurrent Neural Network (RNN) is then fed these features for classification. Comparing the performance of three alternative optimizers, it is found that Adam outperforms the others without leading to model overfitting. The experimental findings showed that, for both binary and multi-class classification problems, the suggested strategy outperforms cutting-edge approaches. Furthermore, the method showed promise for application in the classification of other types of cancer and diseases, making it a versatile and potentially impactful approach.

Moreover, Nandhini Abirami et al. ( 59 ) explored the application of deep Convolutional Neural Networks (CNNs) and deep Generative Adversarial Networks (GANs) in computational visual perception-driven image analysis. To increase the precision and resilience of image analysis tasks, the authors suggested a unique framework that combines the advantages of both CNNs and GANs. The deep GAN is used to create realistic and high-quality synthetic pictures, while the deep CNN is used for feature extraction and capturing high-level visual representations. The combination of these two deep learning models made it possible to analyze images more efficiently, especially when performing tasks like object identification, picture recognition, and image synthesis. Experimental results demonstrated the superiority of the proposed framework over traditional approaches, highlighting the potential of combining deep CNNs and GANs for advanced computational visual perception in image analysis.

Additionally, Yao et al. ( 60 ) proposed a parallel structure deep neural network for breast cancer histology image classification, combining Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) with an attention mechanism. The histology pictures’ ability to extract both local and global characteristics thanks to the parallel construction improved the model’s capacity to gather pertinent data. The CNN component concentrated on obtaining spatial characteristics from picture patches, whereas the RNN component sequentially captured temporal relationships between patches. By focusing attention on key visual areas, the attention mechanism improved the model’s capacity for discrimination. The suggested method’s potential for accurate breast cancer histology picture categorization was shown by experimental findings, which showed that it performs better than baseline approaches. Table 7 discusses the hybrid methods used in medical image analysis.

The methods, properties, and features of hybrid-medical image analysis mechanisms.

Author	Main idea	Simulation environment	Datasets
Shahzadi, et al. ( )	Proposing a novel cascaded framework for accurately classifying brain tumors	MATLAB	100 samples
Srikantamurthy, et al. ( )	Proposing a hybrid approach for accurately classifying benign and malignant subtypes of breast cancer	Python	5,000 breast images
Banerjee, et al. ( )	Presenting a hybrid approach combining CNN and LSTM for the classification of histopathological breast cancer images	Tensorflow	828 samples
Nandhini Abirami, et al. ( )	Exploring the application of deep CNNs and deep GANs in computational visual perception-driven image analysis	–	70,000 images
Yao, et al. ( )	Proposing a parallel structure deep neural network for breast cancer histology image classification	–	100 images

6. Results and comparisons

The utilization of DL algorithms in medical image analysis purposes represents a pioneering stride toward the progress of medical and healthcare industries. This paper presents various innovative applications that demonstrate this paradigm, showcasing advanced knowledge in medical image analysis for motivating readers to explore innovative categories pertaining to DL algorithms in medical image analysis. The primary focus of this work is on different classifications of DL techniques utilized for DL methods in medical image analysis. Through a comprehensive analysis, it has been discovered that most DL methods in medical image analysis concentrate on advanced datasets, combined learning tasks, and annotation protocols. However, a significant limitation toward achieving the same level of functionality in medical images-DL algorithms is the inadequacy of large datasets for training, and standardized collection of data. It is crucial to ensure that diverse types of data require larger and more diverse datasets to provide reliable outcomes. Detection tasks in this field predominantly employ CNN or CNN-based techniques. In most of investigated papers the authors evaluated the topic based on several attributes, including accuracy, F-score, AUC, sensitivity, specificity, robustness, recall, adaptability, and flexibility. Sections 5.1 to 5.5 illustrate the medical image analysis-DL algorithms, where the majority of the proposed methods use both benchmark and real-time data. The DL methods used in these sections has been demonstrated in Figure 8 . The systems employed various datasets in terms of numbers and diverse categories, with accuracy, computational complexity, sensitivity, specificity, robustness, generalizability, adaptability, scalability, and F-score being the primary parameters evaluated. Accuracy was the main parameter for image analysis-based systems, whereas transparency was the least applied parameter as is depicted in Figure 9 . Its importance lies behind its direct impact on patient outcomes and healthcare decision-making. Medical image analysis plays a critical role in diagnosing and monitoring various diseases and conditions, and any inaccuracies or errors in the analysis can have serious consequences. High accuracy ensures that the deep learning algorithms can effectively and reliably detect abnormalities, classify different tissue types, and provide accurate predictions. This enables healthcare professionals to make well-informed decisions regarding treatment plans, surgical interventions, and disease management. Furthermore, accurate analysis helps reduce misdiagnosis rates, minimizes unnecessary procedures or tests, and improves overall patient care by enabling timely and appropriate interventions. In order to guarantee the efficiency and dependability of deep learning algorithms in medical image processing, accuracy acts as a crucial criterion. The majority of the solutions used the data normalization approach to combine photos from various sources that were of comparable size and quality. Some of the systems offered, however, did not provide the compute time since different datasets were utilized in the study. The datasets used in the study varied in terms of sample size, accessibility requirements, picture size, and classes. One of the most often employed algorithms was the RNN method, although cross-validation was seldom ever applied in most studies. Given that it is uncertain how the test results fluctuate, this might potentially reduce the outcomes’ resilience while delivering a high-functioning model. It is worth mentioning that cross-validation is crucial for evaluating the entire dataset. Multiple studies employ DL-based methodologies, and it is challenging to establish clear, robust, and resilient models. Future tasks include minimizing false-positive and false-negative rates to emphasize viral from bacterial pneumonia dependability. Associating DL methods in for developing medical image analysis represents a groundbreaking pace forward in technological development. It is worth mentioning that as is demonstrated in Figure 10 , Python is the most common programming language used in this context due to several key factors. Firstly, Python offers a rich ecosystem of libraries and frameworks specifically tailored for machine learning and deep learning tasks, such as TensorFlow, PyTorch, and Keras. These libraries provide efficient and user-friendly tools for developing and deploying deep learning models. Additionally, Python’s simplicity and readability make it an accessible language for researchers, clinicians, and developers with varying levels of programming expertise. Its extensive community support and vast online resources further contribute to its popularity. Moreover, Python’s versatility allows seamless integration with other scientific computing libraries, enabling researchers to preprocess, visualize, and analyze medical image data efficiently. Its wide adoption in academia, industry, and research communities fosters collaboration and knowledge sharing among experts in the field. Overall, Python’s powerful capabilities, ease of use, and collaborative ecosystem make it the preferred choice for implementing deep learning algorithms in medical image analysis. In the domain of Medical Image Analysis using Deep Learning Algorithms, diverse methodologies are employed to extract meaningful insights from complex medical imagery. CNNs are extensively utilized for their ability to automatically identify intricate patterns and features within images. RNNs, on the other hand, are crucial when dealing with sequential medical image data, such as video sequences or time-series images, as they capture temporal dependencies. Additionally, GANs play a pivotal role, especially in tasks requiring image generation or translation. Hybrid models, which integrate different architectures like CNNs and RNNs, offer a versatile approach for handling diverse types of medical image data that may require both spatial and temporal analysis. These methodologies are implemented and simulated within specialized environments, commonly leveraging Python libraries like TensorFlow, PyTorch, and Keras, which provide comprehensive support for deep learning. GPU acceleration is often utilized to expedite model training due to the computational intensity of deep learning tasks. Furthermore, custom simulation environments may be created to mimic specific aspects of medical imaging processes. The choice of datasets is paramount; researchers may draw from open-access repositories like ImageNet for pre-training, but specialized medical imaging repositories such as TCIA or RSNA are crucial for tasks in healthcare. Additionally, custom-collected datasets tailored to specific medical image analysis tasks are often employed to ensure data relevance and quality. Data augmentation techniques, like rotation and scaling, are applied to expand datasets and mitigate limitations associated with data scarcity. These synergistic efforts in methodologies, simulation environments, and datasets are essential for the successful development and evaluation of deep learning algorithms in medical image analysis, facilitating accurate and reliable results for a wide array of healthcare applications.

An external file that holds a picture, illustration, etc.
Object name is fpubh-11-1273253-g008.jpg

DL methods used in medical image analysis.

An external file that holds a picture, illustration, etc.
Object name is fpubh-11-1273253-g009.jpg

The most important parameters considered in investigated papers.

An external file that holds a picture, illustration, etc.
Object name is fpubh-11-1273253-g010.jpg

Programming languages used in learning algorithms used for medical image analysis.

6.1. Convolutional neural network

CNNs have been used successfully in medical image processing applications, however they also have significant drawbacks and difficulties. Due to the high expense and complexity of image collecting and annotation, it may be challenging to get the vast quantity of labeled data needed to train the network in the context of medical imaging. Additionally, the labeling procedure may add some subjectivity or inter-observer variability, which can influence the CNN models’ accuracy and dependability ( 61 ). A further issue is the possible bias of CNN models toward the distribution of training data, which might result in subpar generalization performance on fresh or untried data. This is particularly relevant in medical imaging, where the patient population may be diverse and heterogeneous, and the image acquisition conditions may vary across different imaging modalities and clinical settings. Furthermore, the interpretability of CNN models in medical imaging is still a major concern, as they typically rely on complex and opaque learned features that are difficult to interpret or explain. This limits the ability of clinicians to understand and trust the decisions made by the CNN models, and may hinder their adoption in clinical practice. Finally, CNN models are computationally intensive and require significant computational resources, which may limit their scalability and practical use in resource-constrained environments or low-resource settings ( 62 ).

The CNN method offers several benefits in the context of healthcare applications. Firstly, CNNs can automatically learn relevant features from raw input data such as medical images or physiological signals, without requiring manual feature extraction. This makes them highly effective for tasks such as image classification, object detection, and segmentation, and can lead to more accurate and efficient analyzes. Secondly, CNNs can handle large amounts of complex data and improve classification accuracy, making them well-suited for medical diagnosis and prediction ( 63 ). Additionally, CNNs can be trained on large datasets, which can help in detecting rare or complex patterns in the data that may be difficult for humans to identify. Finally, the use of deep learning algorithms such as CNNs in healthcare applications has the potential to improve patient outcomes, enable early disease detection, and reduce medical costs.

6.2. Recurrent neural network

Recurrent Neural Networks (RNNs) have shown great success in modeling sequential data such as time series and natural language processing tasks. However, in medical image analysis, there are some challenges and limitations when using RNNs. RNNs are mainly designed to model temporal sequences and do not have a natural way of handling spatial information in images. This can limit their ability to capture local patterns and spatial relationships between pixels in medical images. RNNs require a lot of computational power to train, especially when dealing with large medical image datasets ( 64 ). This can make it difficult to train models with high accuracy. When training deep RNN models, the gradients can either vanish or explode, making it difficult to optimize the model parameters effectively. This can lead to longer training times and lower accuracy. RNNs are prone to overfitting when the size of the training dataset is small. This can result in poor generalization performance when the model is applied to new, unseen data. Unbalanced data: In medical image analysis, the dataset may be highly unbalanced, with a small number of positive cases compared to negative cases. This can make it difficult to train an RNN model that can accurately classify the data. Researchers have created a variety of RNN-based designs, including Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs), which have demonstrated promising performance in applications involving medical picture interpretation. Additionally, combining RNNs with other deep learning techniques such as CNNs can help improve performance by capturing both spatial and temporal features ( 65 ).

It’s possible that these papers faced some challenges when using the RNN method. RNNs can suffer from vanishing gradients, where the gradients used for optimization become very small and make learning slow or even impossible. This can be a challenge for RNNs when working with long sequences of data. Overfitting is a problem with RNNs, when the model gets too complicated and begins to memorize the training set rather than generalizing to new data. When working with little data, like in applications for the healthcare industry, this can be particularly difficult. RNNs may be difficult to train computationally, especially when working with big volumes of data ( 66 ). This can be a challenge when working with IoT devices that have limited computational resources. There are many different types of RNNs and architectures to choose from, each with its own strengths and weaknesses. It can be challenging to select the right architecture for a given task. Overall, while RNNs can be powerful tools for analyzing time-series data in IoT applications, they do come with some potential challenges that must be carefully considered when using them.

6.3. Generative adversarial network

Generative Adversarial Networks (GANs) have shown promising results in various fields, including medical image analysis. However, GANs also face some challenges and limitations, which can affect their performance in medical image analysis. Medical image datasets are often limited due to the cost and difficulty of acquiring large amounts of high-quality data. To correctly understand the underlying distribution of the data, GANs need a lot of data. Therefore, when working with tiny medical picture datasets, the performance of GANs may be constrained ( 67 ). Medical picture databases may not be evenly distributed, which means that some classifications or diseases are underrepresented. For underrepresented classes or circumstances, GANs could find it difficult to provide realistic examples. Regardless of the input, mode collapse happens when a GAN’s generator learns to produce only a small number of samples. Mode collapse in medical image processing can lead to the creation of irrational pictures or the loss of crucial data. Overfitting is a problem with GANs that happens when the model memorizes the training data rather than generalizing to. There is currently no standardization for evaluating GANs in medical image analysis. This can make it challenging to compare different GAN models and assess their performance accurately. Addressing these challenges and limitations requires careful consideration of the specific medical image analysis task, the available data, and the design of the GAN model. Moreover, a multi-disciplinary approach involving clinicians, radiologists, and computer scientists is necessary to ensure that the GAN model’s outputs are meaningful and clinically relevant ( 68 ).

6.4. Long short-term memory

Long Short-Term Memory (LSTM) networks are a type of recurrent neural network that has shown promising results in various applications, including medical image analysis. However, LSTMs also face some challenges and limitations, which can affect their performance in medical image analysis. LSTMs rely on a fixed-length input sequence, and the context provided by the input sequence may be limited, especially in the case of medical image analysis. For example, in a sequence of medical images, it may be challenging to capture the full context of the images in a fixed-length input sequence. LSTMs can be prone to overfitting, especially when dealing with small datasets. When the model starts to memorize the training data instead of generalizing to new, untried data, overfitting might happen. This might lead to subpar performance when the model is tested on fresh medical photos ( 69 ). LSTMs are sometimes referred to as “black box” models since it might be difficult to understand how the model generates its predictions. This can be a limitation in medical image analysis, where clinicians need to understand how the model arrived at its decision. LSTMs can be computationally expensive, especially when dealing with long input sequences or large medical image datasets. This can make it challenging to train the model on a standard computer or within a reasonable time frame. Medical image datasets can be imbalanced, meaning that certain classes or conditions are underrepresented in the dataset. LSTMs may struggle to learn the patterns of underrepresented classes or conditions. LSTMs may have limited generalizability to new medical image datasets or different medical conditions, especially if the model is trained on a specific dataset or medical condition Addressing these challenges and limitations requires careful consideration of the specific medical image analysis task, the available data, and the design of the LSTM model. Moreover, a multi-disciplinary approach involving clinicians, radiologists, and computer scientists is necessary to ensure that the LSTM model’s outputs are meaningful and clinically relevant. Additionally, techniques such as data augmentation, transfer learning, and model compression can be used to improve the performance of LSTMs in medical image analysis ( 70 ).

6.5. Hybrid

The reason for using hybrid methods, such as combining CNN and LSTM, is that they have complementary strengths and weaknesses. CNN is particularly good at extracting spatial features from high-dimensional data such as images, while LSTM is good at modeling temporal dependencies in sequences of data. By combining them, one can leverage the strengths of both to improve the accuracy of the prediction. Additionally, hybrid methods can be used to address challenges such as overfitting, where the model may become too specialized on the training data, and underfitting, where the model may not capture the underlying patterns in the data ( 71 ). Hybrid models can also provide a more robust approach to dealing with noisy or missing data by allowing for more complex interactions between features and time.

The use of hybrid approaches, like CNN-LSTM, in medical image analysis with deep learning algorithms, presents several challenges and limitations. Firstly, the complexity of the network architecture poses a significant hurdle in training these models. Integrating different models with diverse parameters, loss functions, and optimization algorithms can lead to suboptimal performance, potentially causing overfitting or underfitting issues, which adversely impact accuracy and generalizability ( 72 ). Secondly, a major challenge lies in obtaining a substantial amount of data to effectively train hybrid models. Medical image data is often scarce and costly to acquire, thereby restricting the capacity to train deep learning models comprehensively ( 73 ). Furthermore, medical image data’s high variability and subjectivity can compromise the training data quality and model performance. Moreover, interpreting the results generated by hybrid models can be problematic. The models’ complexity may obscure the understanding of how they arrive at predictions or classifications, limiting their practicality in clinical practice and possibly raising doubts or skepticism among medical professionals. Lastly, the computational cost associated with training and deploying hybrid models can be prohibitive ( 74 ). These models demand powerful hardware and are computationally intensive, limiting their applicability in real-world medical settings. The ability to utilize the capabilities of both models and enhance the accuracy and performance of the entire system are two advantages of utilizing hybrid approaches, such as the CNN-LSTM model. The CNN layer is utilized in the CNN-LSTM model-based COVID-19 prediction to extract spatial characteristics from the data, while the LSTM layer is used to capture temporal relationships and provide predictions based on time series data. Similar to how the CNN layer is used to extract spatial information from the EEG data in the low-invasive and low-cost BCI headband, the LSTM layer is used to collect temporal relationships and categorize the signals. When reconstructing an ECG signal using a Doppler sensor, the hybrid. Overall, the hybrid models can provide better performance and accuracy compared to using either model alone ( 75 ).

The utilization of hybrid methods, such as the CNN-LSTM model, offers various advantages, including the amalgamation of both models’ strengths to enhance the overall system’s accuracy and performance. For instance, the CNN layer is used to extract spatial characteristics from the data in the COVID-19 prediction using the CNN-LSTM model, while the LSTM layer collects temporal relationships and makes predictions based on the time series data. Similar to how the CNN layer gets spatial information from the EEG data in the instance of EEG detection using a low-invasive and affordable BCI headband, the LSTM layer captures temporal relationships and categorizes the signals ( 76 ). The hybrid model makes use of the CNN layer to extract high-level features from the Doppler signal in the context of reconstructing an ECG signal using a Doppler sensor, and the LSTM layer makes use of the derived features to help reconstruct the ECG signal. In summary, employing hybrid models can yield superior performance and accuracy compared to using either model individually. This approach enables the combination of spatial and temporal information, harnessing the strengths of both CNN and LSTM models to enhance various applications such as COVID-19 prediction, EEG detection, and ECG signal reconstruction.

6.6. Prevalent evaluation criteria

Due to its capacity to increase the precision and efficacy of medical diagnosis and therapy, deep learning algorithms for medical image analysis have grown in popularity in recent years. In evaluating the performance of deep learning algorithms in medical image analysis, there are several prevalent evaluation criteria, which are described below ( 13 ).

6.6.1. Accuracy

Accuracy is the most commonly used metric for evaluating the performance of deep learning algorithms in medical image analysis. It measures the percentage of correctly classified images or regions of interest (ROIs) in medical images.

6.6.2. Sensitivity and specificity

Sensitivity measures the proportion of true positive results, which are the number of positive cases that are correctly identified by the algorithm. Specificity measures the proportion of true negative results, which are the number of negative cases that are correctly identified by the algorithm. Both metrics are used to evaluate the diagnostic performance of deep learning algorithms in medical image analysis.

6.6.3. Precision and recall

Precision measures the proportion of true positive results among all the positive cases identified by the algorithm. Recall measures the proportion of true positive results among all the positive cases in the ground truth data. Both metrics are used to evaluate the performance of deep learning algorithms in medical image analysis, particularly in binary classification tasks.

6.6.4. F1-score

The F1-score is a metric that combines precision and recall into a single score. It is often used to evaluate the performance of deep learning algorithms in medical image analysis, particularly in binary classification tasks.

6.6.5. Hausdorff distance

The Hausdorff distance is a metric that measures the maximum distance between the boundaries of two sets of ROIs in medical images. It is often used to evaluate the segmentation accuracy of deep learning algorithms in medical image analysis.

In general, the unique task and setting of the medical image analysis determine the selection of assessment criteria. In order to evaluate the outcomes of deep learning algorithms in the context of clinical practice, it is crucial to choose appropriate assessment criteria that are pertinent to the therapeutic demands.

6.7. Challenges of the DL applications in medical image analysis

The lack of high-quality annotated data is one of the greatest problems with deep learning (DL) algorithms used for medical image analysis. For DL models to perform well and generalize, they need a lot of labeled data. But getting high-quality annotations for medical photos is challenging for a number of reasons: restricted accessibility: Because it is expensive and time-consuming to capture and annotate medical pictures, the amount of data from annotated images is constrained ( 76 ). Additionally, the process of annotating calls for medical professionals with particular training and understanding, who are not always available. Due to changes in patient anatomy, imaging modality, and disease pathology, medical pictures are complicated and extremely varied. Annotating medical images requires a high degree of accuracy and consistency, which can be challenging for complex and heterogeneous medical conditions. Privacy and ethical issues: The annotation process has the potential to make medical photographs containing sensitive patient data vulnerable to abuse or unauthorized access. Medical image analysis has a significant difficulty in protecting patient privacy and confidentiality while preserving the caliber of annotated data. Annotating medical pictures requires making subjective assessments, which might result in bias and variability in the annotations. These variables may have an impact on the effectiveness and generalizability of DL models, especially when the annotations are inconsistent among datasets or annotators ( 77 ). To address the challenge of limited availability of high-quality annotated data, several approaches have been proposed, including:

Transfer learning: To enhance the performance of DL models on smaller datasets, transfer learning uses pre-trained models that have been learned on big datasets. By using this method, the volume of annotated data needed to train DL models may be decreased, and the generalizability of the models can be increased.
Data augmentation: By applying modifications to already-existing, annotated data, data augmentation includes creating synthetic data. The diversity and quantity of annotated data available for DL model training may be increased using this method, and it can also raise the models’ resistance to fluctuations in medical pictures.
Active learning: Active learning involves selecting the most informative and uncertain samples for annotation, rather than annotating all the data. This approach can reduce the annotation workload and improve the efficiency of DL model training.
Collaborative annotation: Collaborative annotation involves engaging medical experts, patients, and other stakeholders in the annotation process to ensure the accuracy, consistency, and relevance of annotations to clinical needs and values.

Overall, addressing the challenge of limited availability of high-quality annotated data in medical image analysis requires a combination of technical, ethical, and social solutions that can improve the quality, quantity, and diversity of annotated data while ensuring patient privacy and ethical standards.

Deep learning algorithms for medical image analysis have a significant problem in terms of data quality. The model’s performance may be considerably impacted by the caliber of the data utilized to train the deep learning algorithms ( 78 ). Obtaining medical pictures may be difficult, and their quality can vary based on a number of variables, such as the image capture equipment used, the image resolution, noise, artifacts, and the imaging technique. Furthermore, the annotations or labels used for training can also impact the quality of the data. Annotations may not always be accurate, and they may suffer from inter-and intra-observer variability, which can lead to biased models or models with poor generalization performance. To overcome the challenge of data quality, researchers need to establish robust quality control procedures for both image acquisition and annotation. Additionally, they need to develop algorithms that can handle noisy or low-quality data and improve the accuracy of annotations. Finally, they need to develop methods to evaluate the quality of the data used to train the deep learning models ( 79 ).

Interpretability poses a significant challenge in medical image analysis when employing deep learning algorithms, primarily due to the conventional black-box nature of these models, which makes it arduous to comprehend the reasoning behind their predictions. This lack of interpretability hinders clinical acceptance, as healthcare professionals necessitate understanding and trust in a model’s decision-making process to utilize it effectively. Moreover, interpretability plays a vital role in identifying and mitigating biases within the data and model, ensuring that decisions are not influenced by irrelevant or discriminatory features. Various approaches have been developed to enhance the interpretability of deep learning models for medical image analysis ( 80 ). These approaches include visualization techniques, saliency maps, and model explanations. Nonetheless, achieving complete interpretability remains a challenge in this field as it necessitates a trade-off between model performance and interpretability. Striking the right balance between these factors remains an ongoing endeavor. Transferability refers to the ability of a deep learning model trained on a particular dataset to generalize and perform well on new datasets that have different characteristics. In the context of medical image analysis, transferability is a significant challenge due to the diversity of medical imaging data, such as variations in image quality, imaging protocols, and imaging modalities. Deep learning models that are trained on a specific dataset may not perform well on different datasets that have variations in data quality and imaging characteristics. This can be problematic when developing deep learning models for medical image analysis because it is often not feasible to train a new model for every new dataset. To address this challenge, researchers have explored techniques such as transfer learning and domain adaptation. Transfer learning involves using a pre-trained model on a different but related dataset to initialize the model weights for the new dataset, which can improve performance and reduce the amount of training required. Domain adaptation involves modifying the model to account for the differences between the source and target domains, such as differences in imaging protocols or modalities ( 81 ). However, the challenge of transferability remains a significant issue in medical image analysis, and there is ongoing research to develop more robust and transferable deep learning models for this application.

In deep learning-based medical image analysis, overfitting is a frequent problem when a model gets overly complicated and fits the training data too closely, leading to poor generalization to new, unforeseen data. Numerous factors, including the inclusion of noise in the training data, an unbalanced class distribution, or a lack of training data, can lead to overfitting ( 64 ). The latter is a prevalent problem in medical imaging since the dataset size is constrained by the absence of annotated data. Overfitting can provide erroneous positive or negative findings because it can produce high accuracy on training data but poor performance on validation or testing data. To avoid overfitting in deep learning, several strategies may be used, including regularization, early halting, and data augmentation. In medical image analysis, ensuring the quality of data and increasing the size of the dataset are essential to prevent overfitting.

Clinical adoption refers to the process of integrating new technologies or methodologies into clinical practice. In the context of medical image analysis using deep learning algorithms, clinical adoption is a challenge because it requires a significant change in how physicians and healthcare providers diagnose and treat patients ( 82 ). Clinical adoption involves not only technical considerations such as integrating the algorithms into existing systems and workflows, but also ethical, legal, and regulatory considerations, as well as training healthcare providers to use the new technology effectively and safely. One of the key challenges of clinical adoption is ensuring that the deep learning algorithms are accurate and reliable enough to be used in clinical decision-making. This requires rigorous validation and testing of the algorithms, as well as addressing concerns around the interpretability and generalizability of the results. Additionally, healthcare providers and patients may have concerns about the use of these algorithms in making medical decisions, particularly if the algorithms are seen as replacing or minimizing the role of the human clinician. Another challenge of clinical adoption is the need for regulatory approval, particularly in cases where the algorithms are used to support diagnosis or treatment decisions. Regulatory bodies such as the FDA may require clinical trials to demonstrate the safety and effectiveness of the algorithms before they can be used in clinical practice. The adoption of these technologies may be slowed down by this procedure since it can be time-consuming and expensive. Overall, clinical adoption is an important challenge to consider in the development and deployment of medical image analysis using deep learning algorithms, as it affects the ultimate impact of these technologies on patient care ( 83 ).

6.8. Dataset in medical image analysis using ML algorithms

In medical image analysis, a dataset is a collection of medical images that are used to train machine learning algorithms to detect and classify abnormalities or diseases. The dataset could be obtained from various sources such as clinical trials, imaging studies, or public repositories ( 84 ). The dataset’s data quality and size have a significant impact on how well the machine learning algorithm performs. Therefore, a dataset should be diverse and representative of the population under study to ensure the accuracy and generalizability of the algorithm. In addition, datasets may require pre-processing, such as normalization or augmentation, to address issues such as data imbalance, low contrast, or artifacts. A fundamental issue in the field of medical image analysis is still finding and using big, carefully managed medical picture databases. However, efforts are underway to improve the quality and availability of medical image datasets for researchers to advance the development of ML algorithms for medical diagnosis and treatment. In medical image analysis using machine learning (ML) algorithms, a dataset is a collection of images that are used to train and test ML models. Any ML project must include a dataset since the dataset’s size and quality directly affect how well the model performs. Obtaining and annotating medical photos from a variety of sources, including hospitals, clinics, and research organizations, is a standard step in the process of producing a dataset ( 85 ). To specify the areas of interest or characteristics that the ML model needs to learn, the pictures must be tagged. These labels could provide details about the disease shown in the picture, the anatomy of the area being imaged, or other pertinent facts. The training set and the test set are formed once the dataset is first established. The ML model is trained using the training set, and tested using the test set. As such, there is ongoing research in the field of medical image analysis aimed at improving dataset quality and size, as well as developing better methods for acquiring and labeling medical images ( 74 , 86 ).

6.9. Security issues, challenges, risks, IoT and blockchain usage

Medical image analysis using deep learning algorithms raises several security issues, particularly with regard to patient privacy and data protection. The medical images used for training the deep learning models may contain sensitive information, such as personally identifiable information (PII), health records, and demographic information, which must be kept confidential and secure. One of the main security issues is the risk of data breaches, which can occur during the data collection, storage, and transmission stages. Hackers or unauthorized personnel can intercept the data during transmission, gain access to the storage systems, or exploit vulnerabilities in the software or hardware infrastructure used to process the data ( 13 ). To mitigate this risk, various security measures must be put in place, such as encryption, access controls, and monitoring tools ( 87 ). Another security issue is the possibility of malicious attacks on the deep learning models themselves. Attackers can attempt to manipulate the models’ outputs by feeding them with malicious inputs, exploiting vulnerabilities in the models’ architecture or implementation, or using adversarial attacks to deceive the models into making wrong predictions. This can have serious consequences for patient diagnosis and treatment, and thus, it is critical to design and implement secure deep learning models. In summary, security is a critical concern in medical image analysis using deep learning algorithms, and it is essential to adopt appropriate security measures to protect the confidentiality, integrity, and availability of medical data and deep learning models.

There are several risks associated with medical image analysis using deep learning algorithms. Some of the key risks include:

Inaccuracy: Deep learning algorithms may sometimes provide inaccurate results, which can lead to incorrect diagnoses or treatment decisions.
Bias: Deep learning algorithms may exhibit bias in their decision-making processes, leading to unfair or inaccurate results for certain groups of patients.
Privacy concerns: Medical images often contain sensitive information about patients, and there is a risk that this data could be exposed or compromised during the analysis process.
Cybersecurity risks: As with any technology that is connected to the internet or other networks, there is a risk of cyberattacks that could compromise the security of medical images and patient data.
Lack of transparency: Deep learning algorithms can be difficult to interpret, and it may be challenging to understand how they arrive at their conclusions. This lack of transparency can make it difficult to trust the results of the analysis.

Overall, it is important to carefully consider these risks and take steps to mitigate them when using deep learning algorithms for medical image analysis. This includes implementing strong cybersecurity measures, ensuring data privacy and confidentiality, and thoroughly validating the accuracy and fairness of the algorithms.

The term “Internet of Things” (IoT) describes how physical “things” are linked to the internet so they can trade and gather data. IoT may be used to link medical imaging devices and enable real-time data collecting and analysis in the field of medical image analysis. For instance, a network may be used to connect medical imaging equipment like CT scanners, MRIs, and ultrasounds, which can then transfer data to a cloud-based system for analysis ( 88 ). This can facilitate remote consultations and diagnostics and speed up the examination of medical images. IoT can also make it possible to combine different medical tools and data sources, leading to more thorough and individualized patient treatment. However, the use of IoT in medical image analysis also raises security and privacy concerns, as sensitive patient data is transmitted and stored on a network that can be vulnerable to cyber-attacks.

7. Open issues

There are several open issues related to medical image analysis using deep learning algorithms. These include:

7.1. Data privacy

Data privacy is a major concern in medical image analysis using deep learning algorithms. Medical images contain sensitive patient information that must be kept confidential and secure. In order to secure patient data from illegal access, usage, or disclosure, any algorithm or system used for medical image analysis must follow this rule. This can be particularly difficult since medical image analysis sometimes involves enormous volumes of data, which raises the possibility of data breaches or unwanted access. The need to strike a balance between the demands of data access and patient privacy protection is one of the primary issues with data privacy in medical image analysis. Many medical image analysis algorithms rely on large datasets to achieve high levels of accuracy and performance, which may require sharing data between multiple parties ( 89 ). This can be particularly challenging when dealing with sensitive patient information, as there is a risk of data leakage or misuse. Several methods may be utilized to protect data privacy in medical image analysis in order to deal with these issues. These include rules and processes to guarantee that data is accessed and used only for legal purposes, data anonymization, encryption, and access restrictions. Additionally, to guarantee that patient data is safeguarded and handled properly, healthcare companies must ensure that they adhere to pertinent data privacy laws, such as HIPAA in the United States or GDPR in the European Union.

7.2. Data bias

When employing deep learning algorithms to analyze medical images, data bias is a serious open problem. It alludes to the fact that the data used to train the deep learning models contains systematic flaws ( 90 ). These blunders may result from variables including the choice of training data, how the data is labeled, and how representative the data are of the population of interest. Data bias can result in the creation of models that underperform on particular segments of the population, such as members of underrepresented groups or those who suffer from unusual medical diseases. This can have serious implications for the accuracy and fairness of medical image analysis systems, as well as for the potential harm caused to patients if the models are used in clinical decision-making. Addressing data bias requires careful consideration of the data sources, data labeling, and model training strategies to ensure that the models are representative and unbiased ( 91 ).

7.3. Limited availability of annotated data

Deep learning algorithms in medical image analysis need a lot of annotated data to be taught properly. Annotated data refers to medical images that have been labeled by experts to indicate the location and type of abnormalities, such as tumors, lesions, or other pathologies. However, obtaining annotated medical image datasets is particularly challenging due to several factors. First off, annotating medical photos takes time and requires in-depth understanding. Only experienced radiologists or clinicians can accurately identify and label abnormalities in medical images, which can limit the availability of annotated data. Secondly, there are privacy concerns associated with medical image data. Patient privacy is a critical concern in healthcare, and medical image data is considered particularly sensitive ( 92 ). As a result, obtaining large-scale annotated medical image datasets for deep learning is challenging due to privacy concerns and the need to comply with regulations such as HIPAA. Thirdly, the diversity of medical image data can also pose a challenge. Medical images can vary widely in terms of modality, acquisition protocols, and image quality, making it difficult to create large, diverse datasets for deep learning. Deep learning algorithms for medical image analysis may be limited in their ability to develop and be validated as a result of the difficulties in getting datasets of annotated medical images. In order to decrease the volume of labeled data needed for training, researchers have tackled this issue by adopting methods including transfer learning, data augmentation, and semi-supervised learning ( 93 ). However, these techniques may not be sufficient in all cases, and there is a need for more annotated medical image datasets to be made available to researchers to advance the field of medical image analysis using deep learning.

7.4. Interpretability and transparency

When employing deep learning algorithms for medical picture analysis, interpretability and transparency are crucial concerns. Deep learning models are sometimes referred to as “black boxes” because they can be tricky to read, making it difficult to comprehend how they made judgments. In medical image analysis, interpretability is essential for clinicians to understand and trust the algorithms, as well as to identify potential errors or biases. Interpretability refers to the ability to understand the reasoning behind a model’s decision-making process. Convolutional neural networks (CNNs), one type of deep learning model, can include millions of parameters that interact in intricate ways. This complexity can make it difficult to understand how the model arrived at a particular decision, especially for clinicians who may not have experience with deep learning. Transparency refers to the ability to see inside the model and understand how it works ( 94 ). In other words, transparency means that the model’s decision-making process is clear and understandable, and can be validated and audited. Transparency is essential for ensuring that the model is working correctly and not introducing errors or biases. In medical image analysis, interpretability and transparency are critical because clinicians need to understand how the algorithm arrived at its decisions. This understanding can help clinicians identify errors or biases and ensure that the algorithm is making decisions that are consistent with clinical practice. To increase the interpretability and transparency of deep learning models in medical image analysis, several techniques have been developed. For instance, heatmaps that display which areas of an image the model is utilizing to make judgments may be produced using visualization approaches. Additionally, attention mechanisms can be used to highlight important features in an image and explain the model’s decision-making process. Other techniques include using explainable AI (XAI) methods and incorporating domain knowledge into the models. While these techniques have shown promise, there is still a need for more transparent and interpretable deep learning models in medical image analysis to improve their utility in clinical practice.

7.5. Generalizability

A significant unresolved problem in deep learning-based medical picture analysis is generalizability. The capacity of a model to function effectively on data that differs from the data it was trained on is referred to as generalizability. In other words, a trained model should be able to generalize to other datasets and still perform well. In medical image analysis, generalizability is critical because it ensures that the deep learning algorithms can be used on new patient populations or in different clinical settings. However, deep learning models can be prone to overfitting, which occurs when a model performs well on the data it was trained on but performs poorly on new data. This can be particularly problematic in medical image analysis, where a model that overfits can lead to inaccurate or inconsistent diagnoses. The generalizability of deep learning models for medical image processing might vary depending on a number of variables. For instance, a model’s capacity to generalize can be significantly impacted by the variety of the dataset used to train it ( 95 ). The model might not be able to identify anomalies that it has never seen before if the training dataset is not sufficiently varied. Another factor that can affect generalizability is the performance of the model on different types of medical images. For example, a model that is trained on CT scans may not perform well on MRI scans because the image modality and acquisition protocols are different. Researchers are examining methods including transfer learning, data augmentation, and domain adaptation to increase the generalizability of deep learning models in medical picture analysis. Transfer learning entails fine-tuning a pre-trained model using a fresh dataset as a starting point. Data augmentation entails using transformations like rotations and translations to artificially expand the size and variety of the training dataset. The process of domain adaptation is modifying a model that has been trained on one dataset to function on another dataset with different properties. The generalizability of deep learning models in medical image processing has to be improved in order to assure their safe and efficient application in clinical practice, even if these approaches have showed promise ( 96 ).

7.6. Validation and regulatory approval

Validation and regulatory approval are important open issues in medical image analysis using deep learning algorithms. Validation refers to the process of verifying that a model is accurate and reliable. Regulatory approval refers to the process of obtaining approval from regulatory bodies, such as the FDA in the US, before a model can be used in clinical practice. Validation is critical in medical image analysis because inaccurate or unreliable models can lead to incorrect diagnoses and treatment decisions. Validation involves testing the model on a separate dataset that was not used for training and evaluating its performance on a range of metrics. Validation can also involve comparing the performance of the model to that of human experts. Regulatory approval is important in medical image analysis to ensure that the models are safe and effective for use in clinical practice. Regulatory bodies require evidence of the model’s safety, efficacy, and performance before approving it for use. This evidence can include clinical trials, real-world data studies, and other forms of validation. There are several challenges associated with validation and regulatory approval of deep learning models in medical image analysis. One challenge is the lack of standardized validation protocols, which can make it difficult to compare the performance of different models ( 97 ). Another challenge is the lack of interpretability and transparency of deep learning models, which can make it difficult to validate their performance and ensure their safety and efficacy. Researchers and regulatory organizations are collaborating to provide standardized validation processes and criteria for regulatory approval of deep learning models in medical image analysis in order to overcome these issues. For instance, the FDA has published guidelines for the creation and approval of medical devices based on machine learning and artificial intelligence (AI/ML). These guidelines provide recommendations for the design and validation of AI/ML-based medical devices, including those used for medical image analysis. While these efforts are promising, there is still a need for further research and collaboration between researchers and regulatory bodies to ensure the safe and effective use of deep learning models in medical image analysis ( 98 ).

7.7. Ethical and legal considerations

Deep learning algorithms for medical image processing raise a number of significant outstanding questions about moral and legal dilemmas. These factors concern the use of patient data in research, the possibility of algorithmic biases, and the duty of researchers and healthcare professionals to guarantee the ethical and safe application of these technologies. Use of patient data in research is one ethical issue. Large volumes of patient data are needed for medical image analysis, and the use of this data raises questions concerning patient privacy and permission. Patients’ privacy must be maintained, and researchers and healthcare professionals must make sure that patient data is utilized responsibly ( 99 ). The possibility for prejudice in algorithms is another ethical issue. Deep learning algorithms may be taught on skewed datasets, which might cause the model’s outputs to become biased. Biases can result in incorrect diagnosis and treatment choices in medical image analysis, which can have catastrophic repercussions. Researchers must take action to address any potential biases in their datasets and algorithms. Deep learning algorithms for medical image interpretation raise legal questions around intellectual property, liability, and compliance with regulations. Concerns exist around the possibility of unwanted access to patient data as well as the requirement to uphold data protection regulations in order to preserve patient privacy. To address these ethical and legal considerations, researchers and healthcare providers must ensure that they are following best practices for data privacy and security, obtaining informed consent from patients, and working to mitigate potential biases in their algorithms. It is also important to engage with stakeholders, including patients, regulatory bodies, and legal experts, to ensure that the development and use of these technologies is safe, ethical, and compliant with relevant laws and regulations ( 100 ).

7.8. Future works

Future research in the fast-developing field of medical image analysis utilizing deep learning algorithms has a lot of potential to increase the precision and effectiveness of medical diagnosis and therapy. Some of these areas include:

7.8.1. Multi-modal image analysis

Future research in medical image analysis utilizing deep learning algorithms will focus on multi-modal picture analysis. Utilizing a variety of imaging modalities, including MRI, CT, PET, ultrasound, and optical imaging, allows for a more thorough understanding of a patient’s anatomy and disease ( 101 ). This strategy can aid in enhancing diagnostic precision and lowering the possibility of missing or incorrect diagnoses. Multi-modal picture data may be used to train deep learning algorithms for a range of tasks, including segmentation, registration, classification, and prediction. An algorithm built on MRI and PET data, for instance, might be used to identify areas of the brain afflicted by Alzheimer’s disease. Similarly, a deep learning algorithm could be trained on ultrasound and CT data to identify tumors in the liver. Multi-modal image analysis poses several challenges for deep learning algorithms. For example, different imaging modalities have different resolution, noise, and contrast characteristics, which can affect the performance of the algorithm. Additionally, multi-modal data can be more complex and difficult to interpret than single-modality data, requiring more advanced algorithms and computational resources ( 102 ). To address these challenges, researchers are developing new deep learning models and algorithms that can integrate and analyze data from multiple modalities. For example, multi-modal fusion networks can be used to combine information from different imaging modalities, while attention mechanisms can be used to focus the algorithm’s attention on relevant features in each modality. Overall, multi-modal image analysis holds promise for improving the accuracy and efficiency of medical diagnosis and treatment using deep learning algorithms. As these technologies continue to evolve, it will be important to ensure that they are being used safely, ethically, and in accordance with relevant laws and regulations.

7.8.2. Explainable AI

Future research in deep learning algorithms for medical image analysis will focus on explainable AI (XAI). XAI is the capacity of an AI system to explain its decision-making process in a way that is intelligible to a human ( 103 ). XAI can assist to increase confidence in deep learning algorithms when employed in the context of medical image analysis, guarantee that they are utilized safely and morally, and allow clinicians to base their judgments more intelligently on the results of these algorithms. XAI in medical image analysis involves developing algorithms that can not only make accurate predictions or segmentations but also provide clear and interpretable reasons for their decisions. This can be particularly important in cases where the AI system’s output contradicts or differs from the clinician’s assessment or prior knowledge. One approach to XAI in medical image analysis is to develop visual explanations or heatmaps that highlight the regions of an image that were most important in the algorithm’s decision-making process. These explanations can help to identify regions of interest, detect subtle abnormalities, and provide insight into the algorithm’s thought process ( 104 ). Another approach to XAI in medical image analysis is to incorporate external knowledge or prior information into the algorithm’s decision-making process. For example, an algorithm that analyzes brain MRIs could be designed to incorporate known patterns of disease progression or anatomical landmarks. Overall, XAI holds promise for improving the transparency, interpretability, and trustworthiness of deep learning algorithms in medical image analysis. As these technologies continue to evolve, it will be important to ensure that they are being used safely, ethically, and in accordance with relevant laws and regulations ( 105 ).

7.8.3. Transfer learning

Future research in the field of deep learning-based medical image processing will focus on transfer learning. Transfer learning is the process of using previously trained deep learning models to enhance a model’s performance on a new task or dataset. Transfer learning can be particularly helpful in the interpretation of medical images as it can eliminate the requirement for significant volumes of labeled data, which can be challenging and time-consuming to gather. Researchers can use pre-trained models that have already been trained on huge datasets to increase the precision and effectiveness of their own models by taking advantage of the information and representations acquired by these models. Since transfer learning can do away with the need for large amounts of labeled data, which can be difficult and time-consuming to collect, it can be very useful in the interpretation of medical pictures. By utilizing the knowledge and representations amassed by pre-trained models that have previously been trained on massive datasets, researchers may utilize them to improve the accuracy and efficacy of their own models ( 106 ). The pre-trained model could be a useful place to start for the medical image analysis problem since it enables the model to learn from less data and might lessen the possibility of overfitting. Additionally, transfer learning may increase the generalizability of deep learning models used for medical picture interpretation. Medical image analysis models may be able to develop more reliable and generalizable representations of medical pictures that are relevant to a wider range of tasks and datasets by making use of pre-trained models that have learnt representations of real images. Transfer learning has the potential to enhance the effectiveness, precision, and generalizability of deep learning models used for medical image interpretation. As these technologies continue to evolve, it will be important to ensure that they are being used safely, ethically, and in accordance with relevant laws and regulations.

7.8.4. Federated learning

Future research in deep learning algorithms for medical image analysis will focus on federated learning. Without the need to move the data to a central server, federated learning refers to the training of machine learning models on data that is dispersed among several devices or institutions. Federated learning can be especially helpful in the context of medical image analysis since it permits the exchange of information and expertise between institutions while safeguarding the confidentiality and security of sensitive patient data ( 107 ). In situations where patient data is subject to strong privacy laws, such as the Health Insurance Portability and Accountability Act (HIPAA) in the United States, this can be particularly crucial. Federated learning works by training a central machine learning model on a set of initial weights, which are then sent to each of the participating devices or institutions. Each device or institution then trains the model on their own local data, using the initial weights as a starting point. The updated weights from each device or institution are then sent back to the central server, where they are aggregated to update the central model. This process is repeated iteratively until the model converges. By training models using federated learning, medical institutions can leverage the collective knowledge and expertise of multiple institutions, improving the accuracy and generalizability of the models. Additionally, the confidentiality and privacy of patient data are preserved because the data stays on local devices or organizations. Overall, federated learning shows potential for enhancing deep learning models’ generalizability, speed, and privacy in the context of medical picture analysis ( 108 ). As these technologies continue to evolve, it will be important to ensure that they are being used safely, ethically, and in accordance with relevant laws and regulations.

7.8.5. Integration with electronic health records (EHRs)

Future development in deep learning algorithms for medical image analysis will focus on integration with electronic health records (EHRs). EHRs contain a wealth of clinical information, including patient demographics, medical history, laboratory results, and imaging studies. Researchers and clinicians may be able to increase the precision and effectiveness of medical image analysis by merging deep learning algorithms with EHRs. One potential application of this integration is to improve the interpretation of medical images by incorporating patient-specific information from EHRs. For example, deep learning algorithms could be trained to predict the likelihood of certain diseases or conditions based on a patient’s clinical history, laboratory results, and imaging studies. This may decrease the need for invasive or pricey diagnostic procedures and increase the accuracy of medical picture interpretation. Using deep learning algorithms to automatically extract data from medical photos and incorporate it into EHRs is a further possible use ( 109 ). For example, deep learning algorithms could be trained to automatically segment and measure lesions or tumors in medical images and record this information in the patient’s EHR. This may decrease the need for invasive or pricey diagnostic procedures and increase the accuracy of medical picture interpretation. Using deep learning algorithms to automatically extract data from medical photos and incorporate it into EHRs is a further possible use. This may lessen the strain on physicians and increase the effectiveness and precision of clinical decision-making. Overall, deep learning algorithm integration with EHRs shows potential for enhancing the precision, efficacy, and efficiency of medical picture processing. It will be crucial to make sure that these technologies are utilized safely, morally, and in line with all applicable laws and regulations regarding patient privacy and data security as they continue to advance ( 110 ).

7.8.6. Few-shots learning

Future research in Medical Image Analysis using DL algorithms should delve into the realm of Few-shot Learning. This approach holds great potential for scenarios where labeled data is limited or difficult to obtain, which is often the case in medical imaging ( 111 ). Investigating techniques that enable models to learn from a small set of annotated examples, and potentially even adapt to new, unseen classes, will be instrumental. Meta-learning algorithms, which aim to train models to quickly adapt to new tasks with minimal data, could be explored for their applicability in medical image analysis. Additionally, methods for data augmentation and synthesis specifically tailored for few-shot scenarios could be developed. By advancing Few-shot Learning in the context of medical imaging, we can significantly broaden the scope of applications, improve the accessibility of AI-driven healthcare solutions, and ultimately enhance the quality of patient care ( 112 ).

8. Conclusion and limitation

In recent years, there has been significant progress in medical image analysis using deep learning algorithms, with numerous studies highlighting the effectiveness of DL in various areas like cell, bone, tissue, tumor, vessel, and lesion segmentation. However, as the field continues to evolve, further research is essential to explore new techniques and methodologies that can improve the performance and robustness of DL algorithms in image analysis. Comprehensive evaluations of DL algorithms in real-world scenarios are needed, along with the development of scalable and robust systems for healthcare settings. Continuing research in this area is imperative to fully utilize the potential of DL in medical image segmentation and enhance healthcare outcomes. This article presents a systematic review of DL-based methods for image analysis, discussing advantages, disadvantages, and the strategy employed. The evaluation of DL-image analysis platforms and tools is also covered. Most papers are assessed based on qualitative features, but some important aspects like security and convergence time are overlooked. Various programming languages are used to evaluate the proposed methods. The investigation aims to provide valuable guidance for future research on DL application in medical and healthcare image analysis. However, the study encountered constraints, including limited access to non-English papers and a scarcity of high-quality research focusing on this topic. The heterogeneity in methodologies, datasets, and evaluation metrics used in the studies presents challenges in drawing conclusive insights and performing quantitative meta-analysis. Additionally, the rapidly evolving nature of DL techniques and the emergence of new algorithms may necessitate frequent updates to remain current. Despite these limitations, DL has proven to be a game-changing approach for addressing complex problems, and the study’s results are expected to advance DL approaches in real-world applications.

Data availability statement

Author contributions.

ML: Investigation, Writing – original draft. YJ: Investigation, Writing – review & editing. YZ: Investigation, Supervision, Writing – original draft. HZ: Investigation, Writing – original draft.

Funding Statement

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Information

Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

Active Journals
Find a Journal
Proceedings Series
For Authors
For Reviewers
For Editors
For Librarians
For Publishers
For Societies
For Conference Organizers
Open Access Policy
Institutional Open Access Program
Special Issues Guidelines
Editorial Process
Research and Publication Ethics
Article Processing Charges
Testimonials
Preprints.org
SciProfiles
Encyclopedia

Article Menu

Subscribe SciFeed
Recommended Articles
Google Scholar
on Google Scholar
Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Data management in biobanking: strategies, challenges, and future directions.

1. Introduction

2. biospecimens, 2.1. importance of biospecimens, 2.2. types of biospecimens.

Blood samples: Blood plays a crucial role in the body, transporting oxygen, nutrients, hormones, and waste products. Obtained through procedures like venipuncture or finger pricking, blood samples are rich in information, containing details like blood cell counts, biochemical markers, hormones, and genetic material (DNA and RNA). They are utilized across various medical fields for diagnostics, disease tracking, and research endeavors.
Tissue biopsies: Tissue biopsies involve extracting small tissue samples from organs or lesions for microscopic examination. These samples provide vital diagnostic insights, enabling pathologists to identify cellular irregularities, tissue structures, and molecular markers associated with conditions such as cancer, infections, and autoimmune disorders. Techniques like needle biopsies, surgical excision, and endoscopic procedures are employed to obtain tissue biopsies.
Saliva and oral swabs: Saliva and oral swabs contain a mix of cells, enzymes, proteins, and microorganisms that are present in the oral cavity. These specimens are collected non-invasively and are employed to study oral health, detect oral pathogens, and analyze the oral microbiome. Saliva samples also offer insights into systemic conditions like diabetes, cardiovascular disease, and autoimmune disorders. Oral swabs find utility in genetic testing and forensic analysis.
Urine samples: Urine, a waste product produced by the kidneys, holds metabolic byproducts, electrolytes, hormones, and other substances filtered from the blood. Routinely collected for urinalysis, urine samples help evaluate the kidney function, hydration status, and presence of abnormalities such as urinary tract infections, kidney stones, and proteinuria. They are also utilized in drug screening, pregnancy testing, and research studies.
Stool samples: Stool, or feces, is the waste product expelled from the gastrointestinal tract. Stool samples contain undigested food, water, bacteria, viruses, and other substances. Collected for diagnostic purposes, they help detect gastrointestinal infections, evaluate digestive function, and screen for colorectal cancer. Stool samples are also used to explore the gut microbiome, digestive disorders, and inflammatory bowel diseases.

3. Data Types in Biobanking

3.1. clinical data, 3.2. image data.

Histopathological images: Histopathological images capture tissue samples stained with diverse dyes to visualize cellular structures and arrangements. These images are pivotal in disease diagnosis, tumor evaluation, and prognostic assessment. Biobanks maintain archives of histopathological slides alongside detailed clinical annotations, empowering researchers to correlate histological characteristics with molecular profiles and clinical outcomes.
Medical imaging: Medical imaging encompasses a plethora of techniques including MRI, CT scans, PET scans, ultrasound, X-rays, and thermal imaging, facilitating the non-invasive visualization of anatomical structures, physiological activities, and pathological changes in living organisms. Biobanks curate repositories of medical imaging data obtained from routine clinical procedures, research studies, and clinical trials, enabling retrospective analyses and longitudinal investigations across diverse patient cohorts [ 19 , 20 ].
Microscopy images: Microscopy images capture intricate cellular and subcellular structures with remarkable resolution, providing insights into cellular morphologies, spatial organizations, and dynamic processes. Biobanks preserve microscopy images that are acquired through various techniques such as light microscopy, electron microscopy, and confocal microscopy, supporting research endeavors in fields such as cell biology, neuroscience, and developmental biology. These images facilitate quantitative analyses of cellular phenotypes, protein distributions, and cellular interactions in both healthy and diseased states.

3.3. Omics Data

Genomic data, encapsulating DNA sequences, variations, and structural nuances, constitute an indispensable facet of biobanking. Driven by advances in high-throughput sequencing technologies, biobanks house diverse genomic datasets spanning entire genomes, exomes, and genotyping arrays. These datasets facilitate genome-wide association studies (GWASs), variant exploration, and pharmacogenomic investigations, with the integration of genomic data and clinical insights holding promise for deciphering genotype–phenotype relationships and guiding tailored treatment approaches.
Transcriptomic data: Transcriptomic data capture the expression profiles of genes under various biological conditions, unraveling intricate cellular processes and regulatory networks. Biobanks curate transcriptomic datasets derived from methodologies like microarrays and RNA sequencing (RNA-seq), enabling researchers to probe gene expression patterns linked to disease states, tissue phenotypes, and therapeutic responses. Transcriptomic analyses of biobanked specimens drive biomarker discovery, target identification, and mechanistic inquiries across diverse domains spanning oncology to neurology.
Proteomic data: Proteomic data entail the identification and quantification of proteins within biological samples, offering a snapshot of their cellular functions and signaling pathways. Biobanks store proteomic datasets derived from mass spectrometry-based techniques, immunoassays, and protein arrays, facilitating the characterization of protein expression, modifications, and interactions. The integration of proteomic insights with other omics layers enriches our understanding of disease mechanisms, biomarker profiles, and treatment responses, thereby paving the way for precise therapeutic interventions.
Metabolomic data: Metabolomic data capture the repertoire of small-molecule metabolites within biological samples, serving as mirrors of cellular metabolism and biochemical pathways. Biobanks archive metabolomic profiles obtained using methodologies like nuclear magnetic resonance (NMR) spectroscopy and liquid chromatography–mass spectrometry (LC-MS), enabling investigations into metabolic dysregulations across diseases such as cancer, metabolic disorders, and neurodegenerative conditions. The integration of metabolomic signatures with other omics datasets furnishes holistic insights into disease phenotypes and metabolic imbalances underpinning health and disease.

4. Challenges in Data Management

4.1. data heterogeneity.

Diverse data types: Biobanks collect a wide range of biological samples, including tissues, blood, urine, and cells, each with its unique characteristics and properties. Furthermore, the associated data encompass a wide range of data types, including genomic data, clinical records, imaging data, and information on environmental exposure. Managing such diverse datasets requires robust systems capable of handling multiple data formats, structures, and standards [ 26 ].
Varying data standards: Different biobanks may adhere to varying data standards, terminology, and annotation protocols, leading to inconsistencies in data representation and interoperability challenges. Harmonizing data across multiple biobanks and research studies becomes inherently challenging due to the lack of standardized practices for data collection, annotation, and storage.
Data annotation and metadata: Effective data management relies on accurate metadata annotation to provide context and interpretability to the stored data. However, the heterogeneity of data sources often results in incomplete or inconsistent metadata, making it challenging to interpret and analyze the data accurately. Standardizing metadata annotation practices is essential for ensuring data integrity and facilitating data integration across different biobanks and research projects.
Integration and interoperability: Integrating heterogeneous datasets from multiple sources is crucial for conducting comprehensive analyses and deriving meaningful insights. However, data heterogeneity complicates the integration process, requiring sophisticated data integration methods and tools to reconcile the differences in data formats, semantics, and ontologies. Achieving interoperability across disparate datasets is essential for promoting data sharing and collaboration in the scientific community.
Data quality and reliability: Heterogeneous data sources may vary in quality, completeness, and reliability, posing challenges for ensuring data accuracy and consistency. Quality control measures must be implemented throughout the data lifecycle to identify and rectify errors, outliers, and inconsistencies. Data validation, cleaning, and normalization techniques are essential for maintaining data quality and reliability, particularly in large-scale biobanking initiatives.
Ethical and legal considerations: Data heterogeneity also extends to ethical and legal considerations surrounding data privacy, consent, and ownership. Harmonizing ethical standards and regulatory requirements across different jurisdictions is essential to ensure adherence to data protection regulations like GDPR and HIPAA.

4.2. Data Quality Assurance

Sample integrity and traceability: Biobanks must maintain the integrity and traceability of biological samples throughout their lifecycle, from collection to storage and distribution. Ensuring proper sample handling, storage conditions, and chain-of-custody protocols is crucial for preventing sample degradation, contamination, or mislabeling, which could compromise data quality and research outcomes.
Data accuracy and consistency: The data collected and curated in biobanks must be accurate, consistent, and reliable to support meaningful research conclusions. However, data entry errors, inconsistencies in data annotation, and discrepancies between different data sources can introduce inaccuracies and biases into the dataset. Implementing data validation checks, standardizing data entry procedures, and conducting regular data audits are imperative for upholding data accuracy and consistency.
Missing data and incomplete records: Incomplete or missing data entries are common challenges in biobanking, where data may be unavailable or incomplete due to various reasons such as sample collection limitations, participant non-compliance, or data entry errors. Addressing missing data requires robust data imputation techniques and strategies for data completeness assessment. Additionally, establishing protocols for documenting missing data and mitigating its impact on research outcomes is essential for maintaining data quality.
Data reconciliation and harmonization: Biobanks often aggregate data from multiple sources, including clinical records, laboratory measurements, and genetic analyses. Reconciling and harmonizing heterogeneous data sources to ensure consistency and interoperability pose significant challenges. Establishing standardized data formats, vocabularies, and ontologies, along with data normalization and transformation techniques, is essential for integrating diverse datasets while maintaining data quality.
Quality control processes: Implementing rigorous quality control processes is crucial for identifying and rectifying data errors, outliers, and inconsistencies. Quality control measures might encompass data validation checks, data cleaning procedures, and outlier detection algorithms, all aimed at ensuring the integrity and reliability of the data. Regular quality assessments and audits help monitor data quality over time and ensure adherence to established quality standards.
Long-term data preservation: Preserving data integrity and accessibility over the long term presents a considerable challenge for biobanks, particularly as technology and data formats evolve over time. Establishing robust data stewardship and preservation strategies, including data backup, version control, and migration plans, is essential for safeguarding data integrity and ensuring their longevity for future research endeavors.
Ethical and regulatory compliance: Data quality assurance in biobanking needs to adhere to ethical principles and regulatory requirements governing participant privacy, consent, and data protection. Implementing data governance frameworks, privacy safeguards, and security measures is essential for compliance with legal and ethical guidelines such as GDPR [ 27 ] and HIPAA while maintaining data quality and integrity.

4.3. Privacy and Security

Participant confidentiality: Biobanks hold considerable amounts of data containing sensitive information about participants, including personal identifiers, medical histories, and genetic profiles. Ensuring participant confidentiality and protecting privacy rights are fundamental ethical principles in biobanking. However, the amount and diversity of the data increase the risk of unintended disclosures or privacy breaches, necessitating robust privacy safeguards and access controls.
Encryption and access management: Deploying robust encryption protocols and access management systems is crucial for safeguarding biobank data against unauthorized access or breaches. Encryption methods like data-at-rest and data-in-transit encryption serve to secure data both during storage on servers and while they are being transmitted. Access management strategies, such as role-based access control (RBAC) and multi-factor authentication (MFA), limit access solely to authorized individuals, thereby reducing the potential for insider threats.
Data anonymization and de-identification: Anonymizing or de-identifying data represents a prevalent approach in biobanking, aiming to safeguard participant privacy while retaining data usefulness for research endeavors. However, achieving true anonymity or irreversibility poses challenges, as re-identification risks remain, especially with the proliferation of data linkage and re-identification techniques. Balancing data anonymization with data utility requires the careful consideration of anonymization methods and privacy-preserving techniques.
Data sharing and consent management: Facilitating data sharing while respecting participant consent preferences is a complex undertaking in biobanking. Ensuring that participants have meaningful control over their data and understanding how their data will be used is essential for fostering trust and transparency. Implementing robust consent management systems, including dynamic consent models and granular consent options, enables participants to specify their preferences regarding data sharing and use.
Regulatory compliance: Biobanking data management must comply with a myriad of legal and regulatory requirements governing data privacy and security, including General Data Protection Regulation (GDPR) [ 28 ], Health Insurance Portability and Accountability Act (HIPAA) [ 29 ], and other data protection laws. Adhering to regulatory standards requires implementing comprehensive data governance frameworks, conducting privacy impact assessments, and maintaining documentation of data processing activities. Failure to comply can lead to significant penalties and harm to the reputation of biobanks.
Data breach preparedness and response: Despite best efforts to prevent breaches, biobanks need to be ready to react promptly and efficiently in case of a data breach. Establishing incident response plans, including procedures for breach notification, forensic investigation, and communication with affected parties, is crucial for mitigating the impact of breaches on participant privacy and trust.
Data lifecycle management: Ensuring the effective management of data from its collection to disposal necessitates the implementation of robust data management practices that prioritize privacy and security. Implementing data minimization strategies, secure data disposal procedures, and audit trails for data access and usage enhances accountability and mitigates the risk of unauthorized data exposure

4.4. Data Governance and Regulatory Compliance

Legal and ethical frameworks: Biobanks operate within a framework of legal and ethical guidelines that govern the collection, storage, and use of biological samples and their associated data. Adherence to regulations like the GDPR and HIPAA as well as the ethical principles outlined in documents like the Declaration of Helsinki are prerequisites for the protection of participant rights and ensuring research integrity.
Informed consent and participant privacy: Obtaining informed consent from participants is a cornerstone of ethical biobanking practices, guaranteeing that individuals comprehend the objectives of data collection, the intended utilization of their data, and any potential risks inherent in the process [ 4 ]. However, obtaining meaningful consent can be challenging, especially in longitudinal studies or when data may be used for future, unforeseen research purposes. Balancing participant autonomy with the need for scientific advancement requires clear communication and consent management strategies.
Data ownership and intellectual property: Elucidating rights to data ownership and addressing intellectual property concerns is essential for resolving legal and ethical issues surrounding data usage, access, and commercialization. Biobanks often navigate complex relationships between participants, researchers, institutions, and commercial entities, necessitating clear policies and agreements regarding data ownership, sharing, and commercialization rights.
Data access and sharing policies: Establishing transparent data access and sharing policies is essential for promoting research collaboration, maximizing data utility, and ensuring equitable access to biobank resources. However, balancing openness with privacy concerns and intellectual property rights poses challenges, particularly when sharing data across international borders or with commercial partners. Implementing access control mechanisms and data use agreements helps regulate data access while protecting participant privacy and confidentiality.
Data security and confidentiality: Protecting the security and confidentiality of biobank data is a legal and ethical imperative, requiring robust data security measures and safeguards against unauthorized access or breaches. Adhering to data protection regulations like GDPR and HIPAA entails implementing encryption, access controls, and data anonymization techniques to mitigate privacy risks and safeguard participant confidentiality.
Audit and compliance monitoring: Monitoring compliance with data governance policies and regulatory requirements requires robust audit mechanisms and oversight processes. Conducting regular audits of data management practices, documentation, and security controls helps identify potential compliance gaps and mitigate risks of non-compliance. Establishing clear lines of accountability and oversight responsibilities is essential for ensuring adherence to regulatory standards.
Data retention and disposal: Developing policies for data retention and disposal is essential for effectively managing the data lifecycle and minimizing privacy risks. Determining appropriate retention periods, archival strategies, and secure data disposal procedures requires the consideration of legal requirements, research needs, and participant consent preferences. Implementing data minimization principles and regular data purging practices reduces the risk of unauthorized data exposure and facilitates compliance with data protection laws.

5. Strategies for Effective Data Management

5.1. standardization and metadata annotation.

Data standardization: Standardizing data formats, vocabularies, and ontologies is essential for ensuring consistency and interoperability across the diverse datasets collected and stored in biobanks [ 30 ]. With the adoption of common data standards and terminologies, biobanks facilitate data sharing, integration, and reusability across multiple research studies and platforms [ 31 , 32 ]. Standardization efforts encompass various aspects of data management, including sample metadata, clinical annotations, genomic data formats, and laboratory measurements [ 33 , 34 ].
Harmonization of data: Harmonizing heterogeneous datasets from different sources involves reconciling the differences in data formats, semantics, and structures to enable seamless data integration and analysis. Harmonization efforts aim to ensure that the data collected across multiple biobanks or research studies are compatible and comparable, thereby maximizing the utility of aggregated datasets for research purposes. Establishing harmonization guidelines, mapping protocols, and data transformation procedures helps address discrepancies and inconsistencies in data representation [ 35 ].
Metadata annotation: Metadata annotation provides essential context and descriptive information about biological samples and their associated data, enhancing data interpretability and usability. Metadata encompass a wide range of attributes, including sample characteristics, experimental protocols, data provenance, and quality metrics. Annotating data with standardized metadata terms and controlled vocabularies enables researchers to search, filter, and analyze data effectively, facilitating data discovery and interpretation [ 36 , 37 ].
Data integration platforms: Leveraging data integration platforms and bioinformatics tools streamlines the process of harmonizing and annotating heterogeneous datasets in biobanking. These platforms provide capabilities for data mapping, transformation, and enrichment, enabling researchers to aggregate, query, and analyze diverse datasets from multiple sources. By providing a unified interface for data access and analysis, data integration platforms promote collaboration, accelerate research discoveries, and maximize the value of biobank resources [ 38 ].
Ontology development and adoption: Ontologies play a crucial role in standardizing and formalizing knowledge representation in biobanking, enabling semantic interoperability and data integration [ 39 ]. Ontologies provide structured vocabularies and hierarchical relationships for annotating biological concepts, phenotypic traits, and experimental variables [ 40 ]. Adopting community-developed ontologies, such as the Human Phenotype Ontology (HPO) or the Experimental Factor Ontology (EFO), facilitates data annotation and enhances data interoperability across different biobanks and research domains.
Metadata quality assurance: Ensuring the quality and completeness of metadata annotations is essential for maintaining data integrity and facilitating accurate data interpretation. Metadata quality assurance measures include validation checks, consistency audits, and adherence to metadata standards and best practices. Establishing metadata curation guidelines, metadata validation rules, and quality control procedures helps mitigate errors and inconsistencies in metadata annotations, enhancing the reliability and usability of biobank data.
Community engagement and collaboration: Collaborative efforts within the scientific community are crucial for driving standardization and metadata annotation initiatives in biobanking. Engaging stakeholders, including researchers, data scientists, informaticians, and domain experts, fosters consensus building, promotes knowledge sharing, and accelerates the adoption of standardized data management practices. Community-driven initiatives, such as data standards consortia, working groups, and data harmonization projects, play a vital role in advancing data standardization and metadata annotation efforts across the biobanking community.

5.2. Data Quality Control

Data validation: Data validation verifies the data’s accuracy, consistency, and integrity through systematic checks and predefined criteria. These checks, conducted at data entry or import, identify errors, anomalies, and inconsistencies such as missing values or outliers, ensuring only high-quality data are inputted into the system.
Quality assurance protocols: Developing quality assurance protocols and standard operating procedures (SOPs) are essential for the maintenance of consistent data quality standards across biobank operations. SOPs define procedures for data collection, storage, curation, and documentation, ensuring adherence to best practices and regulatory requirements. Regular training and audits help enforce compliance with quality assurance protocols and identify areas for improvement.
Data cleaning and transformation: Data cleaning addresses errors, inconsistencies, and outliers in the dataset to enhance data quality and reliability. Cleaning procedures may include data deduplication, outlier detection, imputation of missing values, and normalization of data formats. Data transformation techniques, such as standardization or log transformation, help prepare data for analysis and mitigate biases introduced by data heterogeneity.
Standardized data entry and documentation: Standardizing data entry procedures and documentation formats promotes consistency and accuracy in data collection and annotation. Providing clear guidelines, data dictionaries, and templates for data entry facilitates uniform data capture and ensures that relevant metadata are documented consistently [ 41 , 42 ]. Validating data against predefined data standards and vocabularies further enhances data quality and interoperability.
Automated quality control checks: Implementing automated quality control checks and algorithms helps streamline data validation and cleaning processes, reducing manual effort and human errors. Automated checks may include range validation, format validation, and logical consistency checks to flag potential data anomalies in real time. Integrating automated quality control checks into data management workflows improves efficiency and ensures timely detection and resolution of data issues.
Continuous monitoring and improvement: Data quality control is an ongoing process that requires continuous monitoring and enhancement to maintain data integrity over time. Monitoring data quality metrics like data completeness, accuracy rates, and error frequencies allows biobanks to evaluate the effectiveness of quality control measures and identify areas for optimization. Establishing feedback mechanisms and quality improvement initiatives fosters a culture of continuous quality improvement and enhances the reliability of biobank data.
External quality assessment programs: Participating in external quality assessment programs and proficiency testing schemes provides independent validation of data quality and performance against established benchmarks and standards. External assessments help benchmark biobank performance, identify areas for improvement, and demonstrate compliance with regulatory requirements and accreditation standards. Engaging in collaborative quality assurance initiatives strengthens the credibility and trustworthiness of biobank data within the scientific community.

5.3. Secure Data Infrastructure

Data encryption: Deploying strong encryption methods for data, both at rest and in transit, serves to protect biobank data from unauthorized access or interception. Encryption standards such as the Advanced Encryption Standard (AES) for data storage and Transport Layer Security (TLS) for data transmission ensure that data remain encrypted and indecipherable to unauthorized parties, thus mitigating the risk of data breaches or interception during transmission.
Access control and authentication: Establishing policies for access control and authentication mechanisms is essential in governing access to biobank data, ensuring that only authorized personnel can access sensitive information. Role-based access control (RBAC), multi-factor authentication (MFA), and stringent password policies serve to limit access to data based on user roles, privileges, and authentication credentials, thereby reducing the risk of unauthorized data access or insider threats.
Data segregation and isolation: The segregation and isolation of sensitive data within secure environments, such as secure servers or dedicated data centers, help to thwart unauthorized access or tampering with biobank data. The implementation of network segmentation, firewalls, and intrusion detection systems (IDSs) effectively separates sensitive data from less secure networks, minimizing the impact of security breaches or cyberattacks on biobank operations.
Secure data storage and backup: Employing secure data storage solutions, such as encrypted databases or cloud storage with integrated encryption and access controls, serves to safeguard biobank data from loss, theft, or corruption. Regular data backups and comprehensive disaster recovery plans ensure data resilience and enable swift data recovery in the event of hardware failures, natural disasters, or ransomware attacks, thereby minimizing downtime and potential data loss.
Data masking and anonymization: Applying data masking or anonymization techniques to sensitive data helps protect participant privacy and confidentiality while preserving data utility for research purposes. Masking personally identifiable information (PII) or de-identifying data before sharing or analysis reduces the risk of re-identification and unauthorized disclosure of sensitive information, ensuring compliance with privacy regulations and ethical guidelines.
Auditing and monitoring: Integrating robust auditing and monitoring mechanisms empowers biobanks to monitor data access, usage, and modifications, facilitating accountability and compliance with data governance policies. Audit trails, logging mechanisms, and real-time monitoring tools offer visibility into data activities and aid in detecting anomalous behavior or security incidents, enabling prompt response and remediation.
Security awareness and training: Promoting security awareness and providing training to personnel on security best practices, data handling procedures, and incident response protocols is crucial for fostering a culture of security within the biobank. Educating staff about potential security risks, phishing attacks, and social engineering tactics helps mitigate human errors and strengthens defenses against cybersecurity threats, enhancing overall data security posture.
Regulatory compliance and certifications: Ensuring compliance with regulatory requirements, such as GDPR, HIPAA, and ISO/IEC 27001 [ 9 ], demonstrates commitment to data security and privacy best practices. Obtaining certifications and undergoing independent audits validate a biobank’s adherence to industry standards and regulatory guidelines, instilling confidence in data security practices among stakeholders, researchers, and participants.

5.4. Data Sharing and Collaboration

Promoting open data sharing: Embracing a culture of open data sharing facilitates transparency, reproducibility, and innovation in biomedical research [ 44 ]. Biobanks can promote open data sharing by adopting data-sharing policies, releasing datasets to public repositories, and adhering to data sharing mandates from funding agencies or regulatory bodies. Open data sharing fosters collaboration, accelerates scientific progress, and increases the impact of research findings by enabling broader access to biobank resources.
Establishing data access policies: Developing clear and transparent data access policies helps regulate access to biobank data while balancing privacy concerns, data governance requirements, and research needs [ 45 ]. Data access policies outline procedures for requesting, accessing, and sharing data, specifying eligibility criteria, data use restrictions, and compliance requirements. Implementing access control mechanisms, such as data use agreements and data access committees, ensures that data are accessed and used responsibly and ethically.
Creating collaborative platforms: Establishing collaborative platforms and data-sharing portals facilitates communication, collaboration, and data exchange among researchers, biobanks, and other stakeholders. Collaborative platforms provide centralized access to data, tools, and resources, enabling researchers to discover, access, and analyze biobank data efficiently [ 46 ]. These platforms may include data repositories, virtual research environments, or collaborative networks tailored to specific research domains or disease areas.
Data harmonization and integration: Harmonizing and integrating heterogeneous datasets from multiple biobanks or research studies enhances data interoperability and facilitates cross-study comparisons and meta-analyses. Collaborative efforts to standardize data formats, metadata annotations, and ontologies streamline data integration processes and enable researchers to aggregate, analyze, and interpret data from diverse sources effectively. Data harmonization initiatives promote data reuse, reduce redundancy, and maximize the value of biobank resources for research [ 3 ].
Facilitating data-sharing agreements: Negotiating data-sharing agreements and collaborations with external partners, including academic institutions, industry partners, and international consortia, expands research opportunities and promotes knowledge exchange [ 47 ]. Data-sharing agreements delineate the terms and conditions governing data sharing, including data ownership, intellectual property rights, and data use restrictions, ensuring that data are shared responsibly and in compliance with legal and ethical requirements [ 48 ].
Enabling federated data analysis: Federated data analysis approaches enable collaborative analysis of distributed datasets across multiple biobanks or research sites while preserving data privacy and security. Federated analysis platforms facilitate data aggregation, analysis, and knowledge discovery without centrally pooling or sharing sensitive data. By leveraging federated analysis techniques, researchers can collaborate on large-scale data analyses, identify patterns, and derive insights from diverse datasets while protecting participant privacy and data confidentiality.
Promoting data citation and attribution: Encouraging data citation and attribution practices acknowledges the contributions of data contributors, promotes data reuse, and enhances research reproducibility and transparency. Providing persistent identifiers (DOIs) for datasets, citing data sources in publications, and adhering to data citation standards facilitate the proper attribution and recognition of data contributors. Data citation policies and guidelines promote responsible data use and incentivize data sharing within the research community.

6. Literature Reviews

7. future directions, 7.1. integration of advanced technologies.

Blockchain technology: Blockchain technology provides a decentralized and tamper-resistant platform for secure and transparent data management in biobanking [ 79 ]. By utilizing blockchain’s unalterable ledger and cryptographic hashing, biobanks can ensure data integrity, traceability, and auditability throughout the data lifecycle. Blockchain-based solutions enable secure data sharing, provenance tracking, and consent management, fostering trust among data contributors, researchers, and participants [ 80 ].
Post-quantum cryptography and quantum-secure communication: To enhance data security against emerging threats posed by quantum computing, the integration of post-quantum cryptography (PQC) and quantum-secure communication technologies offers a promising path forward. These approaches are designed to counteract vulnerabilities that quantum computing could exploit, potentially compromising existing cryptographic systems. ○ Post-quantum cryptography: This involves developing cryptographic algorithms that are designed to stay secure even when quantum computers are in use. Unlike classical computers that use binary bits, quantum computers utilize qubits, which can exist in multiple states at the same time due to the principle of quantum superposition, allowing for significantly faster computations. This capability poses a threat to cryptographic methods such as RSA and Elliptic Curve Cryptography (ECC), which depend on the difficulty of solving mathematical problems like factoring large numbers or calculating discrete logarithms; these are tasks that quantum algorithms can handle much more efficiently. In biobanking, adopting PQC is vital to protect the vast amounts of sensitive personal and genetic data stored in these repositories. Given the potential for cyberattacks targeting personal identifiers and genetic sequences, PQC algorithms—such as those based on lattice-based cryptography, hash-based signatures, and multivariate quadratic equations—are being developed and standardized. Implementing these algorithms will help ensure that sensitive information remains secure, even as quantum computing becomes more widespread [ 81 ]. ○ Quantum-secure communication: Quantum-secure communication uses the principles of quantum mechanics to safeguard data transmissions. Key techniques encompass Quantum Key Distribution (QKD) and quantum entanglement. QKD enables two parties to create a shared secret key protected by quantum laws. Any eavesdropping attempts would disturb the quantum states, making the intrusion detectable. For biobanks, using quantum-secure communication methods can greatly improve the protection of sensitive data during transmission. Given the frequent exchange of personal and genetic information among researchers, institutions, and regulatory bodies, ensuring the security and confidentiality of these communications is crucial. Technologies like QKD provide strong defenses against interception and tampering, thereby enhancing the security of data exchanges across networks [ 82 , 83 ].
Artificial intelligence and machine learning: Artificial intelligence and machine learning algorithms enable biobanks to analyze large-scale datasets [ 84 , 85 ], identify patterns, and extract actionable insights for precision medicine and personalized healthcare [ 86 ]. AI-driven approaches facilitate data mining, predictive modeling, and biomarker discovery, accelerating the translation of biomedical research into clinical applications [ 87 ]. AI-powered decision support systems aid in clinical diagnosis, treatment optimization, and patient stratification based on genetic and clinical data [ 88 , 89 ].
Federated learning: Federated learning facilitates collaborative model training across dispersed data sources while upholding data privacy and confidentiality. In biobanking, federated learning facilitates multi-center data analysis, enabling researchers to aggregate and analyze data from disparate biobanks without centrally pooling sensitive data. Federated learning platforms empower biobanks to collaborate on large-scale data analyses, share insights, and derive collective knowledge while protecting participant privacy and data security.
Genomic data analysis: Advances in genomic technologies, such as next-generation sequencing (NGS) and single-cell sequencing, revolutionize genomic data analysis in biobanking [ 90 ]. High-throughput sequencing platforms generate vast amounts of genomic data, enabling the comprehensive characterization of genetic variation, gene expression, and epigenetic modifications. Bioinformatics tools and cloud-based analysis platforms facilitate genomic data analysis [ 13 , 91 ], variant interpretation, and genotype–phenotype association studies, advancing our understanding of complex diseases and guiding personalized medicine approaches [ 33 ].
Omics integration: Integrating multi-omics data, including genomics, transcriptomics, proteomics, and metabolomics, offers holistic insights into biological systems and disease mechanisms [ 92 ]. Integrative omics analysis enables researchers to elucidate molecular pathways, identify biomarkers, and uncover therapeutic targets for precision medicine interventions [ 48 ]. Integrative bioinformatics approaches, such as pathway analysis, network modeling, and data fusion techniques, enhance data interpretation and facilitate discovery-driven research in biobanking [ 93 ].
Biobanking informatics platforms: Biobanking informatics platforms provide integrated solutions for data management, analysis, and collaboration, streamlining biobank operations and supporting research workflows [ 45 , 94 , 95 ]. These platforms offer features such as sample tracking, metadata management, data curation, and analysis tools tailored to biobanking needs [ 26 , 96 , 97 ]. Cloud-based informatics platforms enable scalable and secure data storage, analysis, and sharing, empowering biobanks to leverage advanced technologies and collaborate with researchers worldwide [ 98 ].
Emerging technologies: Emerging technologies, such as single-cell analysis, spatial transcriptomics, and organoid modeling, offer novel approaches for studying cellular heterogeneity, tissue architecture, and disease mechanisms in biobanking. These technologies enable researchers to capture fine-grained molecular profiles, spatially resolve cellular interactions, and model complex biological processes in vitro. Integrating emerging technologies into biobanking workflows expands research capabilities, facilitates disease modeling, and accelerates drug discovery efforts [ 99 ].

7.2. Long-Term Data Sustainability

Data stewardship and governance: Establishing robust data stewardship and governance frameworks is essential for ensuring the long-term sustainability of biobank data [ 100 ]. Data stewardship involves the responsible management, curation, and preservation of data assets [ 101 ], while governance encompasses policies, procedures, and oversight mechanisms to ensure compliance with legal, ethical, and regulatory requirements. Implementing clear roles, responsibilities, and accountability structures fosters a culture of data stewardship and ensures the continuity of data management practices over time.
Data preservation and archiving: Preserving data integrity and accessibility over the long term requires establishing archival strategies and preservation methods tailored to the unique characteristics of biobank data. Archiving data in secure, redundant storage systems, such as digital repositories or cloud-based storage solutions, safeguards against data loss, hardware failures, or technological obsolescence. Implementing data backup, versioning, and migration strategies ensures data resilience and facilitates data recovery in the event of system failures or disasters.
Metadata standardization and documentation: Standardizing metadata formats, documentation practices, and data descriptors enhances data discoverability, interoperability, and usability over time [ 34 ]. Documenting metadata attributes, data provenance, and data processing protocols ensures that data remain comprehensible and interpretable by future users. Metadata standards, such as the Minimum Information About a Biobank (MIABIS) or the FAIR (Findable, Accessible, Interoperable, and Reusable) principles [ 30 , 101 ], guide metadata documentation and promote data sustainability by enhancing data reuse and interoperability.
Data quality assurance and maintenance: Maintaining data quality and reliability is essential for preserving the value and integrity of biobank data over time. Implementing data quality assurance measures, such as regular audits, validation checks, and data cleaning procedures, ensures that data remain accurate, consistent, and fit for purpose. Ongoing surveillance of data quality metrics and performance indicators allows biobanks to detect and rectify instances of data degradation or quality issues proactively, thereby sustaining data utility and trustworthiness.
Data security and privacy protection: Safeguarding data security and protecting participant privacy are paramount considerations for ensuring the long-term sustainability of biobank data [ 102 ]. Deploying strong data security measures, encryption techniques, access controls, and privacy safeguards helps alleviate the potential for data breaches, unauthorized access, or the misuse of data. Adhering to data protection laws, ethical guidelines, and best practices for data anonymization and de-identification ensures that data remain ethically and legally compliant while supporting data sharing and research collaboration.
Community engagement and collaboration: Engaging stakeholders, including researchers, participants, funding agencies, and regulatory bodies, fosters collaboration, promotes transparency, and ensures the continued relevance and sustainability of biobank data resources. Soliciting feedback, addressing community needs, and involving stakeholders in decision-making processes empower stakeholders to contribute to data governance, policy development, and resource allocation efforts [ 103 , 104 ]. Collaborative initiatives, such as data-sharing consortia, working groups, and community-driven projects, foster a sense of ownership and collective responsibility for sustaining biobank data resources [ 105 ].

7.3. Ethical and Social Implications

Informed consent and participant autonomy: Upholding the principles of informed consent and participant autonomy is paramount in biobanking to ensure that individuals have the right to make informed decisions about the use of their biological samples and data [ 107 ]. Future directions should focus on enhancing consent processes, providing clear and understandable information to participants, and offering opportunities for dynamic consent, allowing individuals to update their preferences over time [ 108 , 109 ].
Privacy and data confidentiality: Protecting participant privacy and ensuring the confidentiality of sensitive data are ethical imperatives in biobanking [ 110 ]. As biobanks collect and store large volumes of personal health information and genetic data, future directions should prioritize robust data security measures, anonymization techniques, and encryption protocols to mitigate privacy risks and prevent unauthorized access or breaches.
Equitable access and benefit sharing: Addressing issues of equity and justice in biobanking involves ensuring that the benefits derived from research are shared equitably among participants, communities, and stakeholders. Future directions should promote transparent and fair access to biobank resources, prioritize the inclusion of under-represented populations in research, and establish mechanisms for benefit sharing, such as community engagement initiatives, research partnerships, and capacity-building programs.
Data governance and oversight: Implementing effective data governance mechanisms and oversight frameworks is essential for ensuring responsible and ethical conduct in biobanking. Future directions should focus on developing robust data governance policies, establishing independent oversight bodies, and fostering collaboration among stakeholders to promote accountability, transparency, and ethical decision making in data management and research practices.
Cultural sensitivity and respect for diversity: Recognizing and respecting cultural differences, values, and beliefs is essential in biobanking to ensure that research practices are culturally sensitive and inclusive [ 108 ]. Future directions should prioritize culturally tailored approaches to consent processes, engage with diverse communities in research planning and implementation, and address cultural concerns and preferences regarding data sharing, storage, and use [ 111 ].
Public engagement and trust building: Building public trust and fostering the meaningful engagement of stakeholders are critical for success and sustainability of biobanking initiatives. Future directions should emphasize transparency, communication, and dialogue with the public, raise awareness about the benefits and risks of biobanking, and solicit input from diverse perspectives to inform decision-making processes and research priorities.
Ethical use of biobank resources: Ensuring that biobank resources are used ethically and responsibly requires adherence to ethical guidelines, professional standards, and regulatory requirements. Future directions should prioritize ethical considerations in research design, data analysis, and the dissemination of findings, promote responsible conduct of research, and establish mechanisms for ethical review and oversight to safeguard participant welfare and uphold research integrity.

8. Conclusions

Author contributions, institutional review board statement, informed consent statement, data availability statement, acknowledgments, conflicts of interest.

Malsagova, K.; Kopylov, A.; Stepanov, A.; Butkova, T.; Sinitsyna, A.; Izotov, A.; Kaysheva, A. Biobanks—A Platform for Scientific and Biomedical Research. Diagnostics 2020 , 10 , 485. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Harati, M.D.; Williams, R.R.; Movassaghi, M.; Hojat, A.; Lucey, G.M.; Yong, W.H. An Introduction to Starting a Biobank ; Springer: New York, NY, USA, 2019; pp. 7–16. [ Google Scholar ]
Coppola, L.; Cianflone, A.; Grimaldi, A.M.; Incoronato, M.; Bevilacqua, P.; Messina, F.; Baselice, S.; Soricelli, A.; Mirabelli, P.; Salvatore, M. Biobanking in health care: Evolution and future directions. J. Transl. Med. 2019 , 17 , 172. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Annaratone, L.; De Palma, G.; Bonizzi, G.; Sapino, A.; Botti, G.; Berrino, E.; Mannelli, C.; Arcella, P.; Di Martino, S.; Steffan, A.; et al. Basic principles of biobanking: From biological samples to precision medicine for patients. Virchows Arch. 2021 , 479 , 233–246. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Lieb, W.; Strathmann, E.A.; Röder, C.; Jacobs, G.; Gaede, K.I.; Richter, G.; Illig, T.; Krawczak, M. Population-Based Biobanking. Genes 2024 , 15 , 66. [ Google Scholar ] [ CrossRef ]
Lermen, D.; Gwinner, F.; Bartel-Steinbach, M.; Mueller, S.C.; Habermann, J.K.; Balwir, M.-B.; Smits, E.; Virgolino, A.; Fiddicke, U.; Berglund, M.; et al. Towards Harmonized Biobanking for Biomonitoring: A Comparison of Human Biomonitoring-Related and Clinical Biorepositories. Biopreserv. Biobank. 2020 , 18 , 122–135. [ Google Scholar ] [ CrossRef ]
Zeh, R.M.; Glisic, M.; Capossela, S.; Bertolo, A.; Valido, E.; Jordan, X.; Hund-Georgiadis, M.; Pannek, J.; Eriks-Hoogland, I.; Stucki, G.; et al. The Swiss Spinal Cord Injury Cohort Study (SwiSCI) biobank: From concept to reality. Spinal Cord 2024 , 62 , 117–124. [ Google Scholar ] [ CrossRef ]
Poline, J.-B.; Kennedy, D.N.; Sommer, F.T.; Ascoli, G.A.; Van Essen, D.C.; Ferguson, A.R.; Grethe, J.S.; Hawrylycz, M.J.; Thompson, P.M.; Poldrack, R.A.; et al. Is Neuroscience FAIR? A Call for Collaborative Standardisation of Neuroscience Data. Neuroinformatics 2022 , 20 , 507–512. [ Google Scholar ] [ CrossRef ]
De Blasio, P.; Biunno, I. New Challenges for Biobanks: Accreditation to the New ISO 20387:2018 Standard Specific for Biobanks. BioTech 2021 , 10 , 13. [ Google Scholar ] [ CrossRef ]
Lin, Z.; Li, Y.; Tang, S.; Deng, Q.; Jiang, J.; Zhou, C. Comparative analysis of genomic profiles between tissue-based and plasma-based next-generation sequencing in patients with non-small cell lung cancer. Lung Cancer 2023 , 182 , 107282. [ Google Scholar ] [ CrossRef ]
Yoshida, T.; Kates, M.; Fujita, K.; Bivalacqua, T.J.; McConkey, D.J. Predictive biomarkers for drug response in bladder cancer. Int. J. Urol. 2019 , 26 , 1044–1053. [ Google Scholar ] [ CrossRef ]
Beier, K.; Nussbeck, S.; Wemheuer, W. Why brain banking should be regarded as a special type of biobanking: Ethical, practical, and data-management challenges. J. Biorepository Sci. Appl. Med. 2015 , 3 , 3–14. [ Google Scholar ] [ CrossRef ]
Kinkorová, J.; Topolčan, O. Biobanks in the era of big data: Objectives, challenges, perspectives, and innovations for predictive, preventive, and personalised medicine. EPMA J. 2020 , 11 , 333–341. [ Google Scholar ] [ CrossRef ]
Gabelloni, M.; Faggioni, L.; Borgheresi, R.; Restante, G.; Shortrede, J.; Tumminello, L.; Scapicchio, C.; Coppola, F.; Cioni, D.; Gómez-Rico, I.; et al. Bridging gaps between images and data: A systematic update on imaging biobanks. Eur. Radiol. 2022 , 32 , 3173–3186. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Littlejohns, T.J.; Holliday, J.; Gibson, L.M.; Garratt, S.; Oesingmann, N.; Alfaro-Almagro, F.; Bell, J.D.; Boultwood, C.; Collins, R.; Conroy, M.C.; et al. The UK Biobank imaging enhancement of 100,000 participants: rationale, data collection, management and future directions. Nat. Commun. 2020 , 11 , 2624. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Jalloul, R.; Chethan, H.K.; Alkhatib, R. A Review of Machine Learning Techniques for the Classification and Detection of Breast Cancer from Medical Images. Diagnostics 2023 , 13 , 2460. [ Google Scholar ] [ CrossRef ]
Kondylakis, H.; Kalokyri, V.; Sfakianakis, S.; Marias, K.; Tsiknakis, M.; Jimenez-Pastor, A.; Camacho-Ramos, E.; Blanquer, I.; Segrelles, J.D.; López-Huguet, S.; et al. Data infrastructures for AI in medical imaging: A report on the experiences of five EU projects. Eur. Radiol. Exp. 2023 , 7 , 20. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Esposito, G.; Pagliari, G.; Randon, M.; Mirabelli, P.; Lavitrano, M.; Aiello, M.; Salvatore, M. BCU Imaging Biobank, an Innovative Digital Resource for Biomedical Research Collecting Imaging and Clinical Data From Human Healthy and Pathological Subjects. Open J. Bioresour. 2021 , 8 , 1. [ Google Scholar ] [ CrossRef ]
Dregely, I.; Prezzi, D.; Kelly-Morland, C.; Roccia, E.; Neji, R.; Goh, V. Imaging biomarkers in oncology: Basics and application to MRI. J. Magn. Reson. Imaging 2018 , 48 , 13–26. [ Google Scholar ] [ CrossRef ]
Aiello, M.; Baldi, D.; Esposito, G.; Valentino, M.; Randon, M.; Salvatore, M.; Cavaliere, C. Evaluation of AI-based segmentation tools for COVID-19 lung lesions on conventional and ultra-low dose CT scans. Dose-Response 2022 , 20 , 15593258221082896. [ Google Scholar ] [ CrossRef ]
Olund, G.; Lindqvist, P.; Litton, J.E. BIMS: An information management system for biobanking in the 21st century. IBM Systems Journal 2007 , 46 , 171–182. [ Google Scholar ] [ CrossRef ]
Wang, X.; Williams, C.; Liu, Z.H.; Croghan, J. Big data management challenges in health research—A literature review. Brief. Bioinform. 2019 , 20 , 156–167. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Bull, S.; Bhagwandin, N. The ethics of data sharing and biobanking in health research. Wellcome Open Res. 2020 , 5 , 270. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Rychnovská, D. Anticipatory Governance in Biobanking: Security and Risk Management in Digital Health. Sci. Eng. Ethics 2021 , 27 , 30. [ Google Scholar ] [ CrossRef ]
In den Bäumen, T.S.; Paci, D.; Ibarreta, D. Data Protection and Sample Management in Biobanking—A legal dichotomy. Genom. Soc. Policy 2010 , 6 , 33. [ Google Scholar ] [ CrossRef ]
Jacotot, L.; Woodward, M.; de Montalier, A.; Vaglio, P. Utilizing Modular Biobanking Software in Different Types of Biobanking Activities. Biopreserv. Biobank. 2022 , 20 , 417–422. [ Google Scholar ] [ CrossRef ]
General Data Protection Regulation (GDPR). General Data Protection Regulation (GDPR)-Official Legal Text. Available online: https://gdpr-info.eu/ (accessed on 23 April 2023).
Albrecht, J.P. How the GDPR will change the world. Eur. Data Prot. L. Rev. 2016 , 2 , 287. [ Google Scholar ] [ CrossRef ]
Nass, S.J.; Levit, L.A.; Gostin, L.O. (Eds.) Beyond the HIPAA Privacy Rule: Enhancing Privacy, Improving Health through Research ; National Academies Press (US): Washington, DC, USA, 2009. [ Google Scholar ] [ PubMed ]
Scapicchio, C.; Gabelloni, M.; Forte, S.M.; Alberich, L.C.; Faggioni, L.; Borgheresi, R.; Erba, P.; Paiar, F.; Marti-Bonmati, L.; Neri, E. DICOM-MIABIS integration model for biobanks: A use case of the EU PRIMAGE project. Eur. Radiol. Exp. 2021 , 5 , 20. [ Google Scholar ] [ CrossRef ]
Stöhr, M.R.; Günther, A.; Majeed, R.W. The Collaborative Metadata Repository (CoMetaR) Web App: Quantitative and Qualitative Usability Evaluation. JMIR Med. Inform. 2021 , 9 , e30308. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Brancato, V.; Esposito, G.; Coppola, L.; Cavaliere, C.; Mirabelli, P.; Scapicchio, C.; Borgheresi, R.; Neri, E.; Salvatore, M.; Aiello, M. Standardizing digital biobanks: Integrating imaging, genomic, and clinical data for precision medicine. J. Transl. Med. 2024 , 22 , 136. [ Google Scholar ] [ CrossRef ]
Müller, H.; Dagher, G.; Loibner, M.; Stumptner, C.; Kungl, P.; Zatloukal, K. Biobanks for life sciences and personalized medicine: Importance of standardization, biosafety, biosecurity, and data management. Curr. Opin. Biotechnol. 2020 , 65 , 45–51. [ Google Scholar ] [ CrossRef ]
Yeh, C.-Y.; Peng, S.-J.; Yang, H.C.; Islam, M.; Poly, T.N.; Hsu, C.-Y.; Huff, S.M.; Chen, H.-C.; Lin, M.-C. Logical Observation Identifiers Names and Codes (LOINC®) Applied to Microbiology: A National Laboratory Mapping Experience in Taiwan. Diagnostics 2021 , 11 , 8. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Sass, J.; Bartschke, A.; Lehne, M.; Essenwanger, A.; Rinaldi, E.; Rudolph, S.; Heitmann, K.U.; Vehreschild, J.J.; von Kalle, C.; Thun, S. The German Corona Consensus Dataset (GECCO): A standardized dataset for COVID-19 research in university medicine and beyond. BMC Med. Inform. Decis. Mak. 2020 , 20 , 341. [ Google Scholar ] [ CrossRef ]
Kreimeyer, K.; Foster, M.; Pandey, A.; Arya, N.; Halford, G.; Jones, S.F. Natural language processing systems for capturing and standardizing unstructured clinical information: A systematic review. J Biomed Inf. 2017 , 73 , 14–29. [ Google Scholar ] [ CrossRef ]
Ghanem, F.A.; Padma, M.C.; Alkhatib, R. Automatic Short Text Summarization Techniques in Social Media Platforms. Future Internet 2023 , 15 , 311. [ Google Scholar ] [ CrossRef ]
Schüttler, C.; Huth, V.; von Jagwitz-Biegnitz, M.; Lablans, M.; Prokosch, H.-U.; Griebel, L. A Federated Online Search Tool for Biospecimens (Sample Locator): Usability Study. J. Med. Internet Res. 2020 , 22 , e17739. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Shi, Z.; Traverso, A.; Soest, J.; Dekker, A.; Wee, L. Technical note: Ontology-guided radiomics analysis workflow (O-RAW). Med. Phys. 2019 , 46 , 5677–5684. [ Google Scholar ] [ CrossRef ]
Luschi, A.; Petraccone, C.; Fico, G.; Pecchia, L.; Iadanza, E. Semantic Ontologies for Complex Healthcare Structures: A Scoping Review. IEEE Access 2023 , 11 , 19228–19246. [ Google Scholar ] [ CrossRef ]
Goldberg, I.G.; Allan, C.; Burel, J.M.; Creager, D.; Falconi, A.; Hochheiser, H. The Open Microscopy Environment (OME) Data Model and XML file: Open tools for informatics and quantitative analysis in biological imaging. Genome Biol 2005 , 6 , R47. [ Google Scholar ] [ CrossRef ]
Alkhatib, R.; Scholl, M.H. CXQU: A compact XML storage for efficient query and update processing. In Proceedings of the 2008 Third International Conference on Digital Information Management, London, UK, 13–16 November 2008; pp. 605–612. [ Google Scholar ]
Santhosh, B. Internet of Medical Things in Secure Assistive Technologies. In AI-Based Digital Health Communication for Securing Assistive Systems ; IGI Global: Hershey, PA, USA, 2023; pp. 244–270. [ Google Scholar ]
Auray-Blais, C.; Patenaude, J. A biobank management model applicable to biomedical research. BMC Med. Ethics 2006 , 7 , 4. [ Google Scholar ] [ CrossRef ]
Reihs, R.; Proynova, R.; Maqsood, S.; Ataian, M.; Lablans, M.; Quinlan, P.R.; Lawrence, E.; Bowman, E.; van Enckevort, E.; Bučík, D.F.; et al. BBMRI-ERIC Negotiator: Implementing Efficient Access to Biobanks. Biopreserv. Biobank. 2021 , 19 , 414–421. [ Google Scholar ] [ CrossRef ]
Herz, C.; Fillion-Robin, J.-C.; Onken, M.; Riesmeier, J.; Lasso, A.; Pinter, C.; Fichtinger, G.; Pieper, S.; Clunie, D.; Kikinis, R.; et al. dcmqi: An Open Source Library for Standardized Communication of Quantitative Image Analysis Results Using DICOM. Cancer Res. 2017 , 77 , e87–e90. [ Google Scholar ] [ CrossRef ]
Eklund, N.; Andrianarisoa, N.H.; van Enckevort, E.; Anton, G.; Debucquoy, A.; Müller, H.; Zaharenko, L.; Engels, C.; Ebert, L.; Neumann, M.; et al. Extending the Minimum Information About BIobank Data Sharing Terminology to Describe Samples, Sample Donors, and Events. Biopreserv. Biobank. 2020 , 18 , 155–164. [ Google Scholar ] [ CrossRef ]
Chervitz, S.A.; Deutsch, E.W.; Field, D.; Parkinson, H.; Quackenbush, J.; Rocca-Serra, P. Data standards for Omics data: The basis of data sharing and reuse. Methods Mol. Biol. 2011 , 719 , 31–69. [ Google Scholar ] [ PubMed ]
Xu, W.; Liang, X.; Chen, L.; Hong, W.; Hu, X. Biobanks in chronic disease management: A comprehensive review of strategies, challenges, and future directions. Heliyon 2024 , 10 , e32063. [ Google Scholar ] [ CrossRef ]
Sánchez-López, A.M.; Catalina, P.; Franco, F.; Panadero-Fajardo, S.; Rejón, J.D.; Romero-Sánchez, M.C.; Puerta-Puerta, J.M.; Aguilar-Quesada, R. Data Model for the Comprehensive Management of Biobanks and Its Contribution to Personalized Medicine. J. Pers. Med. 2024 , 14 , 668. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Petersen, E.; Chudakova, D.; Shabalina, E.; Shiryaev, A.; Sukortseva, N.; Zhemerikin, G.; Karalkin, P.; Reshetov, I. Biobanks as an important tool in modern translational oncology. Biol. Commun. 2022 , 67 , 301–311. [ Google Scholar ] [ CrossRef ]
Goisauf, M.; Martin, G.; Bentzen, H.B.; Budin-Ljøsne, I.; Ursin, L.; Durnová, A.; Leitsalu, L.; Smith, K.; Casati, S.; Lavitrano, M.; et al. Data in question: A survey of European biobank professionals on ethical, legal and societal challenges of biobank research. PLoS ONE 2019 , 14 , e0221496. [ Google Scholar ]
Ampavathi, A.; T, V.S. Research challenges and future directions towards medical data processing. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 2022 , 10 , 633–652. [ Google Scholar ] [ CrossRef ]
Leusmann, P.; Veeck, J.; Jäkel, J.; Dahl, E.; Knüchel-Clarke, R.; Spreckelsen, C. Towards sustainable data management in professional biobanking. In eHealth2015–Health Informatics Meets eHealth ; IOS Press: Amsterdam, The Netherlands, 2015; pp. 94–102. [ Google Scholar ]
Vaught, J.; Hainaut, P.; Pasterk, M.; Zatloukal, K. The Future of Biobanking: Meeting Tomorrow’s Challenges. In Biobanking of Human Biospecimens ; Springer: Cham, Switzerland, 2021; pp. 187–197. [ Google Scholar ] [ CrossRef ]
Eder, J.; Shekhovtsov, V.A. Managing the Quality of Data and Metadata for Biobanks. In International Conference on Future Data and Security Engineering ; Springer Nature: Singapore, 2022; pp. 52–69. [ Google Scholar ]
Shekhovtsov, V.A.; Eder, J. Metadata Quality for Biobanks. Appl. Sci. 2022 , 12 , 9578. [ Google Scholar ] [ CrossRef ]
Mate, S.; Kampf, M.; Rödle, W.; Kraus, S.; Proynova, R.; Silander, K.; Ebert, L.; Lablans, M.; Schüttler, C.; Knell, C.; et al. Pan-European Data Harmonization for Biobanks in ADOPT BBMRI-ERIC. Appl. Clin. Inform. 2019 , 10 , 679–692. [ Google Scholar ] [ CrossRef ]
Assareh, H.; Waterhouse, M.A.; Moser, C.; Brighouse, R.D.; Foster, K.A.; Smith, I.R.; Mengersen, K. Data Quality Improvement in Clinical Databases Using Statistical Quality Control: Review and Case Study. Ther. Innov. Regul. Sci. 2013 , 47 , 70–81. [ Google Scholar ] [ CrossRef ]
Morehouse, K.N.; Kurdi, B.; Nosek, B.A. Responsible data sharing: Identifying and remedying possible re-identification of human participants. Am. Psychol. 2024 , 5 . Advance online publication . [ Google Scholar ] [ CrossRef ] [ PubMed ]
Thorogood, A. Population Neuroscience: Strategies to Promote Data Sharing While Protecting Privacy. In Current Topics in Behavioral Neurosciences ; Springer: Berlin/Heidelberg, Germany, 2024. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Li, W.; Li, Y.; Zheng, C.; He, R. Blockchain-based Model for Privacy-enhanced Data Sharing. In Proceedings of the 2023 10th International Conference on Dependable Systems and Their Applications (DSA), Tokyo, Japan, 10–11 August 2023; pp. 406–417. [ Google Scholar ]
Molnár, V.; Sági, J.C.; Molnár, M.J. Az érzékeny kutatási adatok megosztása a személyre szabott orvoslás gyakorlatában. Orvosi Hetil. 2023 , 164 , 811–819. [ Google Scholar ] [ CrossRef ]
Kvale, L.H.; Pharo, N.; Darch, P. Sharing Qualitative Interview Data in Dialogue with Research Participants. Proc. Assoc. Inf. Sci. Technol. 2023 , 60 , 223–232. [ Google Scholar ] [ CrossRef ]
Tzortzatou-Nanopoulou, O.; Akyüz, K.; Goisauf, M.; Kozera, Ł.; Mežinska, S.; Mayrhofer, M.T.; Slokenberga, S.; Reichel, J.; Croxton, T.; Ziaka, A.; et al. Ethical, legal, and social implications in research biobanking: A checklist for navigating complexity. Dev. World Bioeth. 2023 , 7 , 1–12. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Ranasinghe, S.; Pichler, H.; Eder, J. Report on Data Quality in Biobanks: Problems, Issues, State-of-the-Art. arXiv 2018 , arXiv:1812.10423. [ Google Scholar ]
Mirkes, E.; Coats, T.; Levesley, J.; Gorban, A. Handling missing data in large healthcare dataset: A case study of unknown trauma outcomes. Comput. Biol. Med. 2016 , 75 , 203–216. [ Google Scholar ] [ CrossRef ]
Heymans, M.W.; Twisk, J.W. Handling missing data in clinical research. J. Clin. Epidemiol. 2022 , 151 , 185–188. [ Google Scholar ] [ CrossRef ]
Georgiev, A.; Valkanov, V. Custom data quality mechanism in Data Warehouse facilitated by data integrity checks. Math. Educ. Math. 2024 , 53 , 67–75. [ Google Scholar ] [ CrossRef ]
Thompson, R. Ethical and Governance Challenges in Population Biobanking: The Case of the Global Anti-Doping Administration & Management System. Ph.D. Thesis, Swansea University, Swansea, UK, 2022. [ Google Scholar ]
Vodosin, P.; Jorgensen, A.K.; Mendy, M.; Kozlakidis, Z.; Caboux, E.; Zawati, M.H. A Review of Regulatory Frameworks Governing Biobanking in the Low and Middle Income Member Countries of BCNet. Biopreserv. Biobank. 2021 , 19 , 444–452. [ Google Scholar ] [ CrossRef ]
Maseme, M. Ethical Considerations for Health Research Data Governance. In Data Integrity and Data Governance ; IntechOpen: London, UK, 2023. [ Google Scholar ]
Kumar, B.S. Introductory Chapter: Data Integrity and Data Governance. In Data Integrity and Data Governance ; IntechOpen: London, UK, 2023. [ Google Scholar ]
Brall, C.; Berlin, C.; Zwahlen, M.; Vayena, E.; Egger, M.; Ormond, K.E. Public preferences towards data management and governance in Swiss biobanks: Results from a nationwide survey. BMJ Open 2022 , 12 , e060844. [ Google Scholar ] [ CrossRef ]
Sanchini, V.; Marelli, L.; Monturano, M.; Bonizzi, G.; Peruzzotti, G.; Orecchia, R.; Pravettoni, G. A comprehensive ethics and data governance framework for data-intensive health research: Lessons from an Italian cancer research institute. Account. Res. 2023 , 1–18. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Schüttler, C.; Buschhüter, N.; Döllinger, C.; Ebert, L.; Hummel, M.; Linde, J.; Prokosch, H.; Proynova, R.; Lablans, M. Anforderungen an eine standortübergreifende Biobanken-IT-Infrastruktur. Der Pathol. 2018 , 39 , 289–296. [ Google Scholar ] [ CrossRef ]
Rajeswari, J.; Jagannath, M. Advances in biomedical signal and image processing—A systematic review. Inf. Med. Unlocked 2017 , 8 , 13–19. [ Google Scholar ] [ CrossRef ]
Bonizzi, G.; Capra, M.; Cassi, C.; Taliento, G.; Pala, O.; Sajjadi, E.; Venetis, K.; Ivanova, M.; Monturano, M.; Renne, G.; et al. Biobank for Translational Medicine: Standard Operating Procedures for Optimal Sample Management. J. Vis. Exp. 2022 , 189 , e63950. [ Google Scholar ]
Stanescu, A.; Vajaiala, C.; Cocirlea, D. Leveraging Distributed Storage Systems in Conjunction with Blockchain Solutions to Enhance Data Redundancy and Privacy in Organizations. 2023. Available online: https://www.researchsquare.com/article/rs-3254210/v1 (accessed on 30 August 2024).
Kimura, L.T.; Shiraishi, F.K.; Andrade, E.R.; Carvalho, T.C.M.B.; Simplicio, M.A. Amazon Biobank: Assessing the Implementation of a Blockchain-Based Genomic Database. IEEE Access 2024 , 12 , 9632–9647. [ Google Scholar ] [ CrossRef ]
Bernstein, D.J.; Lange, T. Post-quantum cryptography. Nature 2017 , 549 , 188–194. [ Google Scholar ] [ CrossRef ]
Cao, Y.; Zhao, Y.; Wang, J.; Yu, X.; Ma, Z.; Zhang, J. KaaS: Key as a Service over Quantum Key Distribution Integrated Optical Networks. IEEE Commun. Mag. 2019 , 57 , 152–159. [ Google Scholar ] [ CrossRef ]
Pan, D.; Lin, Z.; Wu, J.; Zhang, H.; Sun, Z.; Ruan, D.; Yin, L.; Long, G.L. Experimental free-space quantum secure direct communication and its security analysis. Photon. Res. 2020 , 8 , 1522–1531. [ Google Scholar ] [ CrossRef ]
Alkhatib, R.; Sahwan, W.; Alkhatieb, A.; Schütt, B. A Brief Review of Machine Learning Algorithms in Forest Fires Science. Appl. Sci. 2023 , 13 , 8275. [ Google Scholar ] [ CrossRef ]
Wassouf, W.N.; Alkhatib, R.; Salloum, K.; Balloul, S. Predictive analytics using big data for increased customer loyalty: Syriatel Telecom Company case study. J. Big Data 2020 , 7 , 29. [ Google Scholar ] [ CrossRef ]
Ahmed, F.; Kang, I.S.; Kim, K.H.; Asif, A.; Rahim, C.S.A.; Samantasinghar, A. Drug repurposing for viral cancers: A paradigm of machine learning, deep learning, and virtual screening-based approaches. J. Med. Virol. 2023 , 95 , e28693. [ Google Scholar ] [ CrossRef ]
Battineni, G.; Hossain, M.A.; Chintalapudi, N.; Amenta, F. A Survey on the Role of Artificial Intelligence in Biobanking Studies: A Systematic Review. Diagnostics 2022 , 12 , 1179. [ Google Scholar ] [ CrossRef ]
Anas, A.; Xingwang, L.; Ramez, A.; Khaled, R.; Galymzhan, N. Intelligent Reflecting Surface-aided UAV Communications: A survey and Research Opportunities. In Proceedings of the 2022 13th International Symposium on Communication Systems, Networks and Digital Signal Processing (CSNDSP), Porto, Portugal, 20–22 July 2022; pp. 362–367. [ Google Scholar ]
Frascarelli, C.; Bonizzi, G.; Musico, C.R.; Mane, E.; Cassi, C.; Rocco, E.G.; Farina, A.; Scarpa, A.; Lawlor, R.; Bonetti, L.R.; et al. Revolutionizing Cancer Research: The Impact of Artificial Intelligence in Digital Biobanking. J. Pers. Med. 2023 , 13 , 1390. [ Google Scholar ] [ CrossRef ]
Roy, S.; Coldren, C.; Karunamurthy, A.; Kip, N.S.; Klee, E.W.; Lincoln, S.E. Standards and guidelines for validating next-generation sequencing Bioinformatics Pipelines. J. Mol. Diagn. 2018 , 20 , 4–27. [ Google Scholar ] [ CrossRef ]
Mathur, P. Cloud Computing Infrastructure, Platforms, and Software for Scientific Research. In High Performance Computing in Biomimetics: Modeling, Architecture and Applications ; Springer Nature: Singapore, 2024; pp. 89–127. [ Google Scholar ]
Biswas, A.; Kumari, A.; Gaikwad, D.; Pandey, D.K. Revolutionizing Biological Science: The Synergy of Genomics in Health, Bioinformatics, Agriculture, and Artificial Intelligence. OMICS A J. Integr. Biol. 2023 , 27 , 550–569. [ Google Scholar ] [ CrossRef ]
Ibrahim, A.; Primakov, S.; Beuque, M.; Woodruff, H.C.; Halilaj, I.; Wu, G. Radiomics for precision medicine: Current challenges, future prospects, and the proposal of a new framework. Methods 2021 , 188 , 20–29. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Dowst, H.; Pew, B.; Watkins, C.; McOwiti, A.; Barney, J.; Qu, S.; Becnel, L.B. Acquire: An open-source comprehensive cancer biobanking system. Bioinformatics 2015 , 31 , 1655–1662. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Im, K.; Gui, D.; Yong, W.H. An Introduction to Hardware, Software, and Other Information Technology Needs of Biomedical Biobanks ; Springer: New York, NY, USA, 2019; pp. 17–29. [ Google Scholar ]
Kersting, M.; Prokein, J.; Bernemann, I.; Drobek, D.; Illig, T. IT-Systems for Biobanking—A Brief Overview ; Hannover United Biobank, Hannover Medical School: Hannover, Germany, 2014; Available online: http://www.markus-kersting.de/wp-content/uploads/2014/12/Poster_Biobank_Systeme_HUB_2014_12_01_mk_b.pdf (accessed on 30 August 2024).
Öfelein, M.; Reichold, M.; Christian, M. Designing a framework of components to support patient engagement. Stud. Health Technol. Inform. 2019 , 267 , 20–27. [ Google Scholar ]
Medina-Martínez, J.S.; Arango-Ossa, J.E.; Levine, M.F.; Zhou, Y.; Gundem, G.; Kung, A.L.; Papaemmanuil, E. Isabl Platform, a digital biobank for processing multimodal patient data. BMC Bioinform. 2020 , 21 , 549. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Aribi, H.B.; Ghidaoui, M.H.; Fattouch, S. Monitoring Environmental Performance of Agricultural Supply Chains Using Internet of Things. In Integrating Intelligence and Sustainability in Supply Chains ; IGI Global: Hershey, PA, USA, 2023; pp. 273–292. [ Google Scholar ]
Gille, F.; Vayena, E.; Blasimme, A. Future-proofing biobanks’ governance. Eur. J. Hum. Genet. 2020 , 28 , 989–996. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Wilkinson, M.D.; Dumontier, M.; Aalbersberg, I.J.; Appleton, G.; Axton, M.; Baak, A.; Blomberg, N.; Boiten, J.-W.; da Silva Santos, L.B.; Bourne, P.E.; et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci. Data 2016 , 3 , 160018. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Duhm-Harbeck, P.; Habermann, J.K. Data Protection in Healthcare-Integrated Biobanking. Innov. Digit. Health Diagn. Biomark. 2023 , 3 , 1–7. [ Google Scholar ] [ CrossRef ]
Montague, T.; Nemis-White, J.; Aylen, J.; Torr, E.; Martin, L.; Gogovor, A. Canada's Evolving Medicare: Patient-Centred Care. Healthc. Q. 2019 , 22 , 27–31. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Haldeman, K.; Cadigan, R.; Davis, A.; Goldenberg, A.; Henderson, G.; Lassiter, D.; Reavely, E. Community Engagement in US Biobanking: Multiplicity of Meaning and Method. Public Health Genom. 2014 , 17 , 84–94. [ Google Scholar ] [ CrossRef ]
Batra, G.; Aktaa, S.; Wallentin, L.; Maggioni, A.P.; Wilkinson, C.; Casadei, B. Methodology for the development of international clinical data standards for common cardiovascular conditions: European unified registries for Heart Care evaluation and randomised trials (EuroHeart). Eur. Heart J. Qual. Care Clin. Outcomes 2023 , 9 , 161–168. [ Google Scholar ] [ CrossRef ]
Zhang, J.; Zhang, Z.-M. Ethics and governance of trustworthy medical artificial intelligence. BMC Med. Inform. Decis. Mak. 2023 , 23 , 7. [ Google Scholar ] [ CrossRef ]
Ahmed, F.; Samantasinghar, A.; Soomro, A.M.; Kim, S.; Choi, K.H. A systematic review of computational approaches to understand cancer biology for informed drug repurposing. J. Biomed. Inf. 2023 , 142 , 104373. [ Google Scholar ] [ CrossRef ]
Alahmad, G. Informed Consent in Pediatric Oncology. Cancer Control 2018 , 25 , 107327481877372. [ Google Scholar ] [ CrossRef ]
Paskal, W.; Paskal, A.M.; Dębski, T.; Gryziak, M.; Jaworowski, J. Aspects of Modern Biobank Activity—Comprehensive Review. Pathol. Oncol. Res. 2018 , 24 , 771–785. [ Google Scholar ] [ CrossRef ]
Bonomi, L.; Huang, Y.; Ohno-Machado, L. Privacy challenges and research opportunities for genomic data sharing. Nat. Genet. 2020 , 52 , 646–654. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Alahmad, G.; Al-Jumah, M.; Dierickx, K. Review of national research ethics regulations and guidelines in Middle Eastern Arab countries. BMC Med. Ethics 2012 , 13 , 34. [ Google Scholar ] [ CrossRef ] [ PubMed ]

Click here to enlarge figure

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

Alkhatib, R.; Gaede, K.I. Data Management in Biobanking: Strategies, Challenges, and Future Directions. BioTech 2024 , 13 , 34. https://doi.org/10.3390/biotech13030034

Alkhatib R, Gaede KI. Data Management in Biobanking: Strategies, Challenges, and Future Directions. BioTech . 2024; 13(3):34. https://doi.org/10.3390/biotech13030034

Alkhatib, Ramez, and Karoline I. Gaede. 2024. "Data Management in Biobanking: Strategies, Challenges, and Future Directions" BioTech 13, no. 3: 34. https://doi.org/10.3390/biotech13030034

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

IMAGES

(PDF) B Review of Biomedical Image Processing
Top 20 Biomedical Image Processing Projects [Development Tools]
(PDF) Biomedical Signal and Image Processing
biomedical-image-processing · GitHub Topics · GitHub
Research
(PDF) Biomedical image processing in pathology: a review

VIDEO

INTRO TOPIC BIOMEDICAL SIGNAL PROCESSING
Medical Image Processing with MATLAB
AI in Healthcare Series Workshop: 4
BM 3401 SP unit 1 to 5 important topics (R-2021) #annauniversity #biomedical #biomedicalengineering
Image quality evaluation of deep learning image reconstruction and denoising in clinical CT
BiaPy: a ready-to-use library for bioimage analysis workflows

COMMENTS

Research Topics
Research Topics. Biomedical Imaging. The current plethora of imaging technologies such as magnetic resonance imaging (MR), computed tomography (CT), position emission tomography (PET), optical coherence tomography (OCT), and ultrasound provide great insight into the different anatomical and functional processes of the human body. Computer Vision.
Recent Advances in Medical Image Processing
It helps medical personnel to make an early and more accurate diagnosis. Recently, the deep convolution neural network is emerging as a principal machine learning method in computer vision and has received significant attention in medical imaging. Key Message: In this paper, we will review recent advances in artificial intelligence, machine ...
Critical Analysis of the Current Medical Image-Based Processing
This study searched for works related to the topic of the systematic ... The authors downloaded 10,000 medical images for each modality from Open-i Biomedical Image Search Engine, National Institute of Health, and U.S. National Laboratory of Medicine. ... disease detection has become an important topic in medical image processing and medical ...
Recent Advances in Biomedical Image Processing
Dear Colleagues, We invite submissions exploring cutting-edge research and recent developments in the field of biomedical image processing. Advances in biomedical imaging, including digital radiography; X-ray computed tomography (CT); nuclear (positron emission tomography—PET); ultrasound; optical and magnetic resonance imaging (MRI); as well as a variety of new microscopies, including whole ...
Application of Deep Learning in Biomedical Image Processing
This Research Topic aims to highlight the latest research and development initiatives applying deep learning in biomedical image processing. We call upon scholars to contribute original research articles and review pieces. Potential research topics may include, but aren't limited to: 1. Processing techniques for medical images.
Cutting Edge Advances in Medical Image Analysis and Processing
This Research Topic aims at presenting the latest advances in medical image processing methodologies and their contribution to the medical field and leveraging research on medical image processing. This Topic intends to cover the development and implementation of new medical image-based algorithms and strategies using biomedical image datasets.
Modern Trends and Applications of Intelligent Methods in Biomedical
The Special Issue "Modern Trends and Applications of Intelligent Methods in Biomedical Signal and Image Processing" is aimed at the new proposals and intelligent solutions that constitute the state of the art of the intelligent methods for biomedical data processing from selected areas of signal and image processing.
Advances in Biomedical Image Processing and Analysis
Dear Colleagues, Biomedical image analysis plays a vital role in diagnosing numerous pathologies, ranging from infectious diseases to cancer. Advanced methodologies for signal and/or image processing and analysis and biomedical analytics may be a powerful tool for classifying medical data, identifying individualized health trends, and finding evolutionary trajectories between normal and non ...
Biomedical Image Processing
eBook ISBN 978-3-642-15816-2 Published: 01 March 2011. Series ISSN 1618-7210. Series E-ISSN 2197-5647. Edition Number 1. Number of Pages XXXVII, 595. Topics Biological and Medical Physics, Biophysics, Image Processing and Computer Vision, Pattern Recognition, Medicine/Public Health, general, Imaging / Radiology.
Biomedical Image Segmentation and Analysis
Biomedical image processing has seen a significant improvement due to the availability of advanced technologies to capture high resolution images. This leads to demand for automated segmentation of "Regions of Interest" in the images and its analysis from different perspectives. Supervised segmentation techniques require extensive training data, are not generic, and cannot be used for ...
Viewpoints on Medical Image Processing: From Science to Application
Abstract. Medical image processing provides core innovation for medical imaging. This paper is focused on recent developments from science to applications analyzing the past fifteen years of history of the proceedings of the German annual meeting on medical image processing (BVM). Furthermore, some members of the program committee present their ...
Algorithms for Biomedical Image Analysis and Processing
Biomedical imaging is a broad field concerning image capture for diagnostic and therapeutic purposes. Biomedical imaging technologies utilize X-rays (CT scans), magnetism (MRI), sound (ultrasound), radioactive pharmaceuticals (nuclear medicine: SPECT, PET), or light (endoscopy, OCT). Algorithms for processing and analyzing biomedical images are ...
[2408.03393v2] Biomedical Image Segmentation: A Systematic Literature
Biomedical image segmentation plays a vital role in diagnosis of diseases across various organs. Deep learning-based object detection methods are commonly used for such segmentation. There exists an extensive research in this topic. However, there is no standard review on this topic. Existing surveys often lack a standardized approach or focus on broader segmentation techniques. In this paper ...
biomedical-image-processing · GitHub Topics · GitHub
Add this topic to your repo. To associate your repository with the biomedical-image-processing topic, visit your repo's landing page and select "manage topics." GitHub is where people build software. More than 100 million people use GitHub to discover, fork, and contribute to over 420 million projects.
What are the current topics for research in Medical Image Processing
Bahria University Islamabad Campus. Research topics that are currently hot in a study in medical image processing are. 3d reconstruction of organs using 2d images. GAN (generative adversarial ...
Biomedical Imaging
Biomedical Imaging. The current plethora of imaging technologies such as magnetic resonance imaging (MR), computed tomography (CT), position emission tomography (PET), optical coherence tomography (OCT), and ultrasound provide great insight into the different anatomical and functional processes of the human body.
Biomedical Signal and Image Processing
This course presents the fundamentals of digital signal processing with particular emphasis on problems in biomedical research and clinical medicine. It covers principles and algorithms for processing both deterministic and random signals. Topics include data acquisition, imaging, filtering, coding, feature extraction, and modeling. The focus of the course is a series of labs that provide ...
Advances in biomedical signal and image processing
1. Introduction. Biological and medical information processing is a dynamic field of natural science [1]. The biomedical signals have been used by the architects for designing the bioelectrical and biomechanical systems. The physicians and human service experts introduced the diagnosing procedures of medical issues.
Advanced Image Processing
Advanced Image Processing. Image analysis is a powerful tool in cell biology to collect quantitative measurements in time and space. Because microscopy imaging can easily produce terabytes of research data, accurate and automated analysis methods are key to successfully quantifying relevant information in such large image collections.
Review of Biomedical Image Processing
Summary. Biomedical Image Processing, by Thomas M. Deserno [], is a text that is suitable for clinicians, scientists, and engineers interested in the important topic of how clinical and biomedical images can best be processed for features quantification, and to enhance important visual detail.It is compiled from the notes of multiple authors, all eminent scientists in their field of research.
Top 20 Biomedical Image Processing Projects [Development Tools]
Top 20 Major Research Topics in Biomedical Image Processing Below we have mentioned most interesting biomedical image processing projects, we can guide you to formulate best research topics based on biomedical; reach us to know more information. Advances in Medical Image Computation; Registration and Fusion of 3D Multimodal Medical Image
Biomedical Imaging
Biomedical Imaging. Research in Biomedical Imaging brings together engineers, physicists, biologists and chemists engaged in the development of methodology for the examination of biological structure and function through imaging. The work encompasses efforts in magnetic resonance imaging, magnetic resonance spectroscopy, nuclear medicine ...
Biomedical image processing
Biomedical image processing is a very broad field; it covers biomedical signal gathering, image forming, picture processing, and image display to medical diagnosis based on features extracted from images. ... Heart imaging is one of the most interesting and challenging research topics in biomedical image processing; current methods including ...
Deep learning for the harmonization of structural MRI scans: a survey
Medical imaging datasets for research are frequently collected from multiple imaging centers using different scanners, protocols, and settings. These variations affect data consistency and compatibility across different sources. Image harmonization is a critical step to mitigate the effects of factors like inherent differences between various vendors, hardware upgrades, protocol changes, and ...
Medical image analysis using deep learning algorithms
Medical image processing is an area of research that encompasses the creation and application of ... This paper demonstrated the potential of AI/ML in advancing biomedical material design and reducing the need for animal testing. ... including limited access to non-English papers and a scarcity of high-quality research focusing on this topic ...
BioTech
Biobanking plays a pivotal role in biomedical research by providing standardized processing, precise storing, and management of biological sample collections along with the associated data. Effective data management is a prerequisite to ensure the integrity, quality, and accessibility of these resources. This review provides a current landscape of data management in biobanking, discussing key ...

Biomedical Image Processing

Inst. Medizinische Informatik, RWTH Aachen, Aachen, Germany

Access this book

Other ways to access

About this book

Medical Image Processing

Medical Image Processing: Mathematical Modelling and Numerical Resolution

Image Processing in Biomedical Science

Table of contents (22 chapters)

Image Formation

Cardiac 4D Ultrasound Imaging

Image Enhancement

Medical Image Registration

Feature Extraction and Selection

Multi-Scale and Multi-Orientation Medical Image Analysis

Feature Extraction and Selection for Decision Making

Segmentation

Region-Based Segmentation: Fuzzy Connectedness, Graph Cut and Related Algorithms

Model-Based Segmentation

Classification and Measurements

CADx Mammography

Quantitative Medical Image Analysis for Clinical Development of Therapeutics

Editors and Affiliations

Bibliographic Information

Biomedical Image Segmentation and Analysis

Electrical Engineering and Systems Science > Image and Video Processing

Submission history

References & Citations

BibTeX formatted citation

Bibliographic and Citation Tools

arXivLabs: experimental projects with community collaborators

Navigation Menu

Saved searches

biomedical-image-processing

cambridgeltl / visual-med-alpaca

seung-lab / connected-components-3d

jvanvugt / pytorch-unet

seung-lab / euclidean-distance-transform-3d

OpenEIT / OpenEIT

tkuanlun350 / 3DUnet-Tensorflow-Brats18

zudi-lin / pytorch_connectomics

ELEKTRONN / elektronn3

mlcommons / GaNDLF

anjanatiha / Pneumonia-Detection-from-Chest-X-Ray-Images-with-Deep-Learning

seung-lab / kimimaro

sacmehta / YNet

seung-lab / cloud-volume

yuanqing811 / ISIC2018

BiaPyX / BiaPy

mattmacy / torchbiomed

mirzaevinom / promise12_segmentation

seung-lab / dijkstra3d

hoang-ho / Skin_Lesions_Classification_DCNNs

Improve this page

Add this topic to your repo

Biomedical Imaging

Related people

Related demos

Related publications

Conference papers

Book chapters

Browse Course Material

Departments

As Taught In

Learning Resource Types

You are leaving MIT OpenCourseWare

Advanced Image Processing

High-Performance Image Computation

Example 1: Nuclei segmentation of an adult worm

Example 2: Nuclei segmentation of C. elegans embryos

Example 3: Evaluation of image quality of imaging systems

Image Stitching

Rapid Image Deconvolution and Multiview Fusion

Pipeline 1: Joint-view Deconvolution on Cleared-tissue Datasets

Table 1: Computation time on a pair of 3800 x 3400 x 1200 data (28G)

Pipeline 2: Single-view Deconvolution on Cleared-tissue Dataset

Table 2: Computation time on a pair of 3800 x 3400 x 1200 data (28G)

Machine Learning for Image Denoising, Resolution Enhancement, and Segmentation

Table 3: Computation time of 1020 x 2048 x 100 (400M), 300 time points, 2 colors

Image Denoising and Resolution Enhancement