MedMNIST

18x Standardized Datasets for 2D and 3D Biomedical Image Classification

with Multiple Size Options: 28 (MNIST-Like), 64, 128, and 224



1 Shanghai Jiao Tong University, Shanghai, China
2 Boston College, Chestnut Hill, MA
3 RWTH Aachen University, Aachen, Germany
4 Fudan Institute of Metabolic Diseases, Zhongshan Hospital, Fudan University, Shanghai, China
5 Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
6 Harvard University, Cambridge, MA




Abstract

We introduce MedMNIST, a large-scale MNIST-like collection of standardized biomedical images, including 12 datasets for 2D and 6 datasets for 3D. All images are pre-processed into 28x28 (2D) or 28x28x28 (3D) with the corresponding classification labels, so that no background knowledge is required for users. Covering primary data modalities in biomedical images, MedMNIST is designed to perform classification on lightweight 2D and 3D images with various data scales (from 100 to 100,000) and diverse tasks (binary/multi-class, ordinal regression and multi-label). The resulting dataset, consisting of approximately 708K 2D images and 10K 3D images in total, could support numerous research and educational purposes in biomedical image analysis, computer vision and machine learning. We benchmark several baseline methods on MedMNIST, including 2D / 3D neural networks and open-source / commercial AutoML tools.

Update: We are thrilled to release MedMNIST+ with larger sizes: 64x64, 128x128, and 224x224 for 2D, and 64x64x64 for 3D. As a complement to the previous 28-size MedMNIST, the large-size version could serve as a standardized benchmark for medical foundation models.


Key Features

  • Diverse: It covers diverse data modalities, dataset scales (from 100 to 100,000), and tasks (binary/multi-class, multi-label, and ordinal regression). It is as diverse as the VDD and MSD to fairly evaluate the generalizable performance of machine learning algorithms in different settings, but both 2D and 3D biomedical images are provided.
  • Standardized: Each sub-dataset is pre-processed into the same format, which requires no background knowledge for users. As an MNIST-like dataset collection to perform classification tasks on small images, it primarily focuses on the machine learning part rather than the end-to-end system. Furthermore, we provide standard train-validation-test splits for all datasets in MedMNIST, therefore algorithms could be easily compared.
  • User-Friendly: The small size of 28×28 (2D) or 28×28×28 (3D) is lightweight and ideal for evaluating machine learning algorithms. We also offer a larger-size version, MedMNIST+: 64x64 (2D), 128x128 (2D), 224x224 (2D), and 64x64x64 (3D). Serving as a complement to the 28-size MedMNIST, this could be a standardized resource for developing medical foundation models. All these datasets are accessible via the same API.
  • Educational: As an interdisciplinary research area, biomedical image analysis is difficult to hand on for researchers from other communities, as it requires background knowledge from computer vision, machine learning, biomedical imaging, and clinical science. Our data with the Creative Commons (CC) License is easy to use for educational purposes.

Please note that this dataset is NOT intended for clinical use.


Materials

The MedMNIST dataset consists of 12 pre-processed 2D datasets and 6 pre-processed 3D datasets from selected sources covering primary data modalities (e.g., X-Ray, OCT, Ultrasound, CT, Electron Microscope), diverse classification tasks (binary/multi-class, ordinal regression and multi-label) and data scales (from 100 to 100,000). For simplicity, we call the collection of all 2D datasets as MedMNIST2D, and that of 3D as MedMNIST3D.

We recommend our official code to download and use the MedMNIST dataset: pip install medmnist


MedMNIST2D

An Overview of MedMNIST2D in MedMNIST. Click➚ each row to view more details.
MedMNIST2D Data Modality Tasks (# Classes/Labels) # Samples # Training / Validation / Test
PathMNIST Colon Pathology Multi-Class (9) 107,180 89,996 / 10,004 / 7,180
ChestMNIST Chest X-Ray Multi-Label (14) Binary-Class (2) 112,120 78,468 / 11,219 / 22,433
DermaMNIST Dermatoscope Multi-Class (7) 10,015 7,007 / 1,003 / 2,005
OCTMNIST Retinal OCT Multi-Class (4) 109,309 97,477 / 10,832 / 1,000
PneumoniaMNIST Chest X-Ray Binary-Class (2) 5,856 4,708 / 524 / 624
RetinaMNIST Fundus Camera Ordinal Regression (5) 1,600 1,080 / 120 / 400
BreastMNIST Breast Ultrasound Binary-Class (2) 780 546 / 78 / 156
BloodMNIST Blood Cell Microscope Multi-Class (8) 17,092 11,959 / 1,712 / 3,421
TissueMNIST Kidney Cortex Microscope Multi-Class (8) 236,386 165,466 / 23,640 / 47,280
OrganAMNIST Abdominal CT Multi-Class (11) 58,830 34,561 / 6,491 / 17,778
OrganCMNIST Abdominal CT Multi-Class (11) 23,583 12,975 / 2,392 / 8,216
OrganSMNIST Abdominal CT Multi-Class (11) 25,211 13,932 / 2,452 / 8,827

Facts of PathMNIST

Data Modality: Colon Pathology
Task: Multi-Class (9)
Number of Samples: 107,180 (89,996 / 10,004 / 7,180)
Source Data:

Jakob Nikolas Kather, Johannes Krisam, et al., "Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study," PLOS Medicine, vol. 16, no. 1, pp. 1–22, 01 2019.

License: CC BY 4.0

MedMNIST3D

An Overview of MedMNIST3D in MedMNIST. Click➚ each row to view more details.
MedMNIST3D Data Modality Tasks (# Classes/Labels) # Samples # Training / Validation / Test
OrganMNIST3D Abdominal CT Multi-Class (11) 1,742 971 / 161 / 610
NoduleMNIST3D Chest CT Binary-Class (2) 1,633 1,158 / 165 / 310
AdrenalMNIST3D Shape from Abdominal CT Binary-Class (2) 1,584 1,188 / 98 / 298
FractureMNIST3D Chest CT Multi-Class (3) 1,370 1,027 / 103 / 240
VesselMNIST3D Shape from Brain MRA Binary-Class (2) 1,908 1,335 / 191 / 382
SynapseMNIST3D Electron Microscope Binary-Class (2) 1,759 1,230 / 177 / 352

Facts of OrganMNIST3D

jpggif
Data Modality: Abdominal CT
Task: Multi-Class (11)
Number of Samples: 1,742 (971 / 161 / 610)
Source Data:

Patrick Bilic, Patrick Ferdinand Christ, et al., "The liver tumor segmentation benchmark (lits)," arXiv preprint arXiv:1901.04056, 2019.

Xuanang Xu, Fugen Zhou, et al., "Efficient multiple organ localization in ct image using 3d region proposal network," IEEE Transactions on Medical Imaging, vol. 38, no. 8, pp. 1885–1898, 2019.

License: CC BY 4.0

Benchmarking

Methods PathMNIST ChestMNIST DermaMNIST OCTMNIST PneumoniaMNIST RetinaMNIST
AUC ACC AUC ACC AUC ACC AUC ACC AUC ACC AUC ACC
ResNet-18 (28) 0.983 0.907 0.768 0.947 0.917 0.735 0.943 0.743 0.944 0.854 0.717 0.524
ResNet-18 (224) 0.989 0.909 0.773 0.947 0.920 0.754 0.958 0.763 0.956 0.864 0.710 0.493
ResNet-50 (28) 0.990 0.911 0.769 0.947 0.913 0.735 0.952 0.762 0.948 0.854 0.726 0.528
ResNet-50 (224) 0.989 0.892 0.773 0.948 0.912 0.731 0.958 0.776 0.962 0.884 0.716 0.511
auto-sklearn 0.934 0.716 0.649 0.779 0.902 0.719 0.887 0.601 0.942 0.855 0.690 0.515
AutoKeras 0.959 0.834 0.742 0.937 0.915 0.749 0.955 0.763 0.947 0.878 0.719 0.503
Google AutoML Vision 0.944 0.728 0.778 0.948 0.914 0.768 0.963 0.771 0.991 0.946 0.750 0.531
Benchmarking Performance on MedMNIST2D.
Methods BreastMNIST BloodMNIST TissueMNIST OrganAMNIST OrganCMNIST OrganSMNIST
AUC ACC AUC ACC AUC ACC AUC ACC AUC ACC AUC ACC
ResNet-18 (28) 0.901 0.863 0.998 0.958 0.930 0.676 0.997 0.935 0.992 0.900 0.972 0.782
ResNet-18 (224) 0.891 0.833 0.998 0.963 0.933 0.681 0.998 0.951 0.994 0.920 0.974 0.778
ResNet-50 (28) 0.857 0.812 0.997 0.956 0.931 0.680 0.997 0.935 0.992 0.905 0.972 0.770
ResNet-50 (224) 0.866 0.842 0.997 0.950 0.932 0.680 0.998 0.947 0.993 0.911 0.975 0.785
auto-sklearn 0.836 0.803 0.984 0.878 0.828 0.532 0.963 0.762 0.976 0.829 0.945 0.672
AutoKeras 0.871 0.831 0.998 0.961 0.941 0.703 0.994 0.905 0.990 0.879 0.974 0.813
Google AutoML Vision 0.919 0.861 0.998 0.966 0.924 0.673 0.990 0.886 0.988 0.877 0.964 0.749
Benchmarking Performance on MedMNIST3D.
Methods OrganMNIST3D NoduleMNIST3D FractureMNIST3D AdrenalMNIST3D VesselMNIST3D SynapseMNIST3D
AUC ACC AUC ACC AUC ACC AUC ACC AUC ACC AUC ACC
ResNet-18 + 2.5D 0.977 0.788 0.838 0.835 0.587 0.451 0.718 0.772 0.748 0.846 0.634 0.696
ResNet-18 + 3D 0.996 0.907 0.863 0.844 0.712 0.508 0.827 0.721 0.874 0.877 0.820 0.745
ResNet-18 + ACS 0.994 0.900 0.873 0.847 0.714 0.497 0.839 0.754 0.930 0.928 0.705 0.722
ResNet-50 + 2.5D 0.974 0.769 0.835 0.848 0.552 0.397 0.732 0.763 0.751 0.877 0.669 0.735
ResNet-50 + 3D 0.994 0.883 0.875 0.847 0.725 0.494 0.828 0.745 0.907 0.918 0.851 0.795
ResNet-50 + ACS 0.994 0.889 0.886 0.841 0.750 0.517 0.828 0.758 0.912 0.858 0.719 0.709
auto-sklearn 0.977 0.814 0.914 0.874 0.628 0.453 0.828 0.802 0.910 0.915 0.631 0.730
AutoKeras 0.979 0.804 0.844 0.834 0.642 0.458 0.804 0.705 0.773 0.894 0.538 0.724

Citation

If you find this project useful, please cite:

  Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, Bingbing Ni. Yang, Jiancheng, et al. "MedMNIST v2-A large-scale lightweight benchmark for 2D and 3D biomedical image classification." Scientific Data, 2023.
                            
  Jiancheng Yang, Rui Shi, Bingbing Ni. "MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis". IEEE 18th International Symposium on Biomedical Imaging (ISBI), 2021.

or using bibtex:

@article{medmnistv2,
    title={MedMNIST v2-A large-scale lightweight benchmark for 2D and 3D biomedical image classification},
    author={Yang, Jiancheng and Shi, Rui and Wei, Donglai and Liu, Zequan and Zhao, Lin and Ke, Bilian and Pfister, Hanspeter and Ni, Bingbing},
    journal={Scientific Data},
    volume={10},
    number={1},
    pages={41},
    year={2023},
    publisher={Nature Publishing Group UK London}
}

@inproceedings{medmnistv1,
    title={MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis},
    author={Yang, Jiancheng and Shi, Rui and Ni, Bingbing},
    booktitle={IEEE 18th International Symposium on Biomedical Imaging (ISBI)},
    pages={191--195},
    year={2021}
}

Please also cite the corresponding paper(s) of source data if you use any subset of MedMNIST (check this bibtex).


Python Usage

|

License

The MedMNIST dataset is licensed under Creative Commons Attribution 4.0 International (CC BY 4.0), except DermaMNIST under Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0).

The code is under Apache-2.0 License.


Copyright © 2020-2024 MedMNIST Team

This website created by Jiancheng Yang is hosted on GitHub. Updated on Jan, 2024.