A curated list of tools, measures, and references for MRI quality control (QC).
Quality Control (QC) refers to “a real-time prospective process to ensure imaging quality is maintained by comparing it regularly to a defined set of criteria or industry standards” (Sreedher et al., 2021) via (Das et al., 2022).
Inspired by awesome-python and other awesome lists.
- MRIQC: extracts no-reference image quality metrics from structural and functional MRI data.
- QAP: The QAP package allows you to obtain spatial and anatomical data quality measures for your own data. (Precursor to MRIQC.)
- fBIRN QA tools: These tools form the basis of the fBIRN QA procedures (Glover et al., 2012).
- MRQy: A quality assurance and checking tool for quantitative assessment of MRI data
- AFNI: A suite of programs for looking at
and analyzing MRI brain images at all stages of analysis.
3dToutcount: Calculates number of “outliers” a 3D+time dataset, at each time point.3dTqual: Computes a “quality index” for each sub-brick in a 3D+time dataset.
- FSL: A comprehensive library
of analysis tools for FMRI, MRI and DTI brain imaging data.
- EDDY QC tools : Generates single-subject and group QC reports for diffusion MRI.
- FEAT report: HTML report for single-subject and group fMRI analysis.
fsl_motion_outliers: command-line tool for analyzing single-subject motion.
- NiPype: Provides a uniform interface to existing neuroimaging software and facilitates interaction between these packages within a single workflow
- C-PAC: A configurable, open-source, Nipype-based, automated processing pipeline for resting state functional MRI data
- XCP: A free, open-source software package for processing of multimodal neuroimages with extensive QC metrics for T1w and BOLD MRI images.
- DPABI: A toolbox for Data Processing & Analysis for Brain Imaging including GUI-based QC reports.
- DSI Studio: A tractography software tool that maps brain connections and correlates findings with neuropsychological disorders. Includes automated QC metrics.
- QSIprep: Configures pipelines for processing diffusion-weighted MRI (dMRI) data. Includes QC metrics from DSI Studio.
- NiRV: A modern neuroimaging report viewer that aggregates participant level HTML reports for datasets, small and large.
- NiReports: The NiPreps’ Reporting and Visualization system - report templates and “reportlets”
- SQAN: Scalable Quality Assurance for Neuroimaging. A full-stack system for extracting, translating, logging, and visualizing DICOM-formatted medical imaging data.
- MIQA: Efficient and accurate QC processing by leveraging modern UI/UX and deep learning techniques
- MindControl: An app for quality control of neuroimaging pipeline outputs, especially anatomical segmentations
- Braindr: a firebase app for braindr: Tinder for brains
- Fibr: An app for quality control of diffusion MRI images from the Healthy Brain Network
- dmriprep-viewer: Web app to visualize local QSIprep and dMRIprep outputs
- Qoala-T: Qoala-T is a supervised-learning tool for quality control of FreeSurfer segmented MRI data
- mriqc-learn: Learning on MRIQC-generated image quality metrics (IQMs)
- sewar: All image quality metrics you need in one package.
- MATLAB IQMs: Full and no-reference image quality metrics implemented in MATLAB.
- awesome-image-quality-assessment: Awesome list of image quality tools and references.
| Measure | Summary | Interpretation | References |
|---|---|---|---|
| Coefficient of joint variation (CJV) | Larger values indicate head motion and INU artifacts | lower better | MRIQC, (Ganzetti et al., 2016) |
| Contrast-to-noise ratio (CNR) | Larger values indicate more GM to WM contrast | higher better | MRIQC, (Magnotta & Friedman, 2006) |
| Signal-to-noise ratio (SNR) | SNR within brain mask | higher better | MRIQC |
| Dietrich SNR (SNRd) | SNR relative to air background | higher better | MRIQC, (Dietrich et al., 2007) |
| Mortamet’s quality index 1 (QI1) | Proportion of “corrupted” voxels vs number of background voxels | lower better | MRIQC, (Mortamet et al., 2009) |
| Mortamet’s quality index 2 (QI2) | Comparison of background noise with |
lower better | MRIQC, (Mortamet et al., 2009) |
| EFC | Shannon entropy as indicator of ghosting due to head motion | lower better | MRIQC, (Atkinson et al., 1997) |
| FBER | Ratio of “mean energy” within head vs air | higher better | MRIQC, (Shehzad et al., 2015) |
| INU | Summary stats for INU bias field. Values close to 1 mean less bias. | higher better | MRIQC, (Tustison et al., 2010) |
| White matter to maximum ratio (WM2max) | Median WM intensity divided by 95 percentile intensity | values in [0.6, 0.8] are good. |
MRIQC |
| FWHM | Estimation of image smoothness. Higher values mean blurry. | lower better | MRIQC |
| Measure | Summary | Interpretation | References |
|---|---|---|---|
| ICV | Intracranial volume fraction for each tissue type (WM, GM, CSF) | MRIQC | |
| rPVe | Residual partial voluming error for each tissue type | lower better | MRIQC |
| Tissue summary stats | Summary stats for signal within tissue masks (mean, stdev, p05, p95) | MRIQC | |
| Tissue prior overlap | Overlap of estimated tissue probability maps with template priors | higher better | MRIQC |
| Tissue skewness/kurtosis | Skewness and kurtosis of intensity distribution for WM, GM, and background | MRIQC, (Rosen et al., 2018) | |
| WM hypointensities | Voxel count of white matter hypointensities | lower better | Freesurfer, (Klapwijk et al., 2019) |
| Measure | Summary | Interpretation | References |
|---|---|---|---|
| Boundary tissue count | Volume of each tissue type lying on brain mask boundary. Large WM values suggest brain extraction failure. | lower WM values better | (Alfaro-Almagro et al., 2018) |
| Measure | Summary | Interpretation | References |
|---|---|---|---|
| Normalization cost | Cost function between T1 and template under linear and nonlinear alignment | lower better | |
| Normalization magnitude | Amount of nonlinear warping | lower better | (Alfaro-Almagro et al., 2018) |
| Normalized overlap | Dice or Jaccard overlap coefficient between resampled T1 and template for brain and tissue masks | higher better | XCP, (Alfaro-Almagro et al., 2018) |
| Measure | Summary | Interpretation | References |
|---|---|---|---|
| Euler number | 2 - 2g where g is the number of topological holes in the surface. Computed by the Euler–Lhulier formula (V - E + F) |
higher better | (Rosen et al., 2018), (Dale et al., 1999), Freesurfer |
| Local gyrification index (LGI) | Measures degree of cortical folding in neighborhood of each vertex (spatial map). | Freesurfer | |
| BBR criterion | Measures the magnitude of WM/GM contrast across the WM surface boundary | higher better | (Greve & Fischl, 2009), Freesurfer |
| Measure | Summary | Interpretation | References |
|---|---|---|---|
| Subcortical volume | Volume of each subcortical region in a segmentation (vector). | FIRST, Freesurfer | |
| Cortical volume | Volume of each cortical region in a parcellation (vector). | (Alfaro-Almagro et al., 2018), ANTs Freesurfer, Mindboggle | |
| Cortical thickness | Mean thickness of each region in a parcellation (vector). | (Rosen et al., 2018), ANTs, Freesurfer |
| Measure | Summary | Interpretation | References |
|---|---|---|---|
| EFC | Shannon entropy as indicator of ghosting due to head motion | lower better | MRIQC, (Atkinson et al., 1997) |
| FBER | Ratio of “mean energy” within head vs air | higher better | MRIQC, (Shehzad et al., 2015) |
| FWHM | Estimation of image smoothness. Higher values mean blurry. | MRIQC | |
| SNR | SNR within brain mask. | higher better | MRIQC |
| BOLD summary stats | BOLD intensity summary stats (mean, stdev, p95, p05) | MRIQC | |
| Global correlation (GCor) | Average correlation between every voxel and every other voxel | AFNI, MRIQC, (Saad et al., 2013) | |
| Temporal standard deviation (tSD) | Map of temporal standard deviation | lower better | MRIQC, (Marcus et al., 2013) |
| Temporal SNR (tSNR) | Map of temporal mean divided by standard deviation | higher better | MRIQC |
| Ghost to signal ratio (GSR) | Measures amount of signal in regions prone to ghosting | lower better | MRIQC |
| AFNI outlier ratio (AOR) | Mean fraction of “outliers” per fMRI volume using AFNI 3dToutcount |
lower better | AFNI, MRIQC |
| AFNI quality index (AQI) | Mean “quality index”, which for each volume is 1 - correlation to median volume | lower better | AFNI, MRIQC |
| Number of dummy scans | Number of non-steady state dummy scans | MRIQC | |
| Carpet plot | BOLD time series for a set of ROIs arranged in a matrix | MRIQC, (Power, 2017) | |
| Air signal | mean BOLD time series for a set of background/air slices | MRIQC |
| Measure | Summary | Interpretation | References |
|---|---|---|---|
| DVARS | Measures amount of signal change between consecutive time points (time series) | spikes indicate significant motion | MRIQC, NiPype, fsl_motion_outliers, (Power et al., 2012) |
| Framewise displacement (FD) | Sum of absolute translation and rotation displacements in mm at each time point (time series) | spikes indicate significant motion | MRIQC, NiPype, (Jenkinson et al., 2002), (Power et al., 2012) |
| Measure | Summary | Interpretation | References |
|---|---|---|---|
| Co-registration cost | Cost function for rigid registration between BOLD and T1 | lower better | XCP |
| Brain mask overlap | Dice or Jaccard brain mask overlap coefficient between resampled BOLD and T1 | higher better | XCP |
TODO
| Measure | Summary | Interpretation | References |
|---|---|---|---|
| Mean neighbor correlation | Average Pearson correlation between each diffusion image and its q-space nearest neighbor | expected range [0.6, 0.8] |
QSIprep, DSI Studio, (Yeh et al., 2019) |
| Dropout slice count | Count of slices with significant signal dropout | expected less than 0.1% | QSIprep, DSI Studio, (Yeh et al., 2019) |
| Fiber coherence index | Measures how well fibers are connected to each other | low values indicate flipped b-vectors | QSIprep, DSI Studio, (Schilling et al., 2019) |
TODO
TODO
TODO
Alfaro-Almagro, F., Jenkinson, M., Bangerter, N. K., Andersson, J. L., Griffanti, L., Douaud, G., Sotiropoulos, S. N., Jbabdi, S., Hernandez-Fernandez, M., Vallee, E., et al. (2018). Image processing and quality control for the first 10,000 brain imaging datasets from UK biobank. Neuroimage, 166, 400–424. https://doi.org/10.1016/j.neuroimage.2017.10.034
Atkinson, D., Hill, D. L., Stoyle, P. N., Summers, P. E., & Keevil, S. F. (1997). Automatic correction of motion artifacts in magnetic resonance images using an entropy focus criterion. IEEE Transactions on Medical Imaging, 16(6), 903–910. https://doi.org/10.1109/42.650886
Dale, A. M., Fischl, B., & Sereno, M. I. (1999). Cortical surface-based analysis: I. Segmentation and surface reconstruction. Neuroimage, 9(2), 179–194. https://doi.org/10.1006/nimg.1998.0395
Das, D., Etzel, J., Esteban, O., MacNicol, E., Ghosh, S., & Alfaro-Almagro, F. (2022). ISMRM’22 QC book. https://www.nipreps.org/qc-book/welcome.html.
Dietrich, O., Raya, J. G., Reeder, S. B., Reiser, M. F., & Schoenberg, S. O. (2007). Measurement of signal-to-noise ratios in MR images: Influence of multichannel coils, parallel imaging, and reconstruction filters. Journal of Magnetic Resonance Imaging: An Official Journal of the International Society for Magnetic Resonance in Medicine, 26(2), 375–385. https://doi.org/10.1002/jmri.20969
Ganzetti, M., Wenderoth, N., & Mantini, D. (2016). Intensity inhomogeneity correction of structural MR images: A data-driven approach to define input algorithm parameters. Frontiers in Neuroinformatics, 10, 10. https://doi.org/10.3389/fninf.2016.00010
Glover, G. H., Mueller, B. A., Turner, J. A., Van Erp, T. G., Liu, T. T., Greve, D. N., Voyvodic, J. T., Rasmussen, J., Brown, G. G., Keator, D. B., et al. (2012). Function biomedical informatics research network recommendations for prospective multicenter functional MRI studies. Journal of Magnetic Resonance Imaging, 36(1), 39–54. https://doi.org/10.1002/jmri.23572
Greve, D. N., & Fischl, B. (2009). Accurate and robust brain image alignment using boundary-based registration. Neuroimage, 48(1), 63–72. https://doi.org/10.1016/j.neuroimage.2009.06.060
Jenkinson, M., Bannister, P., Brady, M., & Smith, S. (2002). Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage, 17(2), 825–841. https://doi.org/10.1006/nimg.2002.1132
Klapwijk, E. T., Van De Kamp, F., Van Der Meulen, M., Peters, S., & Wierenga, L. M. (2019). Qoala-t: A supervised-learning tool for quality control of FreeSurfer segmented MRI data. Neuroimage, 189, 116–129. https://doi.org/10.1016/j.neuroimage.2019.01.014
Magnotta, V. A., & Friedman, L. (2006). Measurement of signal-to-noise and contrast-to-noise in the fBIRN multicenter imaging study. Journal of Digital Imaging, 19(2), 140–147. https://doi.org/10.1007/s10278-006-0264-x
Marcus, D. S., Harms, M. P., Snyder, A. Z., Jenkinson, M., Wilson, J. A., Glasser, M. F., Barch, D. M., Archie, K. A., Burgess, G. C., Ramaratnam, M., et al. (2013). Human connectome project informatics: Quality control, database services, and data visualization. Neuroimage, 80, 202–219. https://doi.org/10.1016/j.neuroimage.2013.05.077
Mortamet, B., Bernstein, M. A., Jack Jr, C. R., Gunter, J. L., Ward, C., Britson, P. J., Meuli, R., Thiran, J.-P., & Krueger, G. (2009). Automatic quality assessment in structural brain magnetic resonance imaging. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, 62(2), 365–372. https://doi.org/10.1002/mrm.21992
Power, J. D. (2017). A simple but useful way to assess fMRI scan qualities. Neuroimage, 154, 150–158. https://doi.org/10.1016/j.neuroimage.2016.08.009
Power, J. D., Barnes, K. A., Snyder, A. Z., Schlaggar, B. L., & Petersen, S. E. (2012). Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion. Neuroimage, 59(3), 2142–2154. https://doi.org/10.1016/j.neuroimage.2011.10.018
Rosen, A. F., Roalf, D. R., Ruparel, K., Blake, J., Seelaus, K., Villa, L. P., Ciric, R., Cook, P. A., Davatzikos, C., Elliott, M. A., et al. (2018). Quantitative assessment of structural image quality. Neuroimage, 169, 407–418. https://doi.org/10.1016/j.neuroimage.2017.12.059
Saad, Z. S., Reynolds, R. C., Jo, H. J., Gotts, S. J., Chen, G., Martin, A., & Cox, R. W. (2013). Correcting brain-wide correlation differences in resting-state FMRI. Brain Connectivity, 3(4), 339–352. https://doi.org/10.1089/brain.2013.0156
Schilling, K. G., Yeh, F.-C., Nath, V., Hansen, C., Williams, O., Resnick, S., Anderson, A. W., & Landman, B. A. (2019). A fiber coherence index for quality control of b-table orientation in diffusion MRI scans. Magnetic Resonance Imaging, 58, 82–89. https://doi.org/10.1016/j.mri.2019.01.018
Shehzad, Z., Giavasis, S., Li, Q., Benhajali, Y., Yan, C., Yang, Z., Milham, M., Bellec, P., & Craddock, C. (2015). The preprocessed connectomes project quality assessment protocol-a resource for measuring the quality of MRI data. Frontiers in Neuroscience, 47. https://doi.org/10.3389/conf.fnins.2015.91.00047
Sreedher, G., Ho, M.-L., Smith, M., Udayasankar, U. K., Risacher, S., Rapalino, O., Greer, M.-L. C., Doria, A. S., & Gee, M. S. (2021). Magnetic resonance imaging quality control, quality assurance and quality improvement. Pediatric Radiology, 51(5), 698–708. https://doi.org/10.1007/s00247-021-05043-6
Tustison, N. J., Avants, B. B., Cook, P. A., Zheng, Y., Egan, A., Yushkevich, P. A., & Gee, J. C. (2010). N4ITK: Improved N3 bias correction. IEEE Transactions on Medical Imaging, 29(6), 1310–1320. https://doi.org/10.1109/TMI.2010.2046908
Yeh, F.-C., Zaydan, I. M., Suski, V. R., Lacomis, D., Richardson, R. M., Maroon, J. C., & Barrios-Martinez, J. (2019). Differential tractography as a track-based biomarker for neuronal injury. Neuroimage, 202, 116131. https://doi.org/10.1016/j.neuroimage.2019.116131