-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
114 changed files
with
15,106 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
# Knowledge-based in silico models and dataset for the comparative evaluation of mammography AI for a range of breast characteristics, lesion conspicuities and doses | ||
|
||
**This repository contains code used in the paper:** | ||
|
||
"_Knowledge-based in silico models and dataset for the comparative evaluation of mammography AI for a range of breast characteristics, lesion conspicuities and doses_" | ||
|
||
[Elena Sizikova](https://esizikova.github.io/), [Niloufar Saharkhiz](https://www.linkedin.com/in/niloufar-saharkhiz/), [Diksha Sharma](https://www.linkedin.com/in/diksha-sharma-6059977/), [Miguel Lago](https://www.linkedin.com/in/milaan/), [Berkman Sahiner](https://www.linkedin.com/in/berkman-sahiner-6aa9a919/), [Jana Gut Delfino](https://www.linkedin.com/in/janadelfino/), [Aldo Badano](https://www.linkedin.com/in/aldobadano/) | ||
|
||
NeurIPS Datasets and Benchmarks Track 2023. | ||
|
||
**Paper Link**: [https://arxiv.org/pdf/2310.18494.pdf](https://arxiv.org/pdf/2310.18494.pdf) | ||
|
||
![overview](https://raw.githubusercontent.com/DIDSR/msynth-release/main/images/overview.png) | ||
|
||
The contributions of our work are: | ||
* We demonstrate that, using this approach, we can detect differences in AI model performance based on selected image acquisition device or physical object model parameters. Specifically, we evaluate the effect of image acquisition (radiation dose) and object model (breast and mass densities, mass size) parameters on the performance of the AI model. | ||
* We release a synthetic dataset, M-SYNTH, to facilitate testing with pre-computed data using the proposed pipeline. The dataset consists of 1,200 stochastic knowledge-based (KB) models and their associated digital mammography (DM) images with varying physical (breast density, mass size and density) and imaging (dose) characteristics. | ||
|
||
|
||
## Table of Contents | ||
1. Dataset | ||
2. Code | ||
3. Citation | ||
4. Related Links | ||
5. Disclaimer | ||
|
||
## Dataset | ||
M-SYNTH is a synthetic digital mammography (DM) dataset with four breast fibroglandular density distributions imaged using Monte Carlo x-ray simulations with the publicly available [Virtual Imaging Clinical Trial for Regulatory Evaluation (VICTRE)](https://github.com/DIDSR/VICTRE) toolkit. The dataset has the following characteristics: | ||
|
||
* Breast density: dense, hetero, scattered, fatty | ||
* Mass radius (mm): 5.00, 7.00, 9.00 | ||
* Mass density: 1.0, 1.01, 1.1 | ||
* Relative dose: 20%, 40%, 60%, 80%, 100% of the clinically recommended dose for each density | ||
<p align="center"> | ||
<img src='https://raw.githubusercontent.com/DIDSR/msynth-release/main/images/examples.png' width='700'> | ||
</p> | ||
|
||
M-SYNTH contains mammography images (in .raw and DICOM formats), metadata (breast density, mass radius, mass density, relative dose, detector) and coordinate locations of the lesion. | ||
|
||
**New!** We also released pixel-level segmentation masks generated using VICTRE. | ||
|
||
Dataset is hosted on huggingface and can be downloaded at: [https://huggingface.co/datasets/didsr/msynth](https://huggingface.co/datasets/didsr/msynth) | ||
|
||
## Code | ||
1. Code to show sample preprocessing, model training and inference steps is available in `code` directory. | ||
2. Sample code to generate a single synthetic DM image with annotations and segmentation can be found in the following [VICTRE example](https://github.com/DIDSR/VICTRE_PIPELINE/tree/main/examples/msynth). | ||
3. Examples of how to load .raw and .dcm images, obtain lesion locations and lesion segmentations can be found in the ```examples/``` folder. | ||
|
||
## Citation | ||
``` | ||
@article{sizikova2023knowledge, | ||
title={Knowledge-based in silico models and dataset for the comparative evaluation of mammography AI for a range of breast characteristics, lesion conspicuities and doses}, | ||
author={Sizikova, Elena and Saharkhiz, Niloufar and Sharma, Diksha and Lago, Miguel and Sahiner, Berkman and Delfino, Jana G. and Badano, Aldo}, | ||
journal={Advances in Neural Information Processing Systems}, | ||
volume={}, | ||
pages={}, | ||
year={2023} | ||
} | ||
``` | ||
## Related Links | ||
1. [Virtual Imaging Clinical Trial for Regulatory Evaluation (VICTRE)](https://www.fda.gov/medical-devices/science-and-research-medical-devices/victre-silico-breast-imaging-pipeline). | ||
2. [FDA Catalog of Regulatory Science Tools to Help Assess New Medical Devices](https://www.fda.gov/medical-devices/science-and-research-medical-devices/catalog-regulatory-science-tools-help-assess-new-medical-devices). | ||
3. A. Badano, C. G. Graff, A. Badal, D. Sharma, R. Zeng, F. W. Samuelson, S. Glick, K. J. Myers. [Evaluation of Digital Breast Tomosynthesis as Replacement of Full-Field Digital Mammography Using an In Silico Imaging Trial](http://dx.doi.org/10.1001/jamanetworkopen.2018.5474). JAMA Network Open 2018. | ||
4. A. Badano, M. Lago, E. Sizikova, J. G. Delfino, S. Guan, M. A. Anastasio, B. Sahiner. [The stochastic digital human is now enrolling for in silico imaging trials—methods and tools for generating digital cohorts.](http://dx.doi.org/10.1088/2516-1091/ad04c0) Progress in Biomedical Engineering 2023. | ||
5. E. Sizikova, N. Saharkhiz, D. Sharma, M. Lago, B. Sahiner, J. G. Delfino, A. Badano. [Knowledge-based in silico models and dataset for the comparative evaluation of mammography AI](https://github.com/DIDSR/msynth-release). NeurIPS 2023 Workshop on Synthetic Data Generation with Generative AI. | ||
|
||
#### | ||
**Disclaimer** | ||
|
||
<sub> | ||
This software and documentation (the "Software") were developed at the Food and Drug Administration (FDA) by employees of the Federal Government in the course of their official duties. Pursuant to Title 17, Section 105 of the United States Code, this work is not subject to copyright protection and is in the public domain. Permission is hereby granted, free of charge, to any person obtaining a copy of the Software, to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, or sell copies of the Software or derivatives, and to permit persons to whom the Software is furnished to do so. FDA assumes no responsibility whatsoever for use by other parties of the Software, its source code, documentation or compiled executables, and makes no guarantees, expressed or implied, about its quality, reliability, or any other characteristic. Further, use of this code in no way implies endorsement by the FDA or confers any advantage in regulatory decisions. Although this software can be redistributed and/or modified freely, we ask that any derivative works bear some notice that they are derived from it, and any modified versions bear some notice that they have been modified. | ||
</sub> |
Large diffs are not rendered by default.
Oops, something went wrong.
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,109 @@ | ||
body { | ||
font-family: -apple-system, | ||
BlinkMacSystemFont, | ||
'Segoe UI', | ||
Roboto, | ||
Oxygen, | ||
Ubuntu, | ||
Cantarell, | ||
'Open Sans', | ||
'Helvetica Neue', | ||
sans-serif; | ||
height: calc(100% - 18px); | ||
background-color: white; | ||
color: #333333; | ||
margin: auto; | ||
max-width: 1400px; | ||
box-shadow: rgba(0, 0, 0, 0.8) 0px 0px 30px; | ||
height: 100%; | ||
display: block; | ||
} | ||
|
||
a{ | ||
color:rgb(0, 124, 186); | ||
} | ||
|
||
h1 { | ||
text-align: center; | ||
padding:20px; | ||
background:rgb(0, 124, 186); | ||
color:white; | ||
margin:10px 0 0 0; | ||
} | ||
|
||
.content{ | ||
display: none; | ||
padding:10px; | ||
clear:both; | ||
} | ||
|
||
img{ | ||
width:80%; | ||
|
||
} | ||
|
||
.tabs{ | ||
margin:0; | ||
margin-top:-40px; | ||
} | ||
|
||
.tabs li{ | ||
background:rgb(0, 124, 186); | ||
color:white; | ||
float:left; | ||
list-style: none; | ||
padding:5px 10px; | ||
outline: 1px solid white; margin-top: 1px; | ||
margin-left: 1px; | ||
} | ||
|
||
.tabs li.active{ | ||
filter:brightness(1.3); | ||
} | ||
|
||
.tabs li:hover{ | ||
transition: .2s; | ||
cursor: pointer; | ||
filter:brightness(1.3) | ||
} | ||
|
||
p,.dt-container{ | ||
margin:5px 5px; | ||
} | ||
table.dataTable{ | ||
width:100% !important; | ||
} | ||
#ssynth-table tr td, #ssynth-table tr th{ | ||
text-align:center; | ||
padding:5px 0px; | ||
} | ||
|
||
#ssynth-table th{ | ||
background:rgb(0, 124, 186); | ||
color:white; | ||
text-transform: uppercase; | ||
} | ||
|
||
#ssynth-table tr{ | ||
background-color:rgb(198, 236, 255); | ||
} | ||
|
||
#ssynth-table tr:nth-child(even){ | ||
background-color:rgb(224, 244, 255) | ||
} | ||
|
||
input.footer-search { | ||
border: 1px solid #aaa; | ||
border-radius: 3px; | ||
padding: 5px; | ||
background-color: white; | ||
color: black; | ||
margin-left: 3px; | ||
width:75% | ||
} | ||
|
||
img.sample:hover{ | ||
cursor:pointer; | ||
transform: scale(1.1); | ||
transition: 0.25s; | ||
} |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
<!DOCTYPE HTML> | ||
<html> | ||
|
||
<head> | ||
<title>DIDSR DEMO</title> | ||
<!-- <link rel="icon" href="favicon.svg" sizes="any" type="image/svg+xml"> --> | ||
<link rel="stylesheet" type="text/css" href="css/style.css"> | ||
<meta name="color-scheme" content="light dark" /> | ||
<script type="text/javascript" src="js/jquery.min.js"></script> | ||
<script type="text/javascript" src="js/datatables/datatables.min.js" referrerpolicy="origin"></script> | ||
<script src="js/fancybox.umd.js"></script> | ||
<script src="js/marked.min.js"></script> | ||
<script type="text/javascript" src="scripts.js"></script> | ||
<link rel="stylesheet" href="js/datatables/datatables.min.css"> | ||
<link rel="stylesheet" href="css/fancybox.css" /> | ||
</head> | ||
|
||
<body id="consult-body"> | ||
<h1><span class="title"></span></h1> | ||
<ul class="tabs"> | ||
<li content="description" class="active">Description</li> | ||
<li content="demo">Demo</li> | ||
</ul> | ||
<div class="content" content="description"></div> | ||
<div class="content" content="demo"> | ||
<p>This is a sample of 100 images from the <span class="title"></span> database. References: <span | ||
id="references"></span></p> | ||
<table id="data-table"> | ||
<thead> | ||
<tr> | ||
</tr> | ||
</thead> | ||
<tbody> | ||
|
||
</tbody> | ||
<tfoot> | ||
<tr> | ||
</tr> | ||
</tfoot> | ||
</table> | ||
</div> | ||
</body> | ||
|
||
</html> |
Oops, something went wrong.