Skip to content
/ cdeid Public

A framework for training de-identification models to automatically remove protected health information (PHI) from the free text.

License

Notifications You must be signed in to change notification settings

CBDRH/cdeid

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

cDeid

A framework for training de-identification models to automatically remove protected health information (PHI) from the free text.

cDeid is a customized de-identification method. The users can easily train their own de-identification Models on the data sets which are extracted from their own free text corpus. cDeid is based on 3 popular NLP toolkits: spaCy, Stanza and FLAIR.

Installation

This project is based on Python 3.7+. Please install it if you do not have. PyTorch is used by FLAIR and Stanza toolkits. It needs to be installed from here before you install this project.

pip install cdeid

Usage example

We are using the pre-trained word2vec embeddings released from the CoNLL 2017 Shared Task. It is important to specify the customized PHI types in the corpus otherwise it will cause runtime error during training the models.

Using the Python API

Train the models

from cdeid.models.trainer import Trainer

phi_types = ['PHONE', 'PERSON', 'ADDRESS', 'IDN', 'DOB']
nlp = Trainer("C:/data", "C:/workspace", phi_types, "C:/wordvec/English/en.vectors.xz")
nlp.train()

De-identify a sample document

from cdeid.deidentifier.phi_deid import PHIDeid

deider = PHIDeid("C:/workspace", "C:/output")
doc = deider("C:/raw/example.txt")
deider.output(doc)

Using the command line

Train the models

python -m cdeid --command train --workspace C:/workspace --data_dir C:/data --phi_types PHONE PERSON ADDRESS IDN DOB --wordvec_file C:/wordvec/English/en.vectors.xz

De-identify a sample document

python -m cdeid --command deid --workspace C:/workspace --deid_output_dir C:/output --deid_file C:/raw/example.txt

Release History

  • 0.1.1
    • The first release
  • 0.1.2
    • Modify Readme and Setup
  • 0.1.3
    • Update model trainers

Contributors

Leibo Liu - initial work - leiboliu

License

Apache License, Version 2.0

About

A framework for training de-identification models to automatically remove protected health information (PHI) from the free text.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published