Skip to content

Latest commit

 

History

History
67 lines (56 loc) · 2.37 KB

README.md

File metadata and controls

67 lines (56 loc) · 2.37 KB
Raw

cDeid

A framework for training de-identification models to automatically remove protected health information (PHI) from the free text.

cDeid is a customized de-identification method. The users can easily train their own de-identification Models on the data sets which are extracted from their own free text corpus. cDeid is based on 3 popular NLP toolkits: spaCy, Stanza and FLAIR.

Installation

This project is based on Python 3.7+. Please install it if you do not have. PyTorch is used by FLAIR and Stanza toolkits. It needs to be installed from here before you install this project.

pip install cdeid

Usage example

We are using the pre-trained word2vec embeddings released from the CoNLL 2017 Shared Task. It is important to specify the customized PHI types in the corpus otherwise it will cause runtime error during training the models.

Using the Python API

Train the models

from cdeid.models.trainer import Trainer

phi_types = ['PHONE', 'PERSON', 'ADDRESS', 'IDN', 'DOB']
nlp = Trainer("C:/data", "C:/workspace", phi_types, "C:/wordvec/English/en.vectors.xz")
nlp.train()

De-identify a sample document

from cdeid.deidentifier.phi_deid import PHIDeid

deider = PHIDeid("C:/workspace", "C:/output")
doc = deider("C:/raw/example.txt")
deider.output(doc)

Using the command line

Train the models

python -m cdeid --command train --workspace C:/workspace --data_dir C:/data --phi_types PHONE PERSON ADDRESS IDN DOB --wordvec_file C:/wordvec/English/en.vectors.xz

De-identify a sample document

python -m cdeid --command deid --workspace C:/workspace --deid_output_dir C:/output --deid_file C:/raw/example.txt

Release History

  • 0.1.1
    • The first release
  • 0.1.2
    • Modify Readme and Setup
  • 0.1.3
    • Update model trainers

Contributors

Leibo Liu - initial work - leiboliu

License

Apache License, Version 2.0