First, install the requirements in the environment, then navigate tosrc/data/make_dataset.py
.
python3 make_dataset.py
The interim dataset would be stored in the data/interim
directory in the root folder, which would be used for training the model, from this dataset itself test set would be split with a seed of 42
.
First, install the requirements in the environment, then navigate tosrc/models/train_model.py
.
python3 train_model.py
The trained checkpoints will be stored in the models/bloom-detoxification/checkpoint-800/
of the root folder, the repository already has trained checpoints, if its
trained again the checkpoints will be updated.
Navigate tosrc/models/predict_model.py
python3 predict_model.py
You'll be asked for input of the toxic text, and the output would be both the toxic text and the corresponding result.
![image](https://private-user-images.githubusercontent.com/69463767/280515813-05aff7fb-66c7-4db6-8705-b7630caf3a3f.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYwODE1NTksIm5iZiI6MTcyNjA4MTI1OSwicGF0aCI6Ii82OTQ2Mzc2Ny8yODA1MTU4MTMtMDVhZmY3ZmItNjZjNy00ZGI2LTg3MDUtYjc2MzBjYWYzYTNmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MTElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTExVDE5MDA1OVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTIwMWUzMmM4MTIzZmVkZWNlYzRkNTZhNTM1ODczMjJjYTM5NWYzOWQwMWE5Mzc0ZjM0NDU0MWMxYmZhZTZiMzQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.H6ROzqNjI8RrGJeQTwAum-JW71ssXNR-oiwko-ef_oc)