- Review transformers both in in theory and in practice.
- Run several experiments using traditional-chinese corpus
I crawl some articles from PTT-Gossiping here
.
├── README.md
├── data
│ ├── Gossiping-38650-39150.json
│ └── transformer.pkl
├── examples
│ └── pytorch_example.py
└── pytorch
├── positional_encoding.py
└── transformer.py
data
directory restore data andtransformer.pkl
is a example of trained modelpytorch
directory lists useful models that I modified from Pytorchpytorch_example.py
list an example of training a transformer based model
python ../examples/pytorch_example.py \
--epoch 1 \
--encoder_layer 3 \
--word_dimension 256 \
--hidden_dimension 300 \
--attention_head 8 \
--dropout 0.2 \
--learning_rate 0.05 \
--data /Users/admin/Practice/transformer-tutorial/data/Gossiping-38650-39150.json \
--model /Users/admin/Practice/transformer-tutorial/data/transformer.pkl
TensorboardX
is also supported
python ../examples/pytorch_example.py \
...\
--tensorboard