https://arxiv.org/abs/2103.10360
All NLP Tasks Are Generation Tasks: A General Pretraining Framework (Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, Jie Tang)
generative pretraining으로 nlu 모델 성능 끌어올리기. 위의 gpt 파인튜닝과 묘하게 조화가 되네요. ㅎㅎ
#pretraining #language_model