Training
Improving Language Understanding by Generative Pre-Training
OpenAI·June 11, 2018
Alec Radford, Karthik Narasimhan, Tim Salimans
TL;DR
Introduces GPT: pretrain a Transformer decoder on unlabeled text, then fine-tune per task.
Why it matters
The original GPT and the generative-pretraining recipe behind everything that followed.