GPT-2

OpenAI GPT-2 is an open-source artificial intelligence released in February 2019. It was proposed by Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei and Ilya Sutskever in their paper “Language Models are Unsupervised Multitask Learners”. GPT-2 is a unidirectional Transformer model that has been pre-trained using language modeling on a large dataset of approximately 40 gigabytes of text data. The goal of the training process is to predict the next word within some text given all previous words. This simple objective contains demonstrations of many tasks across diverse domains due to the diversity of the dataset used. Compared to its predecessor GPT with fewer parameters and smaller datasets, OpenAI GPT-2 boasts more than 10 times as much data and 10 times as many parameters for improved accuracy and performance.