In 2020, OpenAI created Image GPT (iGPT), a Transformer-based model that works with sequences of pixels rather than sequences of words. They discovered that, similar to GPT models for text which can produce believable samples of natural language, iGPT is able to “generate sensible image completions and examples,” given an input sequence of starting pixels.