Over the past two years, we rebuilt our entire deep learning stack and, together with Azure, co-designed a supercomputer from the ground up for our workload. A year ago, we trained GPT‑3.5 as a first “test run” of the system. We found and fixed some bugs and improved our theoretical foundations. As a result, our GPT‑4 training run was (for us at least!) unprecedentedly stable, becoming our first large model whose training performance we were able to accurately predict ahead of time. As we continue to focus on reliable scaling, we aim to hone our methodology to help us predict and prepare for future capabilities increasingly far in advance—something we view as critical for safety.

We are releasing GPT‑4’s text input capability via ChatGPT and the API (with a waitlist⁠). To prepare the image input capability for wider availability, we’re collaborating closely with a single partner⁠(opens in a new window) to start. We’re also open-sourcing OpenAI Evals⁠(opens in a new window), our framework for automated evaluation of AI model performance, to allow anyone to report shortcomings in our models to help guide further improvements.

In a casual conversation, the distinction between GPT‑3.5 and GPT‑4 can be subtle. The difference comes out when the complexity of the task reaches a sufficient threshold—GPT‑4 is more reliable, creative, and able to handle much more nuanced instructions than GPT‑3.5.

To understand the difference between the two models, we tested on a variety of benchmarks, including simulating exams that were originally designed for humans. We proceeded by using the most recent publicly-available tests (in the case of the Olympiads and AP free response questions) or by purchasing 2022–2023 editions of practice exams. We did no specific training for these exams. A minority of the problems in the exams were seen by the model during training, but we believe the results to be representative—see our technical report⁠(opens in a new window) for details.

We also evaluated GPT‑4 on traditional benchmarks designed for machine learning models. GPT‑4 considerably outperforms existing large language models, alongside most state-of-the-art (SOTA) models which may include benchmark-specific crafting or additional training protocols: