When we release new model versions, our top priority is to make newer models smarter across the board. We are targeting improvements on a large number of axes, such as instruction following, factual accuracy, and refusal behavior. For instance, the gpt-4-0613 model introduced last month resulted in significant improvement on calling functions.

We look at a large number of evaluation metrics to determine if a new model should be released. While the majority of metrics have improved, there may be some tasks where the performance gets worse. This is why we allow API users to pin the model version. For example, you can use gpt-4-0314 instead of the generic gpt-4, which points to the latest model version. Each individually pinned model is stable, meaning that we won’t make changes that impact the outputs.

We are working hard to ensure that new versions result in improvements across a comprehensive range of tasks. That said, our evaluation methodology isn’t perfect, and we’re constantly improving it. One way to help us ensure new models get better at domains you care about, is to contribute to the OpenAI Evals library⁠(opens in a new window) to report shortcomings in our models.

We understand that model upgrades and behavior changes can be disruptive to your applications. We are working on ways to give developers more stability and visibility into how we release and deprecate models.