Model distillation involves fine-tuning smaller, cost-efficient models using outputs from more capable models, allowing them to match the performance of advanced models on specific tasks at a much lower cost. Until now, distillation has been a multi-step, error-prone process, which required developers to manually orchestrate multiple operations across disconnected tools, from generating datasets to fine-tuning models and measuring performance improvements. Since distillation is inherently iterative, developers needed to repeatedly run each step, adding significant effort and complexity.

First, create an evaluation⁠(opens in a new window) to measure the performance of the model you want to distill into, which in this example will be GPT‑4o mini. This evaluation will be used to continuously test the distilled model’s performance, to help you decide whether to deploy it.

Next, use Stored Completions to create a distillation dataset of real-world examples using GPT‑4o’s outputs for the tasks on which you want to fine-tune GPT‑4o mini. You can do this by setting the ‘store:true’ flag in the Chat Completions API to automatically store these input-output pairs without any latency impact. These stored completions can be reviewed, filtered, and tagged to create high-quality datasets for fine-tuning or evaluation.

Finally, use this dataset to fine-tune GPT‑4o mini. Stored Completions can be used as a training file when creating a fine-tuned model. Once the model is fine-tuned, you can go back to Evals to test whether the fine-tuned GPT‑4o mini model meets your performance criteria when compared to GPT‑4o.