The voice capability is powered by a text-to-speech (TTS) model, capable of generating human-like audio from just text and 15-seconds of sample speech.

The TTS system is developed by helping the model understand the nuances of speech from paired audio and transcriptions. The model learns to predict the most probable sounds a speaker will make for a given text transcript, taking into account different voices, accents, and speaking styles. After this, the model can generate not just spoken versions of text, but also spoken utterances that reflect how different types of speakers would say them.

From there, generating audio with the TTS model requires just a 15-second sample from the speaker and the corresponding text. The model is not fine-tuned for any specific speaker, there is no model customization involved. Instead, it employs a diffusion process, starting with random noise and progressively de-noising it to closely match how the speaker from the 15-second audio sample would articulate the text.

We first developed Voice Engine in late 2022. Early on, to assess the capabilities and limitations of our Voice Engine model, we tested it internally using a mix of public and private voice samples. This internal prototype was essential for our alignment and safety research, informing our safeguards, and is a continuation of our commitment to understand the technical frontier.