This post introduces a solution based on Amazon Nova 2 Sonic (Nova Sonic) and Amazon Kinesis Video Streams WebRTC (WebRTC) that addresses these challenges. WebRTC is responsible for dynamically adjusting the bitrate in unstable networks, which helps to maintain audio quality while reducing dropped connections. Nova Sonic provides effective human language dialogues, so users can interact more naturally in their chosen language. Both services are fully managed by AWS, so they scale automatically with high resilience. AWS also provides open-source samples that you can use as a starting point for your own application.

In this post, we’ll walk through the solution architecture, implementation patterns, and two real-world scenario examples.

Traditional voice agent pipelines typically involve separate modules for speech recognition, language processing, and speech synthesis. Nova Sonic offers a unified speech-to-speech architecture that enables real-time voice conversations between users and AI agents with low latency.

With unified speech understanding and generation, Nova Sonic delivers natural, human-like conversational AI. The Nova Sonic model provides different speaking styles and tool interfaces for external agents. You can use it to build a more responsive and intuitive voice interface with higher contextual awareness.

A typical streaming pipeline comprises three main components: media source, media server, and media consumer. The previous diagram shows these components and their respective protocols, such as RTMP, RTSP, HLS, MPEG-DASH, and WebRTC.