Agora Uses AV1 to Ensure a Smoother Live Streaming Experience

Agora has made a name for itself as a global leader in real-time engagement platform-as-a-service (PaaS), powering 70 billion minutes of video and voice every month. With its easy-to-use APIs, developers can seamlessly integrate voice, video, live streaming, chat, whiteboard, and AI features into their apps to create multifaceted and highly immersive live experiences.

However, live video calls and streaming often experience challenges like unstable networks, packet loss, and limited bandwidth. These issues can cause freezing, stuttering, pixelation, and dropped connections, hurting the real-time video experience and lowering user engagement. Agora Video Engineer Wei Dai discussed with AOMedia how Agora's use of AV1 in their encoding technology helps overcome these problems in use cases ranging from telehealth, education, future of work, and gaming to live shopping, media and entertainment, metaverse, and social. Moreover, Agora's AV1 has been widely implemented and is well-suited to support scenarios requiring immersive ultra-high resolution video, such as the next generation of VR headsets.

The Road to Success

At the heart of Agora’s solution is Agora's Coding Technology (ACT), designed to enhance user experiences with the power of AV1. Using AV1 and advanced compression techniques, ACT adjusts in real time to ensure clear visuals, efficient bandwidth use, and smooth, high-quality video from capture to playback.

Building this proprietary solution on AV1 took time. Agora’s team began testing AV1 in 2021 when Chrome first introduced support for the codec.

Dai explains, “After benchmarking all available AV1 open-source codecs, we found their complexity to be excessively high. We launched the Agora AV1 (AAV1) project to simplify the AV1 encoder, making it feasible for most devices to run AV1 encoding with minimal impact.”

After three years of fine-tuning, Agora introduced Agora Video SDK 4.3.1, which supports AV1 encoding and decoding. This SDK uses Adaptive Video Optimization™ (AVO) with machine learning algorithms to dynamically adjust video quality based on network conditions, device performance, and bandwidth. The result? AV1-enabled AVO ensures top-tier video quality, even when network connections fluctuate.

The Key to Overcoming Network issues

To tackle unstable networks, Agora created a strong rate control algorithm that helps AV1 adapt to complex network fluctuations. Since screen sharing is a key feature of live video calls, the specialized rate control algorithm has been optimized for this feature.

“Our AV1 codec now performs seamlessly across various network conditions, delivering the best user experience compared to other real-time codecs,” enthuses Dai.

Live video calls require real-time encoding, so the encoding speed must match the speed of the live video.

“When we initiated the AAV1 project, the encoding speed was 50 times slower than x264, despite achieving a 53% bitrate reduction. This speed was impractical for both mobile devices and PCs,” said Dai.

“After three years of optimization, the complexity of our fastest AAV1 profile is now just 75% of the x264 profile, while still achieving over a 42% reduction in bitrate,” continued Dai.

Dai and his team also heavily optimized the multi-thread in AAV1 to speed up the high-resolution video encoding. Now Agora’s AAV1 can run 720P on most mobile devices and 1080P on almost all PCs.

A Winner with Agora’s Customers

Agora's customers aim to deliver the highest-quality live video experience, and they have several reasons for preferring AV1. Dai notes that their top reasons include its compatibility with WebRTC, its superior encoding speed and compression efficiency, and its royalty-free model.

“There’s a significant improvement in user experience with AV1 compared to H.264, coupled with virtually no increase in power consumption,” said Dai. “With growing support for hardware encoders and decoders across more platforms, we anticipate AV1 will become increasingly widespread in various applications.”

A Vital Part of the Video Ecosystem

Dai cites several reasons for the recent increase in AV1 adoption. “AV1 is the first encoder to support screen sharing tools in its main profile, making it highly useful in many scenarios,” he said.

He continued, “Its advanced coding tools, such as the constrained directional enhancement filter (CDEF) and film grain synthesis, also significantly enhance the viewing experience.”

“We now have AAV1, an AV1 encoder, that operates 25% faster than x264 while achieving over a 42% reduction in bitrate. Our AV1 solution demonstrates that AV1 is a viable choice for both speed and bitrate efficiency among all encoders.”

Future Plans

Dai’s team plans to continue to build successful solutions on top of AV1. “In addition to its current use in real-time communication, we plan to extend AV1 encoding to multi-view, high-bit depth, and high-fidelity scenarios,” said Dai.

“Furthermore, we aim to optimize AV1 transcoding in the near future, broadening its application across various video codec uses.”