AOMedia Video Codec Working Group Announces Availability of Libaom v3.14 “Pinkie Pie”

The AOMedia Video Codec Working Group is pleased to announce that libaom v3.14, aka “Pinkie Pie,” is now available. This release makes available all the improvements the libaom team made during the 8 months since the last libaom feature release (v3.13.0). In addition to video encoding in VOD and realtime modes, this release improves layered image encoding and software security.

What Key New Features Does This Release Include?

Pinkie Pie boasts significant compression efficiency and perceptual quality improvements for layered image encoding. We delve into layered image encoding improvements in the next section.

Low complexity decode mode (the encoder optimizations that produce encoded bitstreams that can be decoded at low computational complexity) now supports good-quality encoding (speed 1 to 3) for not only vertical videos (608p to 1080p) but also horizontal videos (720p to 1080p). It achieves close to 30% decoding cost savings with a tightly constrained compression efficiency loss.

Encoder algorithms in partition/mode search, loop restoration, CDEF, temporal filter, and rate control were tuned or reworked in this release, achieving a massive encoder speedup alongside significant coding efficiency gains.

• Delivered ~40% encoder time reduction for speed 0 and 1.
• Delivered compression efficiency gains across presets: speeds 0 and 1 see a 1.5%+ SSIM gain, a ~0.2% PSNR gain, and a 7% VMAF gain, while speeds 2 and 3 yield a 2.6%+ SSIM gain, a 1.4 -1.7% PSNR gain, and a 7 - 8% VMAF improvement.

The improvements in this release provide substantial egress savings and computational cost reduction for YouTube.

Realtime (RTC) encoding continues to be an active area of development. This release extended encoding with lookahead (also known as lagged encoding) to RTC mode, added support for speculative encoding (encoding a frame without updating encoder state), and improved the quality and scroll detection for high-resolution screen content.

Since February we have been receiving increasingly high-quality reports of software security issues found by AI tools. These powerful AI tools are a great addition to the arsenal of security scanners and complement fuzzing, static analysis, and sanitizers. This release contains several fixes of security issues, such as integer overflows, excessive memory allocations, and memory errors, typically triggered by high bitrates or dynamic changes to SVC encoder settings. (See the release notes for details.) This critical work is ongoing and the next libaom release will include additional security fixes.

Layered Image Encoding Improvements in This Release

The AV1 Image File Format (AVIF) allows a still image to use multiple (up to four) AV1 spatial layers. Given a layered AVIF image, an application can either render only the highest spatial layer, or render all the intermediate layers and the highest spatial layer, resulting in a form of progressive decoding. A visual example of progressive decoding is provided at the end of this section.

The new releases of libavif (v1.4.2) and libaom (v3.14) contain new layered image encoding features and tweaks to address high and unpredictable file size overhead because older versions did not handle this use case properly. In libaom, these improvements mainly come from the following two changes:

• Extended the AOM_TUNE_IQ (Image Quality) tuning mode to work with inter-frame encoding modes (good-quality and realtime), improving the visual quality and compression efficiency of layered images compared with AOM_TUNE_SSIM. When encoding 2-layered images, AOM_TUNE IQ improves compression efficiency by up to 15% in good-quality mode and by up to 30% in realtime mode.
• Added a new encoder option (use_fixed_qp_offsets=2) to disable quality boosting of the key frame, allowing full control of each layer's quality level. In general, a high-quality key frame helps reduce error propagation, so it is advantageous to boost the quality of a key frame. In a layered image, however, the key frame (the base layer) can have a lower quality than the rest of the frames, so quality boosting is counter-productive.

The libavif v1.4.2 release can leverage these two improvements to achieve better quality at the same size, with less file size overhead. Additionally, libavif's 2-layer image encoding (avifenc --progressive) downscales the first layer by a half for further size savings, with a faster time to first render.

We illustrate the improvements with an example. The original image is a JPEG image of size 2,790,013 bytes. To showcase file size differences better, we strip the Exif and XMP metadata from the encoded AVIF images.

Original JPEG image (2,790,013 bytes)

As a baseline, we first encode the image as a regular, non-layered image at speed 6 and quality level 60 (using the command line avifenc -s 6 -q 60 --ignore-exif --ignore-xmp photo.jpg photo.avif). The file size is 785,269 bytes.

Baseline: regular, non-layered AVIF image (785,269 bytes)

We then encode the image as a layered image at speed 6 and quality level 60 (using the command line avifenc --progressive -s 6 -q 60 --ignore-exif --ignore-xmp photo.jpg photo.avif) with different versions of libavif.

• With libavif v1.3.0 and libaom v3.12.1, the file size is 1,542,974 bytes, nearly twice the size of the baseline!
• With libavif v1.4.1 and libaom v3.13.2, all intra encoding is used as an interim workaround, and the file size is 833,672 bytes. Compared with the baseline, the file size overhead is (833,672 - 785,269) / 785,269 = 6%.
• Finally, with libavif v1.4.2 and libaom v3.14.1, improved inter frame encoding is used, and the file size is 801,338 bytes. Compared with the baseline, the file size overhead is (801,338 - 785,269) / 785,269 = 2%.

Layered AVIF image with improved inter frame encoding (801,338 bytes, a 2% file size overhead)

In the final layered image of 801,338 bytes, the base layer (layer 0) is 26,061 bytes and the enhancement layer (layer 1) is 771,821 bytes. When an application decodes this image progressively, it can render the base layer immediately after receiving just 4% of the image (29,517 bytes), and then render the enhancement layer after receiving the whole image.

Base layer (layer 0) of the layered AVIF image (29,517 bytes including file headers, 4% of total image size)

Enhanced layer (layer 1) of the layered AVIF image

Where can the libaom v3.14 Pinkie Pie release be found?

As of this writing, the latest release in the libaom v3.14.x series is libaom v3.14.1. The source code of the release can be checked out from the git repository https://aomedia.googlesource.com/aom using the release tag v3.14.1. Alternatively, the release tarball, along with the signature file, can be downloaded from https://storage.googleapis.com/aom-releases/libaom-3.14.1.tar.gz and https://storage.googleapis.com/aom-releases/libaom-3.14.1.tar.gz.asc respectively.

Thank you to the Video Codec Working Group for their efforts, and a special thanks to the Arm team for their contributions to the AArch64 optimizations in this release.