Khronos Concludes Vulkan Video Extensions for Enhanced H.264 and H.265 Encoding

In a major development for video encoding and decoding, Khronos has announced the finalization of the extensions to enable encoding of H.264 and H.265 video streams with the release of Vulkan 1.3.274. These extensions, collectively known as Vulkan Video, provide a standardized and efficient way to produce high-quality video using hardware accelerators.

The Vulkan Video extensions have undergone significant improvements since their initial release in April 2021. They now include a bidirectional interface, known as overrides, which ensures compliance with output video streams. This interface also allows applications to provide optimization overrides, giving implementations more flexibility to optimize for specific usage scenarios.

One of the key features of the Vulkan Video extensions is the encoder rate control API, which offers a rich set of features for configuring rate control parameters. This API allows users to choose between automated operation or low-level tweaking of frame parameters, providing a balance between efficiency and customization.

Another important aspect of encoder configuration is the encoder quality levels. Video encoder implementations can now report the number of quality levels supported for a given video profile and usage. This information can be retrieved using the vkGetPhysicalDeviceVideoEncodeQualityLevelPropertiesKHR API.

To facilitate implementation overrides and optimizations, applications are expected to retrieve the encoded video session parameter bitstream segments from the implementation using the vkGetEncodedVideoSessionParametersKHR API call. This allows for detailed analysis and compliance with bitstream requirements.

In addition to the encoding extensions, Khronos is also releasing a maintenance extension that incorporates community and industry feedback. This extension improves flexibility for both decoding and encoding, allowing decoding implementations to create images usable with video decoding without explicitly specifying the video profiles.

Furthermore, changes have been made to the Vulkan Video decode extensions. Applications are now required to provide pSetupReferenceSlotKHR for non-reference pictures, as it was discovered that some implementations require the use of reconstructed picture resources and DPB slots during decoding or encoding.

Overall, the finalization of the Vulkan Video extensions marks a significant milestone in video encoding and decoding. With standardized and efficient encoding capabilities, developers can leverage the power of hardware accelerators to produce high-quality video for a wide range of applications, from real-time streaming to server-scale transcoding.