Launched in late March 2018 by the Alliance for Open Media (AOMedia), and backed by Apple, Google, Microsoft, Intel, and Cisco, AV1 challenged traditional business models and enabled innovation in social video, mobile video, VR and online TV without having to pay historical IP owners. AV1 is in solid standing to become a widespread standard in the future. Here are five key encoding and decoding techniques that make AV1 an interesting choice to use in video streaming.
Film grain synthesis: During digital video compression, film grain creates massive problems as it is hard to recognize and creates a lot of traffic in the bitstream. Finding a way to re-apply it later is a desirable solution. This idea forms the base for AV1’s film grain synthesis. The goal is to de-noise the initial content before encoding it and then re-add the noise or grain effect before output during the decoding process. This way, the unnecessary information does not have to be transmitted at all and the overall load of data can be reduced substantially. The potential in bandwidth savings for content providers using this technology is enormous.
Constrained Directional Enhancement Filter: The Constrained Directional Enhancement Filter is among the most impactful of additions to the range of filters. CDEF merges two existing filters: a directional de-ringing filter and the constrained low pass filter (CLPF). CLPF is applied to filter out artifacts that stem from quantization errors and have not been corrected through the preceding application of a de-blocking filter. The directional de-ringing filter works by recognizing edges within each block and identifying their orientation. It then conditionally applies a directional low-pass filter along those edges, resulting in a smoother picture. CDEF merges the two filters and works by analyzing the contents of each block – smoothing out artifacts along edges and de-blocking the picture. The search for the filtering parameters—direction and variance—is applied on the decoder’s end, after the video has already been encoded. The filtering process is also performed by the encoder, in order to get the correct reference frames. Since the filtering operation can be run on the consumer hardware, required network bandwidth can be reduced.
Warped motion and global motion compensation: Warped motion compensation is a particularly interesting technique that predicts spatial movement trajectories within videos. Based on calculated predictions, redundant information is identified and omitted in the coding process, resulting in a significant reduction in the required data load. Global motion compensation predicts motion for an entire frame and uses this analysis to limit the amount of information transmitted in the bitstream. Motion-compensation algorithms have been used and theorized on for some time, but only on a two-dimensional level. AV1 marks the first time that nonplanar motion compensation has been implemented into a video codec. Particularly suited to predict large area movements, like background motion or camera movements, this technique can also handle consistent backgrounds and color schemes very effectively, which is one of the reasons why animated videos tend to deliver great encoding results—even with very high levels of compression.
Increased coding unit size: An increase in block size is an effective way to scale the compression process along with high-resolution contents. Each frame is partitioned up into individual coding units or blocks, which are then processed individually during the coding procedure. Consequently, smaller resolutions like 1280×720 (720p) can be divided into blocks with an individual size of 64×64 quite easily, whereas the same block size is less practical for large resolutions, such as 7680×4320 (8K UHD).
Non-binary arithmetic coding: For codecs such as HEVC or AVC, every symbol which is entered into the arithmetic coding engine has to be binary. With AV1, these symbols can also be non-binary—meaning that they can have up to eight possible values instead of just two. The symbols are then processed by the arithmetic coding engine, which produces a binary bitstream as output. Both ends—encoder and decoder—operate using probability calculations to estimate how many output bits will be created from a given symbol. Theoretically, any given input symbol could therefore produce multiple bits or even just a fraction of a bit. Although non-binary coding renders the coding process more complex by combining multiple values into a single symbol, it is still less complex than if it were one bit per symbol. One major benefit lies in the possibility to process more symbols per clock cycle using this procedure. As clock cycles have to be performed serially, non-binary coding achieves improvements by allowing multiple symbols to be handled during each serial cycle.
Over the last decade, video delivery has evolved rapidly. Up until now, a lot of the focus has been on video quality and improving the user experience, but the attention is now on video delivery. AV1 is going to have massive impact on video streaming in the near future. User demand for high-quality video streaming is already high, and the coming generation of high-resolution mobile devices and VR-enabled devices is about to push its way into mainstream availability.
