Understanding HEVC
Nov 11, 2014 6:38 PM, Jan Ozer
HEVC is the successor to the H.264 codec that has dominated the streaming, broadcasting, and conferencing landscapes for the past five years. HEVC has gotten lots of attention recently, but how close is the technology to actual deployment? Here, I will discuss the factors that impact HEVC’s true market readiness in the broadcasting and streaming market segments. I’ll start with a quick overview of the technology.
HEVC stands for High Efficiency Video Coding, and like H.264, the codec was jointly developed by the ISO/IEC Moving Picture Experts Group and ITU-T Video Coding Experts Group (VCEG). This is significant, because it means that HEVC automatically gains the support of many companies that routinely follow the recommendations of these standards bodies. It also means that the standard has two names, HEVC and H.265.
HEVC/H.265 development started back in 2004, with a first working draft specification approved in July 2012. In January 2013, the initial version of the specification was frozen and released for productization, enabling companies to start building encoders, decoders, and associated products.
WHERE’S THE BEEF?
There are two high-level benefits of HEVC. First is the supposed ability to deliver the same quality as an H.264 encoded stream at around half the data rate. The second is to enable the production of video with a resolution higher than 1080p, including 4K (2160p) and 8K (4320p).
We’ll examine the quality-related claims later in this article. However, in theory, these capabilities enable multiple market opportunities both for existing 1080p footage and 4K content. For example, regarding existing 1080p and lower-resolution footage, in the streaming space, replacing H.264 encoded streams with HEVC could reduce bandwidth costs by as much as 50 percent. Alternatively, video distributors could push higher-resolution video to viewers on lower bit rate connections, like mobile viewers watching via 3G. Similarly, in cable and satellite markets, using HEVC could double the number of channels available over existing connections.
The opportunities relating to 4K video are more discrete and slower-growing, since there is a relative paucity of 4K content and 4K TVs. Still, many movies are being rescanned at 4K resolutions, and 4K production is starting to accelerate. Certainly, there have been a relative flood of 4K TV sets available at rapidly decreasing prices. However, unless you produce TV sets or are in the movie or broadcast business, 4K doesn’t present a significant opportunity for most integrators, beyond potentially deploying 4K displays in boardrooms, trade shows, and other similar locations. Rather, most of the shorter term HEVC-related opportunities are in the streaming space, so I’ll focus the bulk of the analysis there.
For the record, note that the term Ultra High Definition Video, or UHD, technically refers to TVs, monitors, and projectors with at least 8 million active pixels, a display aspect ratio of 16×9, and at least 4K (2160p) resolution.
HEVC COSTS
Like H.264, HEVC will come with a royalty obligation, though the H.264 royalty enjoyed a far lower maximum and better-defined patent group. To explain, the primary H.264 patent pool is administered by MPEG LA, a company who gathers all the patent holders together, creates a royalty policy, and divvies up the proceeds amongst the group. MPEG LA is also administering the HEVC patent pool, and in January 2014, the company announced the proposed license terms for HEVC.
Briefly, the proposed HEVC royalty is 20 cents per HEVC encoder and decoder distributed by a legal entity to end users. You can ship up to 100,000 units per year with no royalty, with a per annum maximum royalty of $25 million, which will increase over time. In contrast, the maximum royalty for H.264 started at $5 million per year and is now $6.5 million per year. The licenses are, of course, separate, so if you license H.264 and HEVC, you pay both royalties.
Unlike H.264, which did impose royalties on pay-per-view and subscription content beyond certain minimums, there is no HEVC royalty on content. However, where the H.264 patent group assembled by MPEG LA included the vast majority of H.264 rights holders, a similar group for HEVC doesn’t include many companies who likely own HEVC-related IP rights, including Microsoft, Panasonic, Sony, Dolby Laboratories, Mitsubishi, Toshiba, Sharp, and Samsung. For this reason, beyond the MPEG LA HEVC-related royalties, companies deploying HEVC may have to write checks to multiple other parties.
That’s what we know about the potential benefits and costs of HEVC; let’s examine how it’s progressing in the marketplace.
ENCODER SIDE
Most encoding vendors have been developing live and on-demand HEVC encoders for more than a year, including Ateme, Digital Rapids, Elemental Technologies, Envivio, Harmonics, Telestream, Thomson, and many others. Several of these companies are already including HEVC encoding in their shipping products. Since the average cost of products distributed by these companies is likely more than $10,000 or so, the 20 cent royalty from MPEG LA and the unknown potential royalty from other HEVC rights holders isn’t a real concern.
HEVC support in lower-cost software encoders has been sparse, with the notable exception of HEVC encoding in the free DivX Converter (Figure 1). So far, Adobe (Media Encoder), Apple (Compressor), Sorenson (Squeeze), and Telestream (Episode) have all been mum about when they will add HEVC support to these products.
There are multiple chips under development that incorporate HEVC-encode from companies such as Broadcom, Sigma Designs, and Qualcomm. While the average selling price of these chips is much lower than the hardware encoders referenced above, any royalty or royalties will simply be passed along to the customer. For this reason, the royalty obligation really hasn’t affected this market.
DECODER SIDE
Again, there are multiple decoder chips under development that are destined for television sets and set top boxes. As on the encode side, the royalty won’t be a significant bar; they will simply be added to the price and paid for by the customer. This same structure works for fee-based decoders, and Adobe announced at NAB 2014 that they would add HEVC decode to PrimeTime, their premium distribution platform, where the costs can be recouped.
Figure 1. The free DivX Plus Converter is one of the few software-based encoders withHEVC support. Click here to see a larger image.
Where the proposed royalty structure fails miserably is on the free decode side. Specifically, today, there are just two free decoders: the VLC Player and the DivX player. In playback tests that I’ve performed, 720p HEVC files play very efficiently on many older computers, going back to the original Core 2 Duo class, which dates back to 2006. Therefore, publishers distributing at a maximum resolution of 720p could distribute HEVC today for computer playback and likely play on the vast majority of computers. The problem is, neither of the free players has the penetration necessary to publish HEVC to a general audience.
As you may recall, the event that forced H.264 to prominence in 2007 was its inclusion in Flash; within a few months, H.264 was available on 97 percent of all computers, and it was free. Of course, when Adobe added H.264 playback to Flash, the annual maximum royalty was a more affordable $5 million, and Flash was a strategic product that Adobe was willing to invest in.
Today, Flash is on the wane as the market moves towards HTML5, and the price tag for HEVC support has quintupled and must be paid in addition to the H.264 royalty. Adobe’s announcement that they plan to include HEVC decode in Prime- Time strongly infers that they won’t include it in the free Flash Player, eliminating the only simple mechanism to near ubiquitous penetration.
Understanding HEVC
Nov 11, 2014 6:38 PM, Jan Ozer
It’s likely that at some point, Apple, Google, and Microsoft will license HEVC for inclusion in their respective browsers, accessed via HTML5 commands. But it’s highly unlikely that Mozilla will pay the $25 million annual toll to include HEVC playback in Firefox; Mozilla is a much smaller company and has yet to license H.264. So even if the “Big Three” add HEVC decode to their browsers, the percentage of viewers who can actually play HEVC encoded video won’t include Firefox users or those running legacy browsers that aren’t HTML5 compatible.
Even on HEVC-capable versions of Safari, Chrome, and Internet Explorer, you’d have to publish via HTML5 to access HEVC playback. Today, however, virtually all major video publishers still publish using Flash, because HTML5 doesn’t support adaptive streaming, comprehensive digital rights management or captioning, not to mention live streaming. So even if HEVC playback is added to these browsers, it doesn’t mean that the major publishers can use it.
That’s just the desktop side. To date, neither Apple nor Google have announced plans to add HEVC playback to their respective operating systems. Until they do announce, there’s no way to tell how much of the installed base will be able to access the HEVC decode capability or will have the CPU horsepower necessary to play it. I will discuss this more in a moment when I cover the effect of HEVC playback on mobile battery life.
So it appears likely that any company wishing to access the bandwidth savings or greater reach afforded by HEVC will have to add HEVC playback to their existing players and pay the associated royalties. This may make economic sense if they can cut their bandwidth costs by half. However, some initial findings released by Netflix bring this number into doubt, at least in the short term.
HEVC VS. H.264 ACCORDING TO NETFLIX
As mentioned prior, most HEVC proponents in the encoder community claim that the new codec should be able to deliver H.264 quality at between 50 percent to 65 percent of the data rate. However, no actual users have confirmed these numbers; in fact, it’s been quite the opposite.
To explain, Netflix was one of the first publishers to actually deploy HEVC, encoding 4K versions of House of Cards with the new technology. At that resolution, HEVC is really the only choice. What about using HEVC to save bandwidth on existing 1080p and lower-resolution videos?
At the May 2014 Streaming Media East conference in New York, David Ronca, Netflix’s manager of encoding technology, stated, “We’re not seeing efficiency gains being claimed by HEVC encoding vendors.” In fact, as you can see from the slide I excerpted from the presentation, Netflix is seeing about the same quality as x264 after trying multiple encoders on 1080p videos and smaller. To add insult to injury, HEVC took about 10 times longer to encode.
Click here to see a larger image.
As you can see on the slide, Netflix is still bullish on HEVC, and most of their new infrastructure development involves 4K/HEVC. Still, if you’re a publisher looking for verification that HEVC delivers H.264-equivalent quality at substantially lower data rates, this clearly wasn’t it. While this isn’t the last word on HEVC quality, Netflix is the first vendor to weigh in. This makes their comments noteworthy, even if, most likely, not all that meaningful in the long term.
HEVC AND MOBILE BATTERY LIFE
The other not-so-bullish news about HEVC relates to battery life. To explain, many HEVC proponents claim that the most modern crop of tablets and smartphones will be sufficiently powerful enough to play HEVC once players are generally available. More conservative analysts say this may be so, but that HEVC will also reduce battery life to unacceptable levels. Most notable is Frost & Sullivan’s Avni Rambhia, who has opined on several occasions that mobile HEVC playback will only take off once HEVC hardware acceleration is available to preserve battery life. If true, this essentially means that HEVC will be a non-factor on all devices without HEVC acceleration, which includes the vast majority of the 2014 installed base.
A recent analysis in “Techspot” put numbers to this discussion. Specifically, Techspot’s Tim Schiesser compared H.264 and HEVC battery life in his recent article, “Media Players and Video Formats: A detailed Battery Life Analysis.” Here he found the HEVC playback reduced battery life by around 50 percent, as shown in the chart on page 48. That is, a 1080p video encoded in H.264 played for 8.6 hours, while the same video encoded in HEVC played for only 4.36 hours.
In this regard, Schiesser concluded, “While HEVC is by far the most impressive format in that it provides astonishing quality with a low bitrate, it’s extremely resource intensive to decode, especially on relatively weak tablet hardware. Until we get efficient hardware decoding for the format in future platforms, it’s unlikely we’ll see it overtake H.264 as the most popular video format.”
ANALYSIS
Where does that leave us? Clearly in the 4K broadcast and production markets, HEVC has its own momentum, which will likely accelerate as UHD sets drop in price and the installed base expands.
In the streaming space, the lack of a general purpose player is having a stifling effect on HEVC distribution on both desktops and mobile devices. Regarding desktop/notebook playback, it’s unlikely that Adobe will break the logjam by adding HEVC to Flash, and even if Apple, Google, and Microsoft add HEVC playback to their browsers, HTML5- based publishing is still too immature for most major distributors. Mobile markets are experiencing their own player deficit, plus the uncertainty regarding backwards compatibility and battery life, and the same quality-related concerns in the desktop space.
It looks like HEVC will be deployed publisher by publisher, rather than en mass, and that’s only after they perform their own comparison tests and determine that the bandwidth savings will justify the expense. The initial deployments will focus on closed systems, such as cable channels distributing 4K videos to a discrete set of users with 4K sets and HEVC capable set-top boxes. In the general streaming space, there are simply too many hurdles to see HEVC having much of an impact in the next year to 18 months.
HEVC THOUGHTS
NETFLIX’S INITIAL IMPRESSIONS OF HEVC
■ Evaluated many HEVC encoders, currently we see
• No efficiency gain vs. x264 (nonrealtime, <=1080p)
• Quality about the same as x264
• Much longer encode time (10x)
• But … very early in HEVC lifecycle
■ In ~2 years, we expect
• 20%-30% encoding efficiency vs. x264
• Encode time near x264
■ We are very excited about HEVC
■ Most all new dev is in 4K/HEVC
