HEVC, the future video codec needs no introduction as it is slowly being injected into multimedia market. Irrespective of very high execution complexity compared to its earlier video standards, there are few software implementations proven to be real time on multi-core architectures. While OTT players like Netflix are already on the process of migrating to HEVC, 4k TV sales are increasing rapidly as HEVC promises to make UHD resolution a reality. ASIC HEVC encoders and decoders are being developed by major SOC vendors that will enable HEVC in hand held battery operated devices in the near future. All these developments are motivated by one major claim
'HEVC achieves 50% better coding efficiency compared to its predecessor AVC'.
Initial experimental results justifies this claim and shows on an average 50% improvement using desktop encoders and approximately 35% improvement using real time encoders compared to AVC. This improvement is mainly achieved by set of new tools and modification introduced in HEVC but, is not limited to higher block sizes, higher transform sizes, extended intra modes and SAO filtering. You need to note that, there is 50% improvement on an average when results for a set of input contents are accounted. This does not ensure half the bit rate with same quality for every input content. Now, we will tear down this 'average' part of the claim and discuss in which situation HEVC is more efficient considering four main factors.
- Resolution
- Bit rate
- Content type
- Delay configuration
1. Resolution:
HEVC is expected to enable higher resolution video and to substantiate this, results prove higher coding efficiency gains at higher resolutions. At 4k resolutions, compression gains can be more than 50%. This makes it possible to encode 4k resolution contents with 30 frames per second at bit rates of 10 to 15 Mbps with decent quality. Reason behind such behavior is a feature/tool introduced in HEVC that contributes more than half the coding efficiency gains i.e. higher coding block sizes. At high resolution, larger block sizes leads to better compression as neighboring pixels have higher correlation among them. It is observed that 1080p sequences have on an average 3-4% better compression gains compared to their 720p counterparts.
2. Bitrate:
Encoder results indicate better compression gains at lower and mid-range bit rates compared to very high bit rates. At low QPs, transform coefficients contribute more than 80% of the bits. Having larger coding unit helps in saving bits used for MVD and other block header coding. Bigger transform blocks will have better compression gains as it has large number of coefficients for energy compaction. But at high bit rate, header bits takes a small percentage of the bit stream suppressing the gains from higher block sizes. And low QPs reduce the number of non-zero coefficients which limit the gains possible from high transform blocks.
We have observed that, for ParkJoy sequence BD-rate gains were 8% better at QP range of 32-40 compared to QP range of 24 - 32. Similar behavior has been found in most of the sequences.
3. Content type:
Video frames with uniform content has shown better compression gains as it aids higher block sizes. Encoder tend to select lower block sizes for frames with very high activity which makes the block division similar to AVC, reducing efficiency of HEVC. Lower correlation between pixels limits compression gains produced by larger transform blocks. Streams such as BlueSky having significant uniform content in a frame produced 10% better gains compared to high activity streams like ParkJoy.
On the other hand, videos with stationary contents or low motion videos produced better gains as higher block sizes are chosen in inter frames for such contents.
4 Delay Configuration:
Video encoders can be configured to use different GOP structures based on application needs. Here, we are going to analyze compression gains of different delay configurations w.r.t AVC. Firstly, all intra frame cases (used for low delay, high quality, error robust broadcast application) produce nearly 30 - 35% gain deviating from 50% claims of HEVC. Higher block sizes are not very effective in intra frames and gains produced with the help of other new tools such as additional intra modes, SAO etc., limits the average gain to 35%. Hence HEVC-Intra will not be 50% better than AVC-Intra.
It is also observed that, Low delay IPP GOP configuration produces slightly better gains (Approximately 3-4% on an average) compared to Random Access GOP configuration with B frames. This could be purely due to the implementation differences in HM reference software.
Thus 50% gains cannot be achieved for all video contents but many encoder implementations have proven nearly 50% or more average gains compared to AVC. Such compression efficiency can have a major impact on broadcast and OTT applications where half the bandwidth consumption can be reduced, thus half the costs. Though other emerging video compression standards such as VPx, Daala claim similar gains, they are yet to be proven. It would be interesting to see how VPx, Daala can change the future of HEVC. But right now one thing is sure, HEVC is the future video codec and it is here!
Nice analysis!
As far as I have been following, I believe that HEVC also incorporates non-square transform blocks identified by contours in an image. So, do non-square transform structures help in providing better compression gains or just the increase in size of the transform blocks has helped more? Would be interesting to analyze which of the two contribute more to compression gains in different activity scenes.
Hi Uday,
Non square transform blocks were removed from the standard due to its complexity (I remember HM6.0 having non-square partition and mostly removed after Hm7.0 or HM8.0). Non-square transform blocks do help in improving compression gains by allowing flexibility in selecting blocks with better correlation at object boundaries. But extra complexity involved is too much of a compromise and hence it was discarded.
There is no external control on sub pixel motion estimation as of now, Don’t you think it is required. i have conducted some experiments, analytic results shows that there is no significant increase in quality in some video. i have been pointed out this also to ITU, Any similar experience ?
Hi Sathyanarayana,
I agree effect of sub pixel Motion estimation can be limited in few low complexity sequences but they become very efficient especially for high motion video content. Sub pixel Motion estimation is one of the major tool which greatly contributes to the compression gains though it is not new in HEVC.
Prashanth,
When it comes to the high motion video content no one will argue that Sub pixel motion estimation is not a major tool, but if standers provides its control as a option like in AVC, we can switch off when the application like video conferencing, low motion videos etc, so that we can avoid additional commutations. what do you think?
Satya
Yes Sathyanarayana. Additional control at PPS level would have helped save bits in MVD coding when encoder does not produce frac-pel MVs for low complexity video solutions. Not sure why this was not included in the standard !!
HI Prashanth shall i know about scope of porting HEVC Main profile ENCODER on iMX (ARM core) processor (1GHz) with 30 fps