HEVC is the new video compression standard, successor to AVC/H264. It is jointly developed by ITU-T VCEG and ISO/IEC MPEG standard bodies and claims to save 50% bit-rate (by keeping same quality) compared to it’s predecessor AVC/H264. Along with primary goal of achieving higher compression and accommodating higher resolutions like 8K, it also embraced parallel architectures by introducing tools for parallel processing. HEVC is also a block based hybrid video codec and mostly similar to AVC, except for completely new in-loop non-linear filter named Sample Adaptive Offset (SAO) module. In order to achieve higher compression, bigger coding block sizes with hierarchical splitting nature are introduced along with tweaking & enhancements in existing video compression modules. The key tweaks/enhancements are (i) increasing number of intra prediction modes from 9 to 35, (ii) replacing MVP & skip with AMVP & merge list concepts, & Asymmetric Motion Partition (AMP) (iii) enhancements for frac-pel interpolation and (iv) variable transform unit sizes from 4x4 to 32x32. For the purpose of parallel processing, HEVC has eased & modified the de-blocking filter operation and introduced tools called Tiles and Wave-front Parallel Processing (WPP). At high level syntax, some modules are overhauled to improve the error resiliency, interface to system and provide new functionalities. The high level syntax includes NAL unit headers, parameter sets, picture partitioning schemes, reference picture management, and SEI messages.
The computational complexity of HEVC decoder is not expected to be much higher than that of H264/AVC decoder. HEVC decoder is considered to be 1.2x – 1.5x complex and possible to have a real-time software solutions on current hardware with minimal effort. But things look different from encoder perspective, the complexity is assumed to be 10x more compared to AVC encoder. Many mode combinations are introduced due to flexibility provided with quadtree structures, increased number of intra prediction modes and SAO offset estimation algorithms. If any encoder has to exploit full capabilities of HEVC, that encoder is expected to be several times more complex than AVC/H264 encoder. In order to achieve real-time encoder or encoder with decent frame rate (fps), it is required to trade-off compression rate marginally by reducing substantial computational complexity. Hence, it is going to be subject of innovating & research area to develop algorithms that handle the balance of trade-off b/w computational complexity & compression rate.
The higher complexity is generally the result of evaluating all the mode combinations in brute-force way. Execution can be accelerated, by developing approaches to figure out relevant combinations and evaluating only them, with little compromise in video quality. Along with avoiding insignificant combinations, introduction of threshold based early termination approaches would help in further improvement of execution time. In general, spatial & temporal neighbor’s statistics get used in above two approaches to predict relevant mode combinations and empirical derivation of thresholds for early termination.
We developed a simple & efficient HEVC encoder software solution (from scratch) by making smart trade-offs between video quality and complexity. This solution is multi-core friendly with good scalability, easily portable on homogeneous/heterogeneous architectures and supports almost all tools of HEVC. Along with efficient prediction & Rate control algorithms, the additional memory transfer overheads in the context of multicore are well understood and so efficient definition of data structures. In our upcoming webinar talk titled “The simplified & efficient HEVC encoder solution”, we present the comparison results of complexity & video quality between our solution and HM13.0. In order to cover various tools, the comparison is carried out for All-Intra (AI), Random Access (RA), Low-delay B (LDB) and Low-delay P (LDP) coding configurations, with fixed Qp and Rate control.
Our publications on HEVC:
- Design approaches for Mode decision in HEVC Encoder – Exploiting Parallelism at CTB Level
- SAO filtering inside CTU loop for High Efficiency Video Coding (HEVC)
- Analysis and approximation of SAO estimation for CTU-level HEVC encoder
- Effective approach to reduce complexity for HEVC intra prediction in inter frames
- Novel approach to limit Non-square prediction unit (PU) estimation in HEVC
- SAO in CTU decoding loop for HEVC video decoder
Does HM Software Implements all the intraprediction modes of H.265 ?????????????????
Yes, HM evaluates all intra prediction modes. It has 2 stage evaluation, first stage considers all modes to filter out top 3 or 8 intra modes based on rough RD. Then, second stage carries out full RDO for all these top 3 or 8 intra modes. Hope it clarifies your question.
What is being done on the hardware front to support the performance requirements of HEVC?
Gagandeep Kaur, HM evaluated all intra prediction modes. It has 2 stage evaluation, first stage considers all modes to filter out top 3 or 8 intra modes based on rough RD. Then, second stage carries out full RDO for all these top 3 or 8 intra modes. Hope it clarifies your question.