Exclusive Articles Online: Abstracts

2021 SMPTE Motion Imaging Journal  
The recent dramatic increase in the consumption of video content requires efficient video coding standards, which is specifically true for ultrahigh-definition (UltraHD) resolutions, such as 4K and 8K (i.e., 3840 × 2160 or 7680 × 4320 resolutions in terms of luma samples, respectively). The well-known high-efficiency video coding (HEVC) [H.265/Moving Pictures Expert Group (MPEG)-H] standard was approved in 2013. Although HEVC provides approximately 50% coding gain, when compared to its
more » ... or advanced video coding (AVC) (H.264/MPEG-4), its adoption is still relatively slow. In addition, larger bitrate savings than those provided by HEVC are currently in demand. At the same time, work on the Versatile Video Coding (VVC) and Essential Video Coding (EVC) standards started in 2018. After intensive development efforts that continued for two-and-a-half years, these two video coding standards have been recently finalized. VVC (H.266/MPEG-I) was developed jointly by the MPEG and the International Telecommunication Union-Telecommunication (ITU-T) Video Coding Experts Group (VCEG), however, EVC (MPEG-5) is developed only by MPEG. In this study, we compare the performance of EVC and VVC, in terms of both coding gains and computational complexity, to their predecessor-the HEVC standard. In addition, given the growing popularity of the AV1 video codec, which was recently developed by the Alliance for Open Media (AOM), we also include AV1 as an alternative baseline and provide corresponding comparison results. According to the experimental results, which have been carried out in a constant bitrate (CBR) mode, EVC saves about 30% bitrate compared to HEVC for encoding 4K/2160p entertainment content (such as video on demand) in terms of Bjøntegaard-delta bitrate (BD-BR) peak-signal-to-noise ratio (PSNR)YUV, while increasing the encoding computational complexity by approximately five times. However, VVC saves about 40% larger bitrate while increasing the encoding computational complexity by more than nine times. When the performance of HEVC CBR encoding (i.e., with the rate control disabled) was compared to that of AV1 VBR encoding (i.e., with the rate control enabled), it was found that AV1 provides bitrate savings of about 20% compared to HEVC for encoding 4K/2160p video sequences, as a tradeoff of an encoding computational complexity increase by a factor of approximately 4. The authors find both EVC and VVC to be very promising successors to HEVC in terms of coding gains and computational complexity, but the jury is still out on the speed of their adoption. Frame interpolation is the process of synthesizing a new frame in between the existing frames in an image sequence. It has emerged as a key algorithmic module in motion picture effects since its large-scale use in the making of the movie The Matrix. This study presents a review and a new unified view of the classical algorithms used to create in-between frames, representing most of the past 20 years of their evolution. This is used to benchmark the recent deep learning algorithms against two of the best industrial retimers available. A significantly expanded data set of 140,000 frames is used for testing. In the context of high-resolution material, we find that techniques relying principally on deep neural networks (DNNs) do not clearly outperform the classical ideas. It is only with the emergence of hybrid approaches in 2019 that we see DNNs adding significantly to the performance in this space. Despite the hype surrounding DNNs, we find that there is still more work to be done.
doi:10.5594/jmi.2021.3076377 fatcat:quro3l4w7vgkzaof7qh7nhr6bm