# Speeding up VP9 Intra Encoder with Hierarchical Deep Learning Based Partition Prediction

## TL;DR

This paper introduces a hierarchical deep learning model to predict VP9 intra-mode superblock partitions, significantly accelerating encoding speed with minimal bitrate increase, outperforming existing speed settings.

## Contribution

A novel hierarchical fully convolutional network for superblock partition prediction that speeds up VP9 intra encoding by nearly 70% with minimal bitrate impact.

## Key findings

- Achieves 69.7% average speedup in intra encoding.
- Increases BD-rate by only 1.71%.
- Outperforms fastest built-in VP9 speed level.

## Abstract

In VP9 video codec, the sizes of blocks are decided during encoding by recursively partitioning 64$\times$64 superblocks using rate-distortion optimization (RDO). This process is computationally intensive because of the combinatorial search space of possible partitions of a superblock. Here, we propose a deep learning based alternative framework to predict the intra-mode superblock partitions in the form of a four-level partition tree, using a hierarchical fully convolutional network (H-FCN). We created a large database of VP9 superblocks and the corresponding partitions to train an H-FCN model, which was subsequently integrated with the VP9 encoder to reduce the intra-mode encoding time. The experimental results establish that our approach speeds up intra-mode encoding by 69.7% on average, at the expense of a 1.71% increase in the Bjontegaard-Delta bitrate (BD-rate). While VP9 provides several built-in speed levels which are designed to provide faster encoding at the expense of decreased rate-distortion performance, we find that our model is able to outperform the fastest recommended speed level of the reference VP9 encoder for the good quality intra encoding configuration, in terms of both speedup and BD-rate.