Analysis of Branch Specialization and its Application in Image Decomposition

TL;DR

This paper provides a methodological analysis of Branch Specialization in neural networks, explaining how gradient descent influences specialization and demonstrating its natural emergence in image decomposition tasks.

Contribution

It offers a new understanding of how branch specialization occurs and its role in decomposing images into meaningful components.

Findings

Gradient descent influences branch specialization.

Branched networks naturally decompose images into meaningful channels.

Specialization aligns with sub-task decomposition in image processing.

Abstract

Branched neural networks have been used extensively for a variety of tasks. Branches are sub-parts of the model that perform independent processing followed by aggregation. It is known that this setting induces a phenomenon called Branch Specialization, where different branches become experts in different sub-tasks. Such observations were qualitative by nature. In this work, we present a methodological analysis of Branch Specialization. We explain the role of gradient descent in this phenomenon. We show that branched generative networks naturally decompose animal images to meaningful channels of fur, whiskers and spots and face images to channels such as different illumination components and face parts.