Towards Disentangling Information Paths with Coded ResNeXt
Apostolos Avranas, Marios Kountouris

TL;DR
This paper introduces a neural network architecture that uses coding theory to create class-specific information paths, enabling interpretability, early predictions, and lightweight classifiers without additional training.
Contribution
It proposes a novel coding theory-based architecture for neural networks that enhances transparency, allows early inference, and produces lightweight class-specific models.
Findings
Paths are designed in advance using coding theory.
Enables early predictions at intermediate layers.
Achieves at least 60% parameter reduction with improved accuracy.
Abstract
The conventional, widely used treatment of deep learning models as black boxes provides limited or no insights into the mechanisms that guide neural network decisions. Significant research effort has been dedicated to building interpretable models to address this issue. Most efforts either focus on the high-level features associated with the last layers, or attempt to interpret the output of a single layer. In this paper, we take a novel approach to enhance the transparency of the function of the whole network. We propose a neural network architecture for classification, in which the information that is relevant to each class flows through specific paths. These paths are designed in advance before training leveraging coding theory and without depending on the semantic similarities between classes. A key property is that each path can be used as an autonomous single-purpose model. This…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsCOVID-19 diagnosis using AI · Generative Adversarial Networks and Image Synthesis · Explainable Artificial Intelligence (XAI)
MethodsAverage Pooling · *Communicated@Fast*How Do I Communicate to Expedia? · Batch Normalization · Kaiming Initialization · 1x1 Convolution · Global Average Pooling · Residual Connection · ResNeXt Block · Grouped Convolution · Convolution
