Horizontal and Vertical Ensemble with Deep Representation for Classification

TL;DR

This paper introduces ensemble methods combining horizontal and vertical voting with deep representations to enhance classification accuracy, demonstrating competitive results in a machine learning challenge.

Contribution

The paper proposes novel ensemble techniques that leverage deep learned features for improved classification performance, validated through competitive leaderboard results.

Findings

Achieved top leaderboard positions using the proposed ensemble methods.

Demonstrated that ensemble strategies improve deep neural network classification.

Validated effectiveness on the ICML 2013 Black Box Challenge.

Abstract

Representation learning, especially which by using deep learning, has been widely applied in classification. However, how to use limited size of labeled data to achieve good classification performance with deep neural network, and how can the learned features further improve classification remain indefinite. In this paper, we propose Horizontal Voting Vertical Voting and Horizontal Stacked Ensemble methods to improve the classification performance of deep neural networks. In the ICML 2013 Black Box Challenge, via using these methods independently, Bing Xu achieved 3rd in public leaderboard, and 7th in private leaderboard; Jingjing Xie achieved 4th in public leaderboard, and 5th in private leaderboard.