Amended Cross Entropy Cost: Framework For Explicit Diversity Encouragement
Ron Shoham, Haim Permuter

TL;DR
This paper introduces the Amended Cross Entropy (ACE), a new cost function that enables training multiple classifiers with explicit control over their diversity, leading to improved ensemble performance.
Contribution
It proposes a novel cost function, ACE, that explicitly encourages diversity among classifiers, enhancing ensemble learning effectiveness.
Findings
ACE outperforms vanilla ensembles in experiments.
Optimal diversity factor improves ensemble accuracy.
Method applicable to classification similarly to NCL for regression.
Abstract
Cross Entropy (CE) has an important role in machine learning and, in particular, in neural networks. It is commonly used in neural networks as the cost between the known distribution of the label and the Softmax/Sigmoid output. In this paper we present a new cost function called the Amended Cross Entropy (ACE). Its novelty lies in its affording the capability to train multiple classifiers while explicitly controlling the diversity between them. We derived the new cost by mathematical analysis and "reverse engineering" of the way we wish the gradients to behave, and produced a tailor-made, elegant and intuitive cost function to achieve the desired result. This process is similar to the way that CE cost is picked as a cost function for the Softmax/Sigmoid classifiers for obtaining linear derivatives. By choosing the optimal diversity factor we produce an ensemble which yields better…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStochastic Gradient Optimization Techniques · Reinforcement Learning in Robotics · Optimization and Search Problems
