
TL;DR
SpicyMKL is a new optimization algorithm for Multiple Kernel Learning that efficiently handles general convex loss functions and regularizations, converges super-linearly, and scales well with many kernels.
Contribution
It introduces SpicyMKL, an iterative smooth minimization method for MKL that avoids solving complex subproblems and extends to general regularizations like elastic-net.
Findings
Faster than existing methods with over 1000 kernels.
Converges super-linearly, ensuring rapid optimization.
Applicable to various regularizations beyond sparsity.
Abstract
We propose a new optimization algorithm for Multiple Kernel Learning (MKL) called SpicyMKL, which is applicable to general convex loss functions and general types of regularization. The proposed SpicyMKL iteratively solves smooth minimization problems. Thus, there is no need of solving SVM, LP, or QP internally. SpicyMKL can be viewed as a proximal minimization method and converges super-linearly. The cost of inner minimization is roughly proportional to the number of active kernels. Therefore, when we aim for a sparse kernel combination, our algorithm scales well against increasing number of kernels. Moreover, we give a general block-norm formulation of MKL that includes non-sparse regularizations, such as elastic-net and \ellp -norm regularizations. Extending SpicyMKL, we propose an efficient optimization method for the general regularization framework. Experimental results show that…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSparse and Compressive Sensing Techniques · Face and Expression Recognition · Machine Learning and ELM
