The Hybrid Bootstrap: A Drop-in Replacement for Dropout
Robert Kosar, David W. Scott

TL;DR
The paper introduces the hybrid bootstrap, a regularization method that resamples features from other training points, showing it outperforms dropout across various models and tasks, with simplified hyperparameter tuning.
Contribution
It presents the hybrid bootstrap as a novel regularization technique, along with sampling methods for hyperparameter selection and application to convolutional neural networks and non-image data.
Findings
Hybrid bootstrap outperforms dropout in predictive accuracy.
Sampling techniques simplify hyperparameter tuning.
Effective on tree-based models for non-image tasks.
Abstract
Regularization is an important component of predictive model building. The hybrid bootstrap is a regularization technique that functions similarly to dropout except that features are resampled from other training points rather than replaced with zeros. We show that the hybrid bootstrap offers superior performance to dropout. We also present a sampling based technique to simplify hyperparameter choice. Next, we provide an alternative sampling technique for convolutional neural networks. Finally, we demonstrate the efficacy of the hybrid bootstrap on non-image tasks using tree-based models.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMachine Learning and Data Classification · Domain Adaptation and Few-Shot Learning · Advanced Neural Network Applications
