A Survey on Dropout Methods and Experimental Verification in Recommendation

TL;DR

This paper provides a comprehensive survey and empirical comparison of over seventy dropout methods in recommendation systems, categorizing them, analyzing their applications, and identifying future research directions.

Contribution

It systematically classifies dropout methods into three categories, reviews their applications, and empirically evaluates their effectiveness in recommendation scenarios.

Findings

Dropout methods vary significantly in effectiveness across recommendation tasks.

Certain dropout techniques outperform others in specific application scenarios.

The survey highlights open problems and future research directions in dropout methods.

Abstract

Overfitting is a common problem in machine learning, which means the model too closely fits the training data while performing poorly in the test data. Among various methods of coping with overfitting, dropout is one of the representative ways. From randomly dropping neurons to dropping neural structures, dropout has achieved great success in improving model performances. Although various dropout methods have been designed and widely applied in past years, their effectiveness, application scenarios, and contributions have not been comprehensively summarized and empirically compared by far. It is the right time to make a comprehensive survey. In this paper, we systematically review previous dropout methods and classify them into three major categories according to the stage where dropout operation is performed. Specifically, more than seventy dropout methods published in top AI conferences or journals (e.g., TKDE, KDD, TheWebConf, SIGIR) are involved. The designed taxonomy is easy to understand and capable of including new dropout methods. Then, we further discuss their application scenarios, connections, and contributions. To verify the effectiveness of distinct dropout methods, extensive experiments are conducted on recommendation scenarios with abundant heterogeneous information. Finally, we propose some open problems and potential research directions about dropout that worth to be further explored.