Robust Boosting Forests with Richer Deep Feature Hierarchy

TL;DR

This paper introduces a conservative-greedy boosting forest method that enhances the robustness of deep neural networks against adversarial attacks, especially in face shape estimation tasks, by integrating feature selection and improved tree growth strategies.

Contribution

It proposes a novel conservative-greedy training approach for boosting forests applied to deep features, improving adversarial robustness in face analysis.

Findings

CGBF significantly outperforms pure deep learning methods under adversarial attacks.

The method effectively maintains face shape estimation accuracy despite perturbations.

The approach demonstrates robustness in a new 3D face model task.

Abstract

We propose a robust variant of boosting forest to the various adversarial defense methods, and apply it to enhance the robustness of the deep neural network. We retain the deep network architecture, weights, and middle layer features, then install gradient boosting forest to select the features from each layer of the deep network, and predict the target. For training each decision tree, we propose a novel conservative and greedy trade-off, with consideration for less misprediction instead of pure gain functions, therefore being suboptimal and conservative. We actively increase tree depth to remedy the accuracy with splits in more features, being more greedy in growing tree depth. We propose a new task on 3D face model, whose robustness has not been carefully studied, despite the great security and privacy concerns related to face analytics. We tried a simple attack method on a pure convolutional neural network (CNN) face shape estimator, making it degenerate to only output average face shape with invisible perturbation. Our conservative-greedy boosting forest (CGBF) on face landmark datasets showed a great improvement over original pure deep learning methods under the adversarial attacks.