Empirical learning aided by weak domain knowledge in the form of feature importance

TL;DR

This paper introduces a simple neural network modification called IANN that leverages weak domain knowledge in the form of feature importance, significantly improving performance in molecular biology tasks.

Contribution

The paper presents IANN, a neural network algorithm that incorporates feature relative importance, demonstrating substantial performance gains with minimal effort.

Findings

IANN outperforms standard backpropagation and SVMs in molecular biology tasks.

IANN's performance is comparable to KBANN, which uses stronger domain knowledge.

Using feature importance as weak knowledge enhances empirical learning algorithms effectively.

Abstract

Standard hybrid learners that use domain knowledge require stronger knowledge that is hard and expensive to acquire. However, weaker domain knowledge can benefit from prior knowledge while being cost effective. Weak knowledge in the form of feature relative importance (FRI) is presented and explained. Feature relative importance is a real valued approximation of a feature's importance provided by experts. Advantage of using this knowledge is demonstrated by IANN, a modified multilayer neural network algorithm. IANN is a very simple modification of standard neural network algorithm but attains significant performance gains. Experimental results in the field of molecular biology show higher performance over other empirical learning algorithms including standard backpropagation and support vector machines. IANN performance is even comparable to a theory refinement system KBANN that uses stronger domain knowledge. This shows Feature relative importance can improve performance of existing empirical learning algorithms significantly with minimal effort.