Towards Integrating Personal Knowledge into Test-Time Predictions

TL;DR

This paper introduces the problem of human feature integration, enabling ML models to incorporate personal knowledge from users during test-time to improve prediction relevance and accuracy.

Contribution

It formally defines human feature integration, compares it to existing methods, and provides a proof-of-concept solution in a semi-realistic setting.

Findings

Defined the problem of human feature integration.

Compared new approach to existing methods.

Demonstrated proof-of-concept solution.

Abstract

Machine learning (ML) models can make decisions based on large amounts of data, but they can be missing personal knowledge available to human users about whom predictions are made. For example, a model trained to predict psychiatric outcomes may know nothing about a patient's social support system, and social support may look different for different patients. In this work, we introduce the problem of human feature integration, which provides a way to incorporate important personal-knowledge from users without domain expertise into ML predictions. We characterize this problem through illustrative user stories and comparisons to existing approaches; we formally describe this problem in a way that paves the ground for future technical solutions; and we provide a proof-of-concept study of a simple version of a solution to this problem in a semi-realistic setting.