Position: Insights from Survey Methodology can Improve Training Data

TL;DR

This paper highlights how survey methodology insights can enhance training data quality for AI models, emphasizing better data collection practices to improve fairness, trustworthiness, and alignment.

Contribution

It introduces survey methodology principles to AI/ML researchers, proposing methods to improve data collection quality and reduce biases in training data.

Findings

Survey techniques can improve data accuracy for AI training.

Applying bias mitigation strategies enhances model fairness.

Collaborative research can address data collection biases.

Abstract

Whether future AI models are fair, trustworthy, and aligned with the public's interests rests in part on our ability to collect accurate data about what we want the models to do. However, collecting high-quality data is difficult, and few AI/ML researchers are trained in data collection methods. Recent research in data-centric AI has show that higher quality training data leads to better performing models, making this the right moment to introduce AI/ML researchers to the field of survey methodology, the science of data collection. We summarize insights from the survey methodology literature and discuss how they can improve the quality of training and feedback data. We also suggest collaborative research ideas into how biases in data collection can be mitigated, making models more accurate and human-centric.