# CS563-QA: A Collection for Evaluating Question Answering Systems

## TL;DR

This paper introduces a small, educational collection of free-text questions designed to evaluate question answering systems across varying difficulty levels, aiding in system assessment and development.

## Contribution

It presents a new, curated collection of QA evaluation cases with increasing difficulty, facilitating rapid and educational assessment of QA systems.

## Key findings

- Collection covers diverse difficulty levels
- Enables quick evaluation of QA systems
- Supports educational and developmental purposes

## Abstract

Question Answering (QA) is a challenging topic since it requires tackling the various difficulties of natural language understanding. Since evaluation is important not only for identifying the strong and weak points of the various techniques for QA, but also for facilitating the inception of new methods and techniques, in this paper we present a collection for evaluating QA methods over free text that we have created. Although it is a small collection, it contains cases of increasing difficulty, therefore it has an educational value and it can be used for rapid evaluation of QA systems.