# Bringing Structure into Summaries: Crowdsourcing a Benchmark Corpus of Concept Maps

## TL;DR

This paper introduces a new benchmark corpus of concept maps created via crowdsourcing to evaluate multi-document summarization in the form of structured concept maps, addressing a key gap in evaluation datasets.

## Contribution

It presents a novel crowdsourcing method to create a large, structured corpus of concept maps for summarizing web document collections on educational topics.

## Key findings

- Created a new benchmark corpus of concept maps
- Developed a crowdsourcing approach for corpus creation
- Provided baseline system and evaluation protocol

## Abstract

Concept maps can be used to concisely represent important information and bring structure into large document collections. Therefore, we study a variant of multi-document summarization that produces summaries in the form of concept maps. However, suitable evaluation datasets for this task are currently missing. To close this gap, we present a newly created corpus of concept maps that summarize heterogeneous collections of web documents on educational topics. It was created using a novel crowdsourcing approach that allows us to efficiently determine important elements in large document collections. We release the corpus along with a baseline system and proposed evaluation protocol to enable further research on this variant of summarization.