Efficient and Optimal Algorithms for Tree Summarization with Weighted Terminologies

TL;DR

This paper introduces both a greedy approximation and an exact dynamic programming algorithm for summarizing hierarchical data with terminologies, optimizing diversity and visualization in ontology structures.

Contribution

It presents a novel dynamic programming approach for optimal tree summarization with theoretical guarantees and an efficient tree reduction technique to improve practical performance.

Findings

The greedy algorithm achieves a (1-1/e) approximation guarantee.

The dynamic programming algorithm guarantees optimal solutions with proven complexity.

Tree reduction significantly accelerates the algorithms in real-world datasets.

Abstract

Data summarization that presents a small subset of a dataset to users has been widely applied in numerous applications and systems. Many datasets are coded with hierarchical terminologies, e.g., the international classification of Diseases-9, Medical Subject Heading, and Gene Ontology, to name a few. In this paper, we study the problem of selecting a diverse set of k elements to summarize an input dataset with hierarchical terminologies, and visualize the summary in an ontology structure. We propose an efficient greedy algorithm to solve the problem with (1-1/e) = 62% -approximation guarantee. Although this greedy solution achieves quality-guaranteed answers approximately but it is still not optimal. To tackle the problem optimally, we further develop a dynamic programming algorithm to obtain optimal answers for graph visualization of log-data using ontology terminologies called OVDO . The complexity and correctness of OVDO are theoretically analyzed. In addition, we propose a useful optimization technique of tree reduction to remove useless nodes with zero weights and shrink the tree into a smaller one, which ensures the efficiency acceleration of OVDO in many real-world applications. Extensive experimental results on real-world datasets show the effectiveness and efficiency of our proposed approximate and exact algorithms for tree data summarization.