The HyperBagGraph DataEdron: An Enriched Browsing Experience of Multimedia Datasets

TL;DR

This paper introduces HyperBagGraph-based visual browsing for multimedia datasets, enabling multi-faceted exploration and insights beyond linear search results, demonstrated on scientific publications from Arxiv.

Contribution

It proposes HyperBagGraphs for multi-facet visualization and navigation in multimedia datasets, enhancing exploratory search capabilities.

Findings

Facets are modeled as HyperBag-Graphs for co-occurrence networks.

Faceted visualization improves exploration of multimedia datasets.

Applied to Arxiv publications for scientific data browsing.

Abstract

Traditional verbatim browsers give back information in a linear way according to a ranking performed by a search engine that may not be optimal for the surfer. The latter may need to assess the pertinence of the information retrieved, particularly when s$\cdot$he wants to explore other facets of a multi-facetted information space. For instance, in a multimedia dataset different facets such as keywords, authors, publication category, organisations and figures can be of interest. The facet simultaneous visualisation can help to gain insights on the information retrieved and call for further searches. Facets are co-occurence networks, modeled by HyperBag-Graphs -- families of multisets -- and are in fact linked not only to the publication itself, but to any chosen reference. These references allow to navigate inside the dataset and perform visual queries. We explore here the case of scientific publications based on Arxiv searches.

Equations

$$A_{m}=\left\{x_{i}^{m_{i}}:i\in\left\llbracket n\right\rrbracket\right\}$$

$$A_m=\left\{\left\{\underset{m_1\text{times}}{\underbrace{x_1,\dots ,x_1}},\dots ,\underset{m_n\text{times}}{\underbrace{x_n,\dots ,x_n}}\right\}\right\}.$$

$${\mathcal{H}}_{\alpha /\rho ,\mathcal{S}}\stackrel{\Delta }{=}\left(\bigcup _{r\in \mathcal{S}}{A}_{\alpha ,r}^{\star },{\left(e_{\alpha ,s}\right)}_{s\in {\Sigma }_{\rho }}\right).$$

Full text

I Introduction

Revealing important information contained in Big Data calls for powerful analytical and visualising tools to reveal important information it contains. In an information space, meaningfull information can be regrouped by hierarchical classification or - non exclusive - by semantically cohesive categories that are combined to express concepts [1].

When browsing a textual database, traditional verbatim browsers give back linear information in the form of a ranked list with a short description of the references. To increase the pertinence of this information, the surfer has often to perform new search either by refining the original keywords s$\cdot$he used or by using other pertinent queries that can help her$\cdot$him to refine the retrieved information.

In fact, in most cases, the information space is multifacetted. Those facets are not independent but are linked by the physical entities contained in the search output. Choosing a type of reference common to these entities enables the construction of a network of co-occurences for each facet. Navigating to reveal information between the different facets helps. For instance, in the case of scientific publications different information of interest for the scientist can be regrouped around the publication reference: the authors, the organization(s) they belong, the countries of these organisations, the main keywords of an article or the figures inside the publication. All this metadata can give insights into the information space and as most of the articles in the database can be systematically regrouped to get meaningfull information. Choosing as reference the article itself, the facets can show different co-occurrence networks such as co-authors, co-keywords and co-categories. The interaction between the different facets is ensured by the links between them. The choice of the reference type is not unique and can be any convenient metadata type in the dataset. Co-occurences can contain repetitions or an individual weighting: multisets, and particularly natural multisets, allow them while sets don’t.

We propose in this article another way to explore an information space by using hyper-bag-graphs (hb-graphs for short), an extension of hypergraphs to families of multisets. In [2], we show that hb-graphs enhance an exchange-based diffusion over co-occurence networks, providing a fine vertex and hb-edge ranking, accounting for a hb-edge based weighting of vertices.

This paper presents a hyper-bag framework of co-occurence networks extending the visualisation part of the hypergraph framework sketched in [3] to hb-graph . This framework supports browsing of an information space and dataset visual queries. The framework is validated from a theoretical point of view and with a use case. We have implemented a 2.5D interface to visualise different facets of the Arxiv information space and perform visual queries.

Section II lists the related work and the mathematical background. Section III presents the hb-graph framework. Section IV gives results and Section V concludes and addresses future work.

II Related work and mathematical background

II-A Information space discovery

Discovering knowledge in an information space requires to gather meaningful information, either hierarchically or semantically organized. Semantic provides support to the definition of facets within an information space [1].

Navigation and visualisation of information spaces have been achieved by several authors in many different ways. In [4], a pivot is used to stroll between three facets of the information space; the approach is limited to the visualisation of a small amount of pivots at the same time. This visualisation is based on a tri-partite graph. In [5], an interactive exploration of implicit and explicit relations in faceted datasets is proposed. The space of visualisation is shared between different metadata with cross findings between metadata, partitioning the space in categories.

[6] proposes a visual analytics graph-based framework that reveals this information and facilitates knowledge discovery. It provides insights into different facets of an information space based on user-selected perspectives. The dataset is stored as a labelled graph in a graph database. Choosing a perspective as reference and a facet as dimension, paths of the labelled graph are retrieved with same dimension extremities going through reference vertices. Visualisation comes in the form of navigable node-link graphs: edges materialise common references between vertices and are seen as pairwise collaboration between two vertices.

II-B Co-occurence networks

Data mining is only one step in the knowledge discovery processing chain [7]. If numerical data allows rich statistics on the instances, non numerical data mining consists often in summarizing data as occurrences. Some other approaches exist. Often they consist in regrouping data instances through similarities using techniques such as $k$-nearest neighbours [8]: using a threshold, links are established between different occurrences. These approaches relies on similarities: the curse of dimensionality is nonetheless a limiting factor in their use [9], even if some techniques exist to limit its impact [10, 11]. A last way of finding occurences is to retrieve links through the dataset itself.

If the dataset reflects existing links the job is easier since an inherent network can be built through the data instances. A typical example is with group of friends in social networks. When the links exist, the collaborations are derived from them. Nonetheless, links are often neither direct nor tangible: in this case occurences need to be built or process from the dataset.

A dataset can be a set of physical references, stored as rows in traditional relational databases. Each physical reference has a metadata instance attached to it. Some of the types of the metadata instances are of interest for visualisation and some for processing additional information. The set of physical references and metadata instances used for visualisation constitute the types of the network, each type being seen either as a reference or a facet of the information space. This allows - as it will be explained in the next section - the retrieval of co-occurrences in one facet, based on one reference type - which can differ from the physical reference.

II-C Multisets and hb-graphs

Co-occurences can be seen as collaborations and therefore constitute with their links a network. A collaboration is a $m$-adic relationship as mentioned in [12] between occurrences, and therefore modelisation is often done with hypergraphs, i.e. family of sets over a vertex set. But hypergraphs don’t support neither hyperedge-based repetition nor hyperedge-based weighting of vertices. We introduce hb-graphs in [13, 2, 14] to extend the concept of hypergraphs to families of multisets with same universe, called the vertex set.

Multisets - also known as bags or msets - have been used for a long time in many domains, in particular in text representation [2]. A multiset $A_{m}$ on a universe $A$ is a couple $\left(A,m\right)$ where $A$ is a set and $m$ is an application from $A$ to $\mathbb{W}\subseteq\mathbb{R}$ called the multiplicity function of the multiset $A_{m}$. Elements that have a nonzero multiplicity are gathered in the support of the multiset, written $A_{m}^{\star}.$ If the range of the multiplicity function is a subset of $\mathbb{N}$, the multiset is called a natural multiset. In this case, elements of the multiset can be seen as a non-ordered list of repeated elements. The empty mset of universe $A$, written $\emptyset_{A}$, is the multiset of empty support on the universe $A$.

Several notations of msets exist. Among the common notations mentioned in [13], we note in this article a mset $A_{m}$ of universe A=\left\{x_{i}:i\in\left\llbracket n\right\rrbracket\right\} by:

[TABLE]

where $m_{i}=m\left(x_{i}\right)$111A natural multiset can also be expressed as an unordered list with repetition:

$A_{m}=\left\{\left\{\underset{m_{1}\,\text{times}}{\underbrace{x_{1},\ldots,x_{1}}},\ldots,\underset{m_{n}\,\text{times}}{\underbrace{x_{n},\ldots,x_{n}}}\right\}\right\}.$

.

Different operations are defined between two multisets $A_{m_{A}}$ and $B_{m_{B}}$ of same universe $U$. Especially, as it is used in this paper, the additive union of $A_{m_{A}}$ and $B_{m_{B}}$ is the multiset $C_{m_{C}}=A_{m_{A}}\uplus B_{m_{B}}$ of universe $U$ such that for all $x\in U$, $m_{C}\left(x\right)=m_{A}\left(x\right)+m_{B}\left(x\right).$

Considering a set $V=\left\{v_{1},...,v_{n}\right\}$ of vertices, a hb-graph $\mathcal{H}=\left(V,E\right)$ is a family of multisets E=\left(e_{i}\right)_{i\in\left\llbracket p\right\rrbracket}having same universe $V$ and with support a subset of $V$. The elements of $E$ are called the hb-edges. As a multiset, each hb-edge $e_{i}\in E$ has a multiplicity function associated to it: $m_{e_{i}}:V\rightarrow\mathbb{W}$ where $\mathbb{W}\subset\mathbb{R}^{+}$. For a general hb-graph, each hb-edge has to be seen as a weighted system of vertices, where the weights of each vertex are hb-edge dependent.

When the multiplicity range of each hb-edge is a subset of $\mathbb{N}$ the hb-graph is said natural. A hypergraph is a natural hb-graph where every vertex in any hb-edge has a binary value

• 0 or 1 - for multiplicity.

Considering the family of hb-edge support of a hb-graph $\mathcal{H}=\left(V,E\right)$, we can define its support hypergraph as the hypergraph $\underline{\mathcal{H}}=\left(V,\underline{E}\right)$, where $\underline{E}=\left\{e^{\star}:e\in E\right\}$. The support hypergraph is unique for a given hb-graph. But reconstructing the hb-graph from a support hypergraph generates an infinite number of hb-graphs, showing that the information contained in a hb-graph is denser than in a hypergraph.

We represent hb-graphs using an unnormalised extra-node representation [13]: an extra-node per hb-edge is added and the link thickness between it and each hb-edge support vertex is proportional to the vertex multiplicity in this hb-edge. The hypergraph support of the hb-graph constitues a simplified representation.

III Hb-graph framework

Multidimensional datasets are formed of data that are linked with physical entities. For instance, a publication, a person, a piece of music are possible physical entities. Some metadata of various kind and types, numerical or not, are attached to the physical entities, including their own reference.

Statistics can be easily performed on numerical types. For non numerical types, only co-occurence gathering is easily achievable with traditional charts and arrays. But choosing one of these non-numerical types as a reference to build co-occurences enhances navigation in the information space. The navigation is a simplification of the one presented in [3]. The hb-graph framework extends the facet visualisation achieved in [3] to support multisets instead of sets using interconnected hb-graphs at the level of the data instances called visualisation hb-graphs.

III-A Enhancing navigation

Traditional database structures can be seen as hypergraphs where the hyperedges reflect the table headers and the vertices the metadata instances. Normalized forms of such databases are linked to properties of the schema hypergraph [15]. In graph databases, the schema222although not required [16] represents the relationships between the vertex types. The schema hypergraph $\mathcal{H}_{\text{Sch}}=\left(V_{\text{Sch}},E_{\text{Sch}}\right)$ represents these relationships as hyperedges.

If database knowledge extraction processing is performed, such as natural language processing, the schema hypergraph becomes an extended schema hypergraph $\overline{\mathcal{H}_{\text{Sch}}}=\left(\overline{V_{\text{Sch}}},\overline{E_{\text{Sch}}}\right)$.

Some of the types in the extended schema have no interest neither for visualisation, nor for being used as reference: we consider the types of interest as a subset $U$ of $\overline{V_{\text{Sch}}}$ from which we generate the extracted extended schema hypergraph $\mathcal{H}_{X}=\left(V_{X},E_{X}\right)$ where $V_{X}=U$, $E_{X}=\left\{e\cap U:e\in\overline{E_{\text{Sch}}}\right\}.$

Only vertices of $\mathcal{H}_{X}$ that are reachable - i.e. vertices with a simple path in between them - can be further navigated and used either as reference or visualisation type. Therefore, we build the reachability hypergraph $\mathcal{H}_{R}=\left(V_{R},E_{R}\right)$ with $V_{R}=V_{X}$ as its vertex set, individual hyperedges of $\mathcal{H}_{R}$ being the connected components $E_{\text{cc}}\left(\subset V_{X}\right)$ of $\mathcal{H}_{X}$. The hyperedges of $\mathcal{H}_{R}$ are not connected as the connected components of a hypergraph constitute a partition of its vertex set. When the reachability hypergraph has only one hyperedge, the whole dataset is navigable: it is the ideal situation.

We make the assumption that in each of the hyperedge of the reachability hypergraph, it exists a metadata type or a combination of metadata types that can be chosen as the physical reference. The data instances related to this references are supposed to be unique. For instance, in a publication dataset the physical reference is the id of the publication itself.

Last hypergraph at the metadata level, the navigation hypergraph $\mathcal{H}_{N}=\left(V_{N},E_{N}\right)$ is defined by choosing a hyperedge $e_{r}\in E_{R}$ of the reachability hypergraph and a non-empty subset $R_{\text{ref}}$ of $e_{r}$ of possible reference types of interest. The choice of a subset $R$ of $R_{\text{ref}}$ allows to consider the remaining vertices of $e_{r}\backslash R$ as visualisation vertex types, that will be used to generate the facet visualisation hb-graphs and are called the visualisation types. Hence: $E_{N}=\left\{e_{r}\backslash R:R\subseteq R_{\text{ref}}\land R\neq\emptyset\right\}.$ Navigation without changing references is possible only in one hyperedge of $E_{N}$ at a time. The simplest case happens when there is only one reference of interest selected at a time in $R_{\text{ref}}$; we restrict ourselves to this case for the moment, i.e. we consider for $E_{N}$ the set $E_{N/1}=\left\{e_{r}\backslash R:R\subseteq R_{\text{ref}}\land\left|R\right|=1\right\}.$

In a publication dataset, typical metadata types are: publication id, title, abstract, authors, affiliations, addresses, author keywords, publication categories,* countries*, organisations,…333Metadata of interest for visualisation or referencing are in italic There are many different navigation hyperedge possible: for instance choosing as reference the publication ids, the navigation hyperedge is: {authors, author keywords, organisations, country, publication categories}; choosing author keywords as reference the navigation hyperedge is: {authors, organisations, country, publication category, publication ids}.

III-B Facet visualisation hb-graphs

Each physical entity $d$ in a dataset is described by a unique physical reference $r$ and a set of data instances of different types $\alpha\in\overline{V_{\text{Sch}}}$. In [3], we use sets to store co-occurrences. Nonetheless in many cases, it is worth storing additional information by joining a multiplicity - with nonnegative integer or real values

• to elements of co-occurences. For instance in a publication dataset, different authors can have the same affiliation organization; retrieving one occurence of organization per author enforces repetitions and a natural multiset. When considering keywords, their relative frequency in the document can be used as multiplicity. From [2], we know that hb-graphs allow a refined ranking of the information. Hence, the above facts motivate the usage of multisets to store co-occurrences.

We write $A_{\alpha,r}=\left\{a_{1}^{m_{\alpha}\left(a_{1}\right)},...,a_{\alpha_{r}}^{m_{\alpha}\left(a_{\alpha_{r}}\right)}\right\}$ the multiset of values of type $\alpha$ - possibly empty - that are attached to $d$, the physical entity. $d$ is entirely described by its reference and the family of multisets that corresponds to co-occurrences of the different types $\alpha$ in $\overline{V_{\text{Sch}}}$ linked to the physical reference, i.e. $\left(r,\left(A_{\alpha,r}\right)_{\alpha\in\overline{V_{\text{Sch}}}}\right).$

Performing a search on the dataset retrieves a set $\mathcal{S}$ of physical references $r$. In the single-reference-restricted navigation hypergraph, each hyperedge $e_{N}\in E_{N/1}$ describes accessible facets relatively to a chosen reference type $\rho\in V_{N}\backslash e_{N}.$ Given a type $\alpha\in e_{N}$, the associated facet shows the visualisation hb-graph $\mathcal{H}_{\alpha/\rho,\mathcal{S}}$ where the hb-edges are the co-occurrences of type $\alpha$ relatively to reference instances of type $\rho$ ($\alpha/\rho$ as short) retrieved from the different references in $\mathcal{S}.$

We then build the co-occurrences $\alpha/\rho$ by considering the set of all values of type $\rho$ attached to all the references $r\in\mathcal{S}$: $\Sigma_{\rho}=\bigcup\limits_{r\in\mathcal{S}}A_{\rho,r}^{\star}.$ Each element $s$ of $\Sigma_{\rho}$ is mapped to a set of physical references $R_{s}=\left\{r:s\in A_{\rho,r}\right\}\in\mathcal{P}\left(\mathcal{S}\right)$ in which they appear: we write $r_{\rho}$ the mapping. The multiset of values $e_{\alpha,s}$ of type $\alpha$ relatively to the reference instance $s$ is $e_{\alpha,s}=\biguplus\limits_{r\in R_{s}}A_{\alpha,r}.$

The raw visualisation hb-graph for the facet of type $\alpha/\rho$ attached to the search $\mathcal{S}$ is then defined as:

[TABLE]

Since some hb-edges can possibly point to the same sub-mset of vertices, we build a reduced visualisation weighted hb-graph from the raw visualisation hb-graph. To achieve it we define: $g_{\alpha}:s\mapsto e_{\alpha,s}$ and $\mathcal{R}$ the equivalence relation such that: $\forall s_{1}\in\Sigma_{\rho}$, $\forall s_{2}\in\Sigma_{\rho}$: $s_{1}\mathcal{R}s_{2}\Leftrightarrow g_{\alpha}\left(s_{1}\right)=g_{\alpha}\left(s_{2}\right).$

Considering a quotient class \overline{s}\in\Sigma_{\rho}\big{/}\mathcal{R}444\Sigma_{\rho}\big{/}\mathcal{R} is the quotient set of $\Sigma_{\rho}$ by $\mathcal{R}$, we write $\overline{e_{\alpha,\overline{s}}}=g_{\alpha}\left(s_{0}\right)$ where $s_{0}\in\overline{s}$.

\text{\overline{E_{\alpha}}}=\left\{\overline{e_{\alpha,\overline{s}}}:\overline{s}\in\Sigma_{\rho}\big{/}\mathcal{R}\right\} is the support set of the multiset $\left\{\left\{e_{\alpha,s}:s\in\Sigma_{\rho}\right\}\right\}$: $\overline{e_{\alpha,\overline{s}}}\in\overline{E_{\alpha}}$ is of multiplicity $w_{\alpha}\left(\overline{e_{\alpha,\overline{s}}}\right)=\left|\overline{s}\right|$ in this multiset.

It yields: \left\{\left\{e_{\alpha,s}:s\in\Sigma_{\rho}\right\}\right\}=\left\{\overline{e_{\alpha,\overline{s}}}^{w_{\alpha}\left(\overline{e_{\alpha,\overline{s}}}\right)}:\overline{s}\in\mathcal{S_{\rho}}\big{/}\mathcal{R}\right\}

Let \tilde{g_{\alpha}}:\overline{s}\in\Sigma_{\rho}\big{/}\mathcal{R}\mapsto e\in\overline{E_{\alpha}}, then $\tilde{g_{\alpha}}$ is bijective. $\tilde{g_{\alpha}}^{-1}$ allows to retrieve the class associated to a given hb-edge; hence the associated values of $\Sigma_{\rho}$ to this class - which will be important for navigation. The references associated to $e\in\overline{E_{\alpha}}$ are $\bigcup\limits_{s\in\tilde{g_{\alpha}}^{-1}(e)}r_{\rho}\left(s\right).$ The reduced visualisation weighted hb-graph for the search $\mathcal{S}$ is defined as $\mathcal{H}_{\alpha/\rho,w_{\alpha},\mathcal{S}}\overset{\Delta}{=}\left(\bigcup\limits_{r\in\mathcal{S}}A_{\alpha,r}^{\star},\overline{E_{\alpha}},w_{\alpha}\right).$

The hb-graph support hypergraphs can be used to retrieve the results given for hypergraphs in [3].

III-C Navigability through facets

As for a given search $\mathcal{S}$ and a given reference $\rho$, the sets $\Sigma_{\rho}$ and $R_{s},s\in\Sigma_{\rho}$ are fixed, the navigability can be ensured between the different facets. We consider a type $\alpha$, its visualisation hb-graph $\mathcal{H}_{\alpha/\rho,w_{\alpha}}$ and a subset $A$ of the vertex set of $\mathcal{H}_{\alpha/\rho,w_{\alpha}}$. We target another type $\alpha^{\prime}$ of co-occurrences referring to $\rho$ to be visualised. We illustrate the navigation in Figure 1.

test

We suppose that the user selects elements of $A$ as vertices of interest from which s$\cdot$he wants to switch facet. Hb-edges of $\overline{E_{\alpha}}$ which contains at least one element of $A$ are gathered in $\left.\overline{E_{\alpha}}\right|_{A}=\left\{e:e\in\overline{E_{\alpha}}\land\left(\exists x\in e:x\in A\right)\right\}.$ Using the application $\tilde{g_{\alpha}}^{-1}$ we retrieve the corresponding class of references of type $\rho$ associated to the elements of $\left.\overline{E_{\alpha}}\right|_{A}$, to build the set of references $\left.\overline{V}\right|_{A}$ of type $\rho$ involved in the building of co-occurences of type $\alpha^{\prime}.$ Each of the classes in $\left.\overline{V}\right|_{A}$ contains instances of type $\rho$ that are gathered in a set $\mathcal{V}_{\rho,A}.$ Each element of $\mathcal{V}_{\rho,A}$ is linked to a set of physical references by $r_{\rho}.$ Hence we obtain the physical reference set involving elements of $A$: $\mathcal{S}_{A}=\bigcup\limits_{s\in\nu_{\rho,A}}R_{s}.$

The raw visualisation hb-graph $\left.\mathcal{H}_{\alpha^{\prime}/\rho}\right|_{A}=\left(\bigcup\limits_{r\in\mathcal{S}_{A}}A_{\alpha^{\prime},r}^{\star},\left(e_{\alpha^{\prime},s}\right)_{s\in\mathcal{V}_{\rho,A}}\right)$ in the targeted facet is now enhanced using $\mathcal{S}_{A}$ as search set $\mathcal{S}.$ To obtain the reduced weighted version we use the same approach as above. The multiset of co-occurrences retrieved includes all occurrences that have co-occurred with the references attached to one of the elements of $A$ selected in the first facet. Of course if $A=A_{\alpha,S}$ the reduced visualisation hb-graph contains all the instances of type $\alpha^{\prime}$ attached to physical entities of the search $\mathcal{S}$.

The reference type can always be shown in one of the facet as a visualisation hb-graph, that is in fact an hypergraph where all the hb-edges are constituted of the reference itself in multiplicity the number of time the reference occurs in the hb-graph.

Ultimately, by building a multi-dimensional network organized around types, one can retrieve very valuable information from combined data sources. This process can be extended to any number of data sources as long as they share one or more types. Otherwise the reachability hypergraph is not connected and only separated navigations are possible. Figure 2 shows some examples of visualisation hypergraphs.

III-D The DataHbEdron

The DataHbEdron provides soft navigation between the different facets of the information space. It has been introduced in [3] for hypergraphs and its principle is similar with the hb-graph support.

Each facet of the information space corresponding to a visualisation type includes a visualisation hb-graph viewed in its 2D extra-node representation with a normalised thickness on hb-edges [17]. The different facets are embedded in a 2.5D representation called the DataHbEdron. The DataHbEdron can be toggled between a cube - Figure 3 - and a carousel shape to ease the navigation between facets. The reference facet is presented as a list of references corresponding to the search output.

In the DataHbEdron, the faces show different facets of the information space: the underlying visualisation hb-graphs allow as previously explained the navigability through facets. Hb-edges are selectable interactively between the different facets; as each hb-edge is linked to a subset of the references, the corresponding references can be used to highlight information in the different facets as well as in the face containing the reference visualisation hb-graph.

IV Results

We applied this framework to perform searches and visual queries on the Arxiv database. The results are visualised in the DataHbEdron allowing simultaneous visualisation of the different facets of the information space constitutes of authors, extracted keywords and subject categories. The tool developped is now part of the Collaboration Spotting family555http://collspotting.web.cern.ch/. When performing a search, the standard Arxiv API666https://arxiv.org/help/api/index is used to query the Arxiv database. The queries can be formulated either by a text entry or done interactively directly using the visualisation: queries include single words or multiple words, with AND, OR and NOT possible operators and parenthesis groupings. The querying history is stored and presented as an interactive hb-graph to allow the visual construction of complex queries including refinement of the queries already performed. Each time a new query is formulated, the corresponding metadata is retrieved by the Arxiv API.

When performing a search on Arxiv, the query is transformed into a vector of words. The most relevant documents are retrieved based on a similarity measure between the query vector and the word vectors associated to individual documents. Arxiv relies on Lucene’s built-in Vector Space Model of information retrieval and the boolean model777https://lucene.apache.org/core/2_9_4/scoring.html. The Arxiv API returns the metadata associated to the document with highest scores for the query performed. We keep only the first $n$ answers, with $n$ tunable by the end user. This metadata, filled by authors during their submission of a preprint, contains different information such as authors, Arxiv categories and abstract.

The information space contains four main facets: the first facet shows the Arxiv reference visualisation hb-graph with a contextual sentence related to the query, links to Arxiv article’s presentation and pdf. This first facet layout is similar to classical textual search engines

• Figure 4.

The second facet corresponds to co-authors of the articles using as reference the publication. The third facet depicts the co-keywords extracted from the abstracts. The fourth facet shows the Arxiv categories involved in the references.

Co-keywords are extracted from the abstracts using TextBlob, a natural language processing Python library888https://textblob.readthedocs.io/en/dev/. We extract only nouns using the tagged text, which has been lemmatized and singularized.

Nouns in the abstract of each document are scored with TF-IDF, the Term Frequency - Invert Document Frequency. Scoring each noun in each abstract of the retrieved documents generates a hb-graphs $\mathcal{H}_{Q}$ of universe the nouns contained in the abstracts. Each hb-edge contains a set of nouns extracted from a given abstract with a multiplicity function that represents the TF-IDF score of each noun. In order to limit the hb-edge size, we keep only the first $w$ words related to an abstract, where $w$ is tunable by the end-user.

The fifth facet shows the queries that have been performed during the session: the graph of those queries can be saved. The sixth facet is reserved to show additional information such as the pdf of publications.

Any node on any facet is interactive, allowing to highlight information from one facet to another by showing the hb-edges that are mapped through the references. Queries can be build using the vertices of the hb-graph, either isolated or in combination with the current search using AND, OR and NOT through keyboard shortcuts and mouse. The first query is the only one to be performed by typing it. Merging queries between different users is immediate as they correspond to hb-edges of a hb-graph. Queries are evolutive, gathered, stored and resketchable months later.

The surfer has the possibility to display additional information related to authors using dblp, to keywords using DuckDuckGo for disambiguation and Wikipedia.

V Future work and Conclusion

The framework presented in this paper supports visual query of datasets: it enables full navigability of the corresponding of the corresponding information space. It provides powerful insights into datasets using simultaneous facet visualisation of the information space constructed from the query results. The 2.5D visualisation helps to understand the links in the dataset. This framework is flexible enough to enhance user insight into many other multimedia content. Nonetheless, the evaluation of such a human computer interface, as well as the evaluation of the hb-graph visualisation remain open questions for experts of these fields.

Acknowledgments

This work is part of the PhD of Xavier OUVRARD, done at UniGe, co-supervised by Pr. Stéphane MARCHAND-MAILLET and Dr Jean-Marie LE GOFF, head of the Collaboration Spotting Project. The research is founded by a doctoral position at CERN. The authors want to thanks Tullio BASAGLIA from the CERN Library for his very precious advices and feedbacks on the interface.