Generalized Median Graph via Iterative Alternate Minimizations
Nicolas Boria, S'ebastien Bougleux, Benoit Ga\"uz\`ere (LITIS), Luc, Brun

TL;DR
This paper introduces an efficient iterative method to compute a generalized median graph from a set of graphs, addressing the NP-hard challenge with a block coordinate descent approach that optimizes node and edge labelings.
Contribution
It proposes a novel block coordinate descent algorithm for median graph computation that handles labeling on both nodes and edges, improving efficiency.
Findings
Demonstrates efficiency through experiments on multiple datasets
Handles labeling on nodes and edges effectively
Provides a clear optimization framework for median graph computation
Abstract
Computing a graph prototype may constitute a core element for clustering or classification tasks. However, its computation is an NP-Hard problem, even for simple classes of graphs. In this paper, we propose an efficient approach based on block coordinate descent to compute a generalized median graph from a set of graphs. This approach relies on a clear definition of the optimization process and handles labeling on both edges and nodes. This iterative process optimizes the edit operations to perform on a graph alternatively on nodes and edges. Several experiments on different datasets show the efficiency of our approach.
| Algorithms | Letter (HIGH) | Monoterpenoides | |||||||
|---|---|---|---|---|---|---|---|---|---|
| 1st phase | 2nd phase | SOD SM | t(SM) | SOD GM | t(GM) | SOD SM | t(SM) | SOD GM | t(GM) |
| Bipartite | Bipartite | 142.69 | 0.01 | 87.80 | 402.50 | 0.002 | 253.11 | ||
| Bipartite | IPFP | 142.87 | 0.013 | 87.61 | 0.003 | 398.01 | 0.002 | 128.45 | 0.179 |
| IPFP | IPFP | 135.99 | 0.057 | 87.22 | 0.003 | 202.75 | 0.162 | 104.11 | 0.136 |
| Bipartite | Bipartite | 142.04 | 0.014 | 89.47 | 283.94 | 0.027 | 186.15 | 0.01 | |
| Bipartite | IPFP | 142.19 | 0.018 | 87.66 | 0.013 | 281.14 | 0.031 | 83.11 | 0.545 |
| IPFP | IPFP | 135.99 | 0.274 | 87.23 | 0.015 | 106.10 | 1.159 | 75.08 | 0.288 |
| Letter (HIGH) Dataset | |||||||||
| TS | 1st phase | 2nd phase | pt | % SM | t(SM) | % GM | t(GM) | % TS | t(TS) |
| 10% | Bipartite | Bipartite | 0.023 | 76.42 | 0.325 | 82.82 | 0.325 | 83.01 | 5.275 |
| Bipartite | IPFP | 0.195 | 77.40 | 5.857 | 84.16 | 5.771 | 83.30 | 110.48 | |
| IPFP | IPFP | 0.447 | 78.24 | 5.951 | 84.60 | 5.801 | 82.95 | 111.84 | |
| 30% | Bipartite | Bipartite | 0.181 | 79.94 | 0.251 | 84.24 | 0.250 | 87.24 | 11.44 |
| Bipartite | IPFP | 0.878 | 81.83 | 4.323 | 86.06 | 4.234 | 86.86 | 239.14 | |
| IPFP | IPFP | 3.437 | 81.59 | 4.316 | 86.08 | 4.245 | 86.86 | 240.96 | |
| Monoterpenoides Dataset | |||||||||
| TS | 1st phase | 2nd phase | pt | % SM | t(SM) | % GM | t(GM) | % TS | t(TS) |
| 10% | Bipartite | Bipartite | 0.054 | 32 | 0.984 | 29.44 | 0.957 | 51.86 | 3.830 |
| Bipartite | IPFP | 1.586 | 53.38 | 47.96 | 57.49 | 51.03 | 60.69 | 186.85 | |
| IPFP | IPFP | 2.044 | 54.06 | 47.31 | 62.38 | 48.01 | 60.69 | 187.83 | |
| 30% | Bipartite | Bipartite | 0.373 | 36.39 | 0.747 | 34.28 | 0.732 | 67.92 | 8.571 |
| Bipartite | IPFP | 5.148 | 54.06 | 36.54 | 67.79 | 37.07 | 75.82 | 419.81 | |
| IPFP | IPFP | 15.38 | 58.37 | 36.15 | 74.12 | 36.57 | 75.94 | 419.31 | |
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGraph Theory and Algorithms · Advanced Graph Neural Networks · Advanced Clustering Algorithms Research
Generalized Median Graph via Iterative Alternate Minimizations
Nicolas Boria† and Sébastien Bougleux‡ and Benoit Gaüzère∗ and Luc Brun†
Normandie Univ, ENSICAEN, UNICAEN, CNRS, GREYC, Caen, France
Normandie Univ, UNICAEN, ENSICAEN, CNRS, GREYC, Caen, France
Normandie Univ, INSA ROUEN Normandie, LITIS, Rouen, France
Abstract
Computing a graph prototype may constitute a core element for clustering or classification tasks. However, its computation is an NP-Hard problem, even for simple classes of graphs. In this paper, we propose an efficient approach based on block coordinate descent to compute a generalized median graph from a set of graphs. This approach relies on a clear definition of the optimization process and handles labeling on both edges and nodes. This iterative process optimizes the edit operations to perform on a graph alternatively on nodes and edges. Several experiments on different datasets show the efficiency of our approach.
1 Introduction
In a wide variety of scientific domains, attributed graphs provide a powerful structure to represent, process and analyze data. However, determining fundamental tools such as a distance or an average graph is non trivial. Given a space of attributed graphs, Graph Edit Distance (GED) is a natural choice for comparing graphs [2, 16]. It measures the minimal amount of distortion needed to transform a graph into another by means of edit operations. It can be defined as a minimal-path problem which relies on a cost function acting as a metric in , and rewritten as a special quadratic assignment problem close to the graph matching problem. Computing Graph Edit Distance is NP-Hard and still cannot be solved in a reasonable time for graphs exceeding a dozen of vertices, even for simple cost functions. Therefore, several strategies have been explored to provide tight upper-bounds in polynomial time [16]. Computing a representative of a set of graphs is even more difficult. It commonly consists in finding a generalized median graph, ie. a graph that minimizes the sum of distances (SOD) to all the graphs in [10]:
[TABLE]
where denotes Graph Edit Distance. Exact methods are restricted to labeled graphs with particular cost functions or datasets containing a small total number of vertices [5]. To estimate median graphs in a reasonable computational time, several methods reduce the SOD by a local search around an initial candidate graph, by genetic search [10], greedy search based on partitioning vertices of different graphs [9], greedy adaptive search [13], or linearization and discrete optimization [12]. A different strategy is based on graph embedding [8, 7, 6, 14, 3], usually with distances between graphs as coordinates. A representative is more easily computed within this space. Then a median graph is reconstructed by going back to the original space of graphs. While these approaches are able to tackle the complexity of the previous ones, the link with the definition of a generalized median graph is not trivial and difficult to analyze. Other approaches use the relationship between common-labeling and the median graph to derive bounds on the SOD [15], or extend the concept of representative to correspondences between graphs [11].
In this paper, we propose to estimate a generalized median graph by a block coordinate descent that iterates two minimization steps from an initial candidate (Sec. 3): one for updating the SOD w.r.t. edges and attributes on nodes and on edges, and the other w.r.t. distances. The order of the resulting graph is fixed before the descent process by the order of the initial candidate. This candidate is set to a set-median, i.e. a graph of minimizing the SOD ( restricted to in Eq. 1). While the first step of the descent shares similarities with the update presented in [10], the update rules are not the same, and any algorithm can be used to estimate GED in the second step or for initialization. The first empirical results on two datasets (Sec. 4) show on the one hand that the proposed method systematically reduces the SOD associated with the initial candidate, i.e. a set-median, and on the other hand that the accuracy of the approximate GED has more impact on the descent than on the computation of a set-median. The following section introduces the expressions we use to facilitate the derivation of the proposed algorithm.
2 Graph Transformations and Graph Edit Distance
We consider simple undirected attributed graphs. An attributed graph of order can be encoded by a triplet (Fig. 1). The -tuple associates an attribute (or feature) of a space to each integer (vertices are represented by the set ). is the vertex-vertex adjacency matrix of , i.e. if there is an edge , else . associates an attribute of a space to each pair . When is not an edge, can be equal to any value, it does not affect the following expressions. Obviously, and are symmetric. Let be the space of all attributed graphs for and fixed. In this paper, each space of attributes is restricted to a finite set of positive integer labels, or to the Euclidean space.
A graph of order can be transformed into a graph of order by applying a composition of elementary transformations, a.k.a. edit operations, to . An edit operation transforms a graph into another by either removing an element (a vertex or an edge), substituting an attribute attached to an element by another attribute, or by inserting an element and its attribute (between two existing vertices for edges). Moreover, if each element of both graphs is assumed to be involved in exactly one edit operation, the number of operations is minimized, and the transformation of into is fully described by the transformation of the vertices of into the ones of . Here, this transformation, a.k.a. error-correcting matching [2, 16], is defined as a pair so that (Fig. 1). Each vertex of is either substituted by a vertex of ( and ), or removed (). Each vertex of that is not substituted to a vertex of is inserted (). The transformation of the edges of into the ones of is induced by the transformation of the vertices. The set defines the substituted edges, the set defines the removed edges, and the set defines the inserted edges. Since can be obtained from , we omit for simplicity, and we denote by all the transformations of to .
A transformation is said to be minimal if its cost is minimal, i.e. if , with the cost for transforming into using , and
[TABLE]
the costs for transforming attributed vertices and edges, respectively. if , else [math], and . Functions and measure costs to substitute vertices and edges. In this paper, the costs for removing and inserting elements are restricted to positive constants, denoted , , , . When any substitution of elements is no more expensive than removing and inserting these elements, Graph Edit Distance (GED) between and is equal to the cost of a minimal transformation [16]: . This case is considered in the sequel.
3 Estimating a Generalized Median Graph
Given a set of graphs , with of order , a generalized median graph of minimizes the sum of distances (SOD) to the graphs of [10, 5]: , with . We propose to use a block coordinate descent to estimate both and the minimal transformations .
3.1 Proposed algorithm
First, is initialized to a set-median of , i.e. . It can be computed in time [5], where is the complexity of the algorithm used for computing or estimating GED. This also provides the minimal transformations from to the graphs of . The order of is then fixed, i.e. considered as a constant in the optimization process. Then, and are alternatively updated as follows:
[TABLE]
until convergence, that is, until a stability is reached both in and . The resolution of the minimization of the sum of distances when the transformations are fixed (Eq. 7) mainly depends on the nature of and , as well as the form of the cost functions and . This is detailed later in this section, in particular it can be solved in time under some conditions. The update of the transformations (Eq. 8) consists in solving times GED problem, so in time. Since the order is fixed, and GED can usually be only estimated, the algorithm may not converge to the true generalized median graph.
We assume that an algorithm for computing GED is given, and we focus on the minimization of the sum of distances w.r.t. the graph (Eq. 7). It can be decomposed into two independent minimizations as long as the attributes and are independent for each , that we consider in this paper:
[TABLE]
with and . The minimization of each term is detailed in the two following sections. Note that some results are already presented in [10], in particular for vertices. There are obtained in a different way, allowing to take into account more easily different spaces of attributes and cost functions associated to edit operations.
3.2 Updating vertex attributes
Only the cost function depends on vertex attributes in the expression of (Eq. 2). So the attributes in Eq. 9 are updated by solving the equivalent problem with the function defined by . The objective function is a sum of positive and independent terms , so the attributes are updated by:
[TABLE]
The solution depends on and on the cost function .
When attributes are labels (), the cost for substituting a label by a label is defined as , with a constant, i.e. [math] if the labels are the same, and otherwise. Then can be rewritten as , where is the set of vertices that are substituted to by the mappings , and , , counts the number of times is substituted by a vertex having the same label (with zero cost). So the attributes (Eq. 10) are updated by:
[TABLE]
Notice that can be pre-computed in time for each label of . The labels are thus updated for all the vertices of in time at each iteration.
When is equipped with the scalar product and the -norm , the cost for substituting an attribute by an attribute is defined by . In this case, we have: . Any attribute satisfies , i.e. , or:
[TABLE]
In other words, the optimal attribute for a vertex is given by the mean attribute of the vertices substituted to (the set defined in the previous paragraph). Once more, updating all the attributes is done in time at each iteration.
3.3 Updating edges and their attributes
The edges of , and their attributes, are computed at each step of the descent (Eq. 7) by minimizing (Eq. 9). By removing the constant terms in , i.e. in (Eq. 6), it is easy to show that the minimization of can be rewritten as:
[TABLE]
with the function defined by:
[TABLE]
where is the set of edges that are substituted to by the mappings . The terms are positive and independent from each others, so Eq. 13 is equivalent to:
[TABLE]
Since can only take two values, if (no edge) then for any , and if then is minimized for any
[TABLE]
By consequence is minimized for and
[TABLE]
Solutions are finally obtained by solving Eq. 16. It depends on and .
When and , with a constant, is the classical cost for labels, then (Eq. 14) becomes
[TABLE]
where counts the number of times is substituted by an edge having the label . Then and are updated for all by:
[TABLE]
and
[TABLE]
Each edge is thus labeled with one of the most present labels among the ones substituted to . Notice that and can be computed in time. So and are computed in time.
Unlabeled graphs can be considered as labeled with a unique label, e.g. . In this case and , so from Eq. 19 can be computed in time by:
[TABLE]
Remark.
Similar results can be derived for directed graphs, other spaces of attributes and other cost functions, for both vertices and edges. Due to limited space, it is restricted here to the cases considered in the experiments.
4 Experimental results
In order to evaluate the validity of our method, the algorithm was implemented in C++ and tested on the datasets Letter (HIGH) [16] and Monoterpenoides 111GREYC Chemistry dataset: https://brunl01.users.greyc.fr/CHEMISTRY/, a chemical dataset, on a computer using an intel(R) i7-8700 CPU with 12 parallel threads. The Monoterpenoides dataset has 286 graphs unevenly divided in 8 classes of at least 10 graphs. Both nodes and egdes are labeled, and the average order is 11.003. Edit costs were set to and .
Remember that, in a first phase, the proposed algorithm (Sec. 3.1) identifies a set-median by computing all pairwise distances in the dataset. These distances are computed through two heuristics: bipartite [16], and IPFP [1]. In a second phase, the algorithm iterates the update of a triplet according to Eq. 9 (i.e. for vertices either Eq. 11 for Monoterpenoides or Eq. 12 for Letter, and for edges, Eq.18–19 for Monoterpenoides or Eq. 20 for Letter), and the update of the transformations using either bipartite or IPFP. We denote by Bipartite and IPFP the multistart counterparts of Bipartite, and IPFP [4], where the number of randomly generated initializations was set to 40.
Table 1 sums up our results regarding SOD. In Letter and Monoterpenoides, respectively 50 and 10 graphs were picked randomly in each class, and each experiment was repeated 50 times. The results presented in Table 1 represent the averages over all classes and all experiments. The four columns SOD SM, t(SM), SOD GM and t(GM) list the SODs and computation times in seconds for the set-median (SM), and the generalized median (GM). Note that t(GM) refers to the computation time of the second phase only. Using state of the art GED heuristics and making the most of the computed transformations to efficiently perform the descent (conversely to many other approaches which use GED only to evaluate candidate medians, without using the detailed transformations), our algorithm produces median graphs with SODs much lower than the set-medians’ with a very low running time. It is noteworthy that the time dedicated to identify the set-median (first phase) is systematically higher than the one dedicated to the generalized median (second phase). Indeed, distances must be computed in the first phase, while distances are computed in the second phase, where denotes the number of iterations before convergence. In practice, we verified that, in most cases, on the letter dataset, and on Monoterpenoides. Interestingly enough, in the hybrid versions of the algorithm (using Bipartite in the first phase and IPFP in the second phase), the alternate descent still produces median graph with reasonably low SOD while starting from a set-median of lesser quality (i.e. with higher SODs).
Finally, note that the range between best and worst computed SODs is particularily low on the Letter dataset, while it is rather high on the Monoterpenoides dataset. This seems to indicate that approximate computed distances are close to the optimum in Letter, and far from it in Monoterpenoides.
Picking random trainsets in each class 10% and 30% the size of the class, set-medians and generalized medians were computed for each class, and the classification accuracy of a 1-nn algorithm [5] was evaluated using as training examples: (SM) only the set-median, (GM) only the generalized medians and finally (TS) the whole trainset. Each experiment was repeated 50 times, and Table 2 presents our results, giving the average preprocessing time pt (i.e. the time spent in computation of set-medians and generalized medians), as well as classification precisions (denoted by %) and times for all three training examples considered. Note that the GED heuristic used in the second phase of the algorithms were also used in computing distances by the classifier.
Let us note that our approach competes with a 1-nn classification over the whole trainset, especially when all the distances are computed with a more precise heuristic, such as IPFP. Whenever a precise heuristic is used to compute it, the generalized median appears as a better representative of the class than the set-median. Obviously, classification times are much faster using only the median graphs as training example.
In few cases, the classification accuracy enabled by set-medians is higher than that enabled by generalized medians. This only happens in cases where computed distances and edit-paths are looser approximations, i.e. this always happens on the Monoterpenoides dataset with the Bipartite heuristic used in the initialization phase.
5 Conclusion
We proposed an innovative general method to compute the generalized median graph based on an alternate gradient descent. We showed its efficiency through experiments on two datasets using different edit-cost structures. Computed graphs have much lower SODs than set-medians, and can efficiently be used as representatives in a clustering framework. Quality of computed graph median increases when using accurate rather than fast GED approximation algorithms as sub-routines, especially in the alternate descent phase, while the initialization phase may use different GED heuristics to reach different time/quality compromises. Future developments regarding this promising method include the extension to new edit-cost structures, as well as the possibility to modify the order of the median graph during the optimization process.
Acknowledgments.
This work is supported by Région Normandie through RIN AGAC project.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] Bougleux, S., Gaüzère, B., Brun, L.: Graph edit distance as a quadratic program. In: International Conference on Pattern Recognition. pp. 1701–1706 (2016). https://doi.org/10.1109/ICPR.2016.7899881
- 2[2] Bunke, H., Allermann, G.: Inexact graph matching for structural pattern recognition. Pattern Recognition Letters 1 (4), 245–253 (1983). https://doi.org/10.1016/0167-8655(83)90033-8
- 3[3] Chaieb, R., Kalti, K., Luqman, M.M., Coustaty, M., Ogier, J.M., Amara, N.E.B.: Fuzzy generalized median graphs computation: Application to content-based document retrieval. Pattern Recognition 72 , 266–284 (2017). https://doi.org/10.1016/j.patcog.2017.07.030
- 4[4] Daller, É., Bougleux, S., Gaüzère, B., Brun, L.: Approximate graph edit distance by several local searches in parallel. In: International Conference on Pattern Recognition Applications and Methods. pp. 149–158 (2018). https://doi.org/10.5220/0006599901490158
- 5[5] Ferrer, M.: Theory and Algorithms on the Median Graph. Application to Graph-based Classification and Clustering. Ph.D. thesis, Universitat Autònoma de Barcelona (2008), http://hdl.handle.net/10803/5788
- 6[6] Ferrer, M., Bardají, I., Valveny, E., Karatzas, D., Bunke, H.: Median graph computation by means of graph embedding into vector spaces. In: Graph Embedding for Pattern Analysis, pp. 45–71. Springer New York (2013). https://doi.org/10.1007/978-1-4614-4457-2_3
- 7[7] Ferrer, M., Karatzas, D., Valveny, E., Bardaji, I., Bunke, H.: A generic framework for median graph computation based on a recursive embedding approach. Computer Vision and Image Understanding 115 (7), 919–928 (2011). https://doi.org/10.1016/j.cviu.2010.12.010
- 8[8] Ferrer, M., Valveny, E., Serratosa, F., Riesen, K., Bunke, H.: Generalized median graph computation by means of graph embedding in vector spaces. Pattern Recognition 43 (4), 1642–1655 (2010). https://doi.org/10.1016/j.patcog.2009.10.013
