Constructing a Consensus Phylogeny from a Leaf-Removal Distance

TL;DR

This paper introduces a new NP-hard problem of constructing a consensus phylogeny from discordant trees, providing approximation and parameterized algorithms to address discrepancies in evolutionary scenarios.

Contribution

It defines a novel leaf-removal distance problem for consensus phylogeny, proves its NP-hardness, and offers approximation and parameterized algorithms for practical solutions.

Findings

The problem is NP-hard.

A 2-approximation algorithm is provided.

A parameterized algorithm is developed for the max leaf removal variant.

Abstract

Understanding the evolution of a set of genes or species is a fundamental problem in evolutionary biology. The problem we study here takes as input a set of trees describing {possibly discordant} evolutionary scenarios for a given set of genes or species, and aims at finding a single tree that minimizes the leaf-removal distance to the input trees. This problem is a specific instance of the general consensus/supertree problem, widely used to combine or summarize discordant evolutionary trees. The problem we introduce is specifically tailored to address the case of discrepancies between the input trees due to the misplacement of individual taxa. Most supertree or consensus tree problems are computationally intractable, and we show that the problem we introduce is also NP-hard. We provide tractability results in form of a 2-approximation algorithm. We also introduce a variant that minimizes the maximum number $d$ of leaves that are removed from any input tree, and provide a parameterized algorithm for this problem with parameter $d$.

Figures

Equations

$$d_{LR}(T_1,T_2)=\min \{|X|:X\subseteq \mathcal{X}\text{mbox}and{T}_1-X=T_2-X\}$$

$$\sum _{T_i\in \mathcal{T}}{d}_{LR}(T^{\prime },T_i)\le \sum _{T_i\in \mathcal{T}}\left(d_{LR}(T^{\prime },T^{\ast })+d_{LR}(T^{\ast },T_i)\right)\le 2\sum _{T_i\in \mathcal{T}}{d}_{LR}(T^{\ast },T_i)$$

$$\mathbb{E}\left[\sum _{1\le i<j\le t}{d}_{LR}(T_i,T_j)\right]=\sum _{1\le i<j\le t}\mathbb{E}[d_{LR}(T_i,T_j)]\ge \left(\genfrac{}{}{0ex}{}{t}{2}\right)(n-c\sqrt{n})$$

$$\sum _{1\le i<j\le t}{d}_{LR}(T_i,T_j)$$

$$=(t-1)\sum _{i=1}^t{d}_{LR}(T_i,T^{\prime })$$

$$\le (t-1)2S$$

$$\left(\genfrac{}{}{0ex}{}{t}{2}\right)(n-c\sqrt{n})\le \mathbb{E}\left[\sum _{1\le i<j\le t}{d}_{LR}(T_i,T_j)\right]\le \mathbb{E}[(t-1)2S]=2(t-1)\mathbb{E}[S]$$

Full text

1 Introduction

In the present paper, we consider a very generic computational biology problem: given a collection of trees representing, possibly discordant, evolutionary scenarios for a set of biological entities (genes or species – also called taxa in the following), we want to compute a single tree that agrees as much as possible with the input trees. Several questions in computational biology can be phrased in this generic framework. For example, for a given set of homologous gene sequences that have been aligned, one can sample evolutionary trees for this gene family according to a well defined posterior distribution and then ask how this collection of trees can be combined into a single gene tree, a problem known as tree amalgamation [13]. In phylogenomics, one aims at inferring a species tree from a collection of input trees obtained from whole-genome sequence data. A first approach considers gene families and proceeds by computing individual gene trees from a large set of gene families, and then combining this collection of gene trees into a unique species tree for the given set of taxa; this requires handling the discordant signal observed in the gene trees due to evolutionary processes such as gene duplication and loss [10], lateral gene transfer [14], or incomplete lineage sorting [12]. Another approach concatenates the sequence data into a single large multiple sequence alignment, that is then partitioned into overlapping subsets of taxa for which partial evolutionary trees are computed, and a unique species tree is then inferred by combining the resulting collection of partial trees [11].

For example, the Maximum Agreement Subtree (MAST) problem considers a collection of input trees111All trees we consider here are uniquely leaf-labeled, rooted (i.e. are out-trees) and binary; see next section for formal definitions., all having the same leaf labels and looks for a tree of maximum size (number of leaves), which agrees with each of the input trees. This problem is tractable for trees with bounded degree but NP-hard generally [1]. The MAST problem is a consensus problem, because the input trees share the same leaf labels set, and the output tree is called a consensus tree. In the supertree framework, the input trees might not all have identical label sets, but the output is a tree on the whole label set, called a supertree. For example, in the Robinson-Foulds (RF) supertree problem, the goal is to find a supertree that minimizes the sum of the RF-distances to the individual input trees [15]. One way to compute consensus trees and supertrees that is closely related to our work is to modify the collection of input trees minimally in such a way that the resulting modified trees all agree. For example, in the MAST problem, modifications of the input trees consist in removing a minimum number of taxa from the whole label set, while in the Agreement Supertree by Edge Contraction (AST-EC) problem, one is asked to contract a minimum number of edges of the input trees such that the resulting (possibly non-binary) trees all agree with at least one supertree [8]; in the case where the input trees are all triplets (rooted trees on three leaves), this supertree problem is known as the Minimum Rooted Triplets Inconsistency problem [4]. The SPR Supertree problem considers a similar problem where the input trees can be modified with the Subtree-Prune-and-Regraft (SPR) operator [16].

In the present work, we introduce a new consensus problem, called LR-Consensus. Given a collection of input trees having the same leaf labels set, we want to remove a minimum number of leaves – an operation called a Leaf-Removal (LR) – from the input trees such that the resulting pruned trees all agree. Alternatively, this can be stated as finding a consensus tree that minimizes the cumulated leaf-removal distance to the collection of input trees. This problem also applies to tree amalgamation and to species tree inference from one-to-one orthologous gene families, where the LR operation aims at correcting the misplacement of a single taxon in an input tree.

In the next section, we formally define the problems we consider, and how they relate to other supertree problems. Next we show that the LR-Consensus problem is NP-hard and that in some instances, a large number of leaves need to be removed to lead to a consensus tree. We then provide a 2-approximation algorithm222In a previous version of this work, we claimed a fixed-parameter tractable result for LR-Consensus, which turned out to be inaccurate. See Section 4 for details.. We then introduce a variant of the LR-Consensus problem, where we ask if a consensus tree can be obtained by removing at most $d$ leaves from each input tree, and describe a fixed-parameter tractable (FPT) algorithm with parameter $d$.

2 Preliminary notions and problems statement

Trees.

All trees in the rest of the document are assumed to be rooted and binary. If $T$ is a tree, we denote its root by $r(T)$ and its leaf set by $\mathcal{L}(T)$. Each leaf is labeled by a distinct element from a label set $\mathcal{X}$, and we denote by $\mathcal{X}(T)$ the set of labels of the leaves of $T$. We may sometimes use $\mathcal{L}(T)$ and $\mathcal{X}(T)$ interchangeably. For some $X\subseteq\mathcal{X}$, we denote by $lca_{T}(X)$ the least common ancestor of $X$ in $T$. The subtree rooted at a node $u\in V(T)$ is denoted $T_{u}$ and we may write $\mathcal{L}_{T}(u)$ for $\mathcal{L}(T_{u})$. If $T_{1}$ and $T_{2}$ are two trees and $e$ is an edge of $T_{1}$, grafting $T_{2}$ on $e$ consists in subdividing $e$ and letting the resulting degree $2$ node become the parent of $r(T_{2})$. Grafting $T_{2}$ above $T_{1}$ consists in creating a new node $r$, then letting $r$ become the parent of $r(T_{1})$ and $r(T_{2})$. Grafting $T_{2}$ on $T_{1}$ means grafting $T_{2}$ either on an edge of $T_{1}$ or above $T_{1}$.

The leaf removal operation.

For a subset $L\subseteq\mathcal{X}$, we denote by $T-L$ the tree obtained from $T$ by removing every leaf labeled by $L$, contracting the resulting non-root vertices of degree two, and repeatedly deleting the resulting root vertex while it has degree one. The restriction $T|_{L}$ of $T$ to $L$ is the tree $T-(\mathcal{X}\setminus L)$, i.e. the tree obtained by removing every leaf not in $L$. A triplet is a rooted tree on $3$ leaves. We denote a triplet $R$ with leaf set $\{a,b,c\}$ by $ab|c$ if $c$ is the leaf that is a direct child of the root (the parent of $a$ and $b$ being its other child). We say $R=ab|c$ is a triplet of a tree $T$ if $T|_{\{a,b,c\}}=R$. We denote $tr(T)=\{ab|c:ab|c$ is a triplet of $T\}$.

We define a distance function $d_{LR}$ between two trees $T_{1}$ and $T_{2}$ on the same label set $\mathcal{X}$ consisting in the minimum number of labels to remove from $\mathcal{X}$ so that the two trees are equal. That is,

[TABLE]

Note that $d_{LR}$ is closely related to the Maximum Agreement Subtree (MAST) between two trees on the same label set $\mathcal{X}$, which consists in a subset $X^{\prime}\subseteq\mathcal{X}$ of maximum size such that $T_{1}|_{X^{\prime}}=T_{2}|_{X^{\prime}}$: $d_{LR}(T_{1},T_{2})=|\mathcal{X}|-|X^{\prime}|$. The MAST of two binary trees on the same label set can be computed in time $O(n\log n)$, where $n=|\mathcal{X}|$ [6], and so $d_{LR}$ can be found within the same time complexity.

Problem statements.

In this paper, we are interested in finding a tree $T$ on $\mathcal{X}$ minimizing the sum of $d_{LR}$ distances to a given set of input trees.

LR-Consensus

Given: a set of trees $\mathcal{T}=\{T_{1},\ldots,T_{t}\}$ with $\mathcal{X}(T_{1})=\ldots=\mathcal{X}(T_{t})=\mathcal{X}$.

Find: a tree $T$ on label set $\mathcal{X}$ that minimizes $\sum_{T_{i}\in\mathcal{T}}d_{LR}(T,T_{i})$.

We can reformulate the LR-Consensus problem as the problem of removing a minimum number of leaves from the input trees so that they are compatible. Although the equivalence between both formulations is obvious, the later formulation will often be more convenient. We need to introduce more definitions in order to establish this equivalence.

A set of trees $\mathcal{T}=\{T_{1},\ldots,T_{t}\}$ is called compatible if there is a tree $T$ such that $\mathcal{X}(T)=\bigcup_{T_{i}\in\mathcal{T}}\mathcal{X}(T_{i})$ and $T|_{\mathcal{X}(T_{i})}=T_{i}$ for every $i\in[t]$. In this case, we say that $T$ displays $\mathcal{T}$. A list $\mathcal{C}=(\mathcal{X}_{1},\ldots,\mathcal{X}_{t})$ of subsets of $\mathcal{X}$ is a leaf-disagreement for $\mathcal{T}$ if $\{T_{1}-\mathcal{X}_{1},\ldots,T_{t}-\mathcal{X}_{t}\}$ is compatible. The size of $\mathcal{C}$ is $\sum_{i\in[t]}|\mathcal{X}_{i}|$. We denote by $AST_{LR}(\mathcal{T})$ the minimum size of a leaf-disagreement for $\mathcal{T}$, and may sometimes write $AST_{LR}(T_{1},\ldots,T_{t})$ instead of $AST_{LR}(\mathcal{T})$. A subset $\mathcal{X}^{\prime}\subseteq\mathcal{X}$ of labels is a label-disagreement for $\mathcal{T}$ if $\{T_{1}-\mathcal{X}^{\prime},\ldots,T_{t}-\mathcal{X}^{\prime}\}$ is compatible. Note that, if $\mathcal{T}=\{T_{1},T_{2}\}$, then the minimum size of a label-disagreement for $\mathcal{T}$ is $d_{LR}(T_{1},T_{2})$. We may now define the AST-LR problem (see Figure 1 for an example).

Agreement Subtrees by Leaf-Removals (AST-LR)

Given: a set of trees $\mathcal{T}=\{T_{1},\ldots,T_{t}\}$ with $\mathcal{X}(T_{1})=\ldots=\mathcal{X}(T_{t})=\mathcal{X}$.

Find: a leaf-disagreement $\mathcal{C}$ for $\mathcal{T}$ of minimum size.

Lemma 1

Let $\mathcal{T}=\{T_{1},\ldots,T_{t}\}$ be a set of trees on the same label set $\mathcal{X}$, with $n=|\mathcal{X}|$. Given a supertree $T$ such that $v:=\sum_{T_{i}\in\mathcal{T}}d_{LR}(T,T_{i})$, one can compute in time $O(tn\log(n))$ a leaf-disagreement $\mathcal{C}$ of size at most $v$. Conversely, given a leaf-disagreement $\mathcal{C}$ for $\mathcal{T}$ of size $v$, one can compute in time $O(tn\log^{2}(tn))$ a supertree $T$ such that $\sum_{T_{i}\in\mathcal{T}}d_{LR}(T,T_{i})\leq v$.

From Lemma 1333All missing proofs are provided in Appendix. both problems share the same optimality value, the NP-hardness of one implies the hardness of the other and approximating one problem within a factor $c$ implies that the other problem can be approximated within a factor $c$. We conclude this subsection with the introduction of a parameterized variant of the AST-LR problem.

AST-LR-d

Input: a set of trees $\mathcal{T}=\{T_{1},\ldots,T_{t}\}$ with $\mathcal{L}(T_{1})=\ldots=\mathcal{L}(T_{t})=\mathcal{X}$, and an integer $d$.

Question: Are there $\mathcal{X}_{1},\ldots,\mathcal{X}_{t}\subseteq\mathcal{X}$ such that $|\mathcal{X}_{i}|\leq d$ for each $i\in[t]$, and $\{T_{1}-\mathcal{X}_{1},\ldots,T_{t}-\mathcal{X}_{t}\}$ is compatible?

We call a tree $T^{*}$ a solution to the AST-LR-d instance if $d_{LR}(T_{i},T^{*})\leq d$ for each $i\in[t]$.

Relation to other supertree/consensus tree problems.

The most widely studied supertree problem based on modifying the input trees is the SPR Supertree problem, where arbitrarily large subtrees can be moved in the input trees to make them all agree (see [16] and references there). The interest of this problem is that the SPR operation is very general, modelling lateral gene transfer and introgression. The LR operation we introduce is a limited SPR, where the displaced subtree is composed of a single leaf. An alternative to the SPR operation to move subtrees within a tree is the Edge Contraction (EC) operation, that contracts an edge of an input tree, thus increasing the degree of the parent node. This operation allows correcting the local misplacement of a full subtree. AST-EC is NP-complete but can be solved in $O((2t)^{p}tn^{2})$ time where $p$ is the number of required EC operations [8].

Compared to the two problems described above, an LR models a very specific type of error in evolutionary trees, that is the misplacement of a single taxon (a single leaf) in one of the input trees. This error occurs frequently in reconstructing evolutionary trees, and can be caused for example by some evolutionary process specific to the corresponding input tree (recent incomplete lineage sorting, or recent lateral transfer for example). Conversely, it is not well adapted to model errors, due for example to ancient evolutionary events that impacts large subtrees. However, an attractive feature of the LR operation is that computing the LR distance is equivalent to computing the MAST cost and is thus tractable, unlike the SPR distance which is hard to compute. This suggests that the LR-Consensus problem might be easier to solve than the SPR Supertree problem, and we provide indeed several tractability results. Compared to the AST-EC problem, the AST-LR problem is naturally more adapted to correct single taxa misplacements as the EC operation is very local and the number of EC required to correct a taxon misplacement is linear in the length of the path to its correct location, while the LR cost of correcting this is unitary. Last, LR-Consensus is more flexible than the MAST problem as it relies on modifications of the input trees, while with the way MAST corrects a misplaced leaf requires to remove this leaf from all input trees. This shows that the problems AST-LR and AST-LR-d complement well the existing corpus of gene trees correction models.

3 Hardness and approximability of AST-LR

In this section, we show that the AST-LR problem is NP-hard, from which the LR-Consensus hardness follows. We then describe a simple factor $2$ approximation algorithm. The algorithm turns out to be useful for analyzing the worst case scenario for AST-LR in terms of the required number of leaves to remove, as we show that there are AST-LR instances that require removing about $n-\sqrt{n}$ leaves in each input tree.

NP-hardness of AST-LR

We assume here that we are considering the decision version of AST-LR, i.e. deciding whether there is a leaf-disagreement of size at most $\ell$ for a given $\ell$. We use a reduction from the MinRTI problem: given a set $\mathcal{R}$ of rooted triplets, find a subset $\mathcal{R}^{\prime}\subset\mathcal{R}$ of minimum cardinality such that $\mathcal{R}\setminus\mathcal{R}^{\prime}$ is compatible. The MinRTI problem is NP-Hard [4] (even $W[2]$-hard and hard to approximate within a $O(\log n)$ factor). Denote by $MINRTI(\mathcal{R})$ the minimum number of triplets to remove from $\mathcal{R}$ to attain compatibility. We describe the reduction here.

Let $\mathcal{R}=\{R_{1},\ldots,R_{t}\}$ be an instance of MinRTI, with the label set $L:=\bigcup_{i=1}^{t}\mathcal{X}(R_{i})$. For a given integer $m$, we construct an AST-LR instance $\mathcal{T}=\{T_{1},\ldots,T_{t}\}$ which is such that $MINRTI(\mathcal{R})\leq m$ if and only if $AST_{LR}(\mathcal{T})\leq t(|L|-3)+m$.

We first construct a tree $Z$ with additional labels which will serve as our main gadget. Let $\{L_{i}\}_{1\leq i\leq t}$ be a collection of $t$ new label sets, each of size $(|L|t)^{10}$, all disjoint from each other and all disjoint from $L$. Each tree in our AST-LR instance will be on label set $\mathcal{X}=L\cup L_{1}\cup\ldots\cup L_{t}$. For each $i\in[t]$, let $X_{i}$ be any tree with label set $L_{i}$. Obtain $Z$ by taking any tree on $t$ leaves $l_{1},\ldots,l_{t}$, then replacing each leaf $l_{i}$ by the $X_{i}$ tree (i.e. $l_{i}$ is replaced by $r(X_{i})$). Denote by $r_{Z}(X_{i})$ the root of the $X_{i}$ subtree in $Z$.

Then for each $i\in[t]$, we construct $T_{i}$ from $R_{i}$ as follows. Let $L^{\prime}=L\setminus\mathcal{X}(R_{i})$ be the set of labels not appearing in $R_{i}$, noting that $|L^{\prime}|=|L|-3$. Let $T_{L^{\prime}}$ be any tree with label set $L^{\prime}$, and obtain the tree $Z_{i}$ by grafting $T_{L^{\prime}}$ on the edge between $r_{Z}(X_{i})$ and its parent. Finally, $T_{i}$ is obtained by grafting $R_{i}$ above $Z_{i}$. See Figure 2 for an example. Note that each tree $T_{i}$ has label set $\mathcal{X}$ as desired. Also, it is not difficult to see that this reduction can be carried out in polynomial time. This construction can now be used to show the following.

Theorem 3.1

The AST-LR and LR-Consensus problems are NP-hard.

The idea of the proof is to show that in the constructed AST-LR instance, we are ”forced” to solve the corresponding MinRTI instance. In more detail, we show that $MINRTI(\mathcal{R})\leq m$ if and only if $AST_{LR}(\mathcal{T})\leq t(|L|-3)+m$. In one direction, given a set $\mathcal{R}^{\prime}$ of size $m$ such that $\mathcal{R}\setminus\mathcal{R}^{\prime}$ is compatible, one can show that the following leaf removals from $\mathcal{T}$ make it compatible: remove, from each $T_{i}$, the leaves $L^{\prime}=L\setminus\mathcal{X}(R_{i})$ that were inserted into the $Z$ subtree, then for each $R_{i}\in\mathcal{R}^{\prime}$, remove a single leaf in $\mathcal{X}(R_{i})$ from $T_{i}$. This sums up to $t(|L|-3)+m$ leaf removals. Conversely, it can be shown that there always exists an optimal solution for $\mathcal{T}$ that removes, for each $T_{i}$, all the leaves $L^{\prime}=L\setminus\mathcal{X}(R_{i})$ inserted in the $Z$ subtree, plus an additional single leaf $l$ from $m$ trees $T_{i_{1}},\dots,T_{i_{m}}$ such that $l\in L$. The corresponding triplets $R_{i_{1}},\dots,R_{i_{m}}$ can be removed from $\mathcal{R}$ so that it becomes compatible.

Approximating AST-LR and bounding worst-case scenarios

Given the above result, it is natural to turn to approximation algorithms in order to solve AST-LR or LR-Consensus instances. It turns out that there is a simple factor $2$ approximation for LR-Consensus which is achieved by interpreting the problem as finding a median in a metric space. Indeed, it is not hard to see that $d_{LR}$ is a metric (over the space of trees on the same label set $\mathcal{X}$). A direct consequence, using an argument akin to the one in [9, p.351], is the following.

Theorem 3.2

The following is a factor $2$ approximation algorithm for LR-Consensus: return the tree $T\in\mathcal{T}$ that minimizes $\sum_{T_{i}\in\mathcal{T}}d_{LR}(T,T_{i})$.

Theorem 3.2 can be used to lower-bound the ‘worst’ possible instance of AST-LR. We show that in some cases, we can only keep about $\sqrt{|\mathcal{X}|}$ leaves per tree. That is, there are instances for which $AST_{LR}(\mathcal{T})=\Omega(t(n-\sqrt{n}))$, where $t$ is the number of trees and $n=|\mathcal{X}|$. The argument is based on a probabilistic argument, for which we will make use of the following result [3, Theorem 4.3.iv].

Theorem 3.3 ([3])

For any constant $c>e/\sqrt{2}$, there is some $n_{0}$ such that for all $n\geq n_{0}$, the following holds: if $T_{1}$ and $T_{2}$ are two binary trees on $n$ leaves chosen randomly, uniformly and independently, then $\mathbb{E}[d_{LR}(T_{1},T_{2})]\geq n-c\sqrt{n}$.

Corollary 1

There are instances of AST-LR in which $\Omega(t(n-\sqrt{n}))$ leaves need to be deleted.

The above is shown by demonstrating that, by picking a set $\mathcal{T}$ of $t$ random trees, the expected optimal sum of distances $\min_{T}\sum_{T_{i}\in\mathcal{T}}d_{LR}(T,T_{i})$ is $\Omega(t(n-\sqrt{n})$. This is not direct though, since the tree $T^{*}$ that minimizes this sum is not itself random, and so we cannot apply Theorem 3.3 directly on $T^{*}$. We can however, show that the tree $T^{\prime}\in\mathcal{T}$ obtained using the 2-approximation, which is random, has expected sum of distances $\Omega(t(n-\sqrt{n}))$. Since $T^{*}$ requires, at best, half the leaf deletions of $T^{\prime}$, the result follows. Note that finding a non-trivial upper bound on $AST_{LR}(\mathcal{T})$ is open.

4 Fixed-parameter tractability of AST-LR and AST-LR-d.

An alternative way to deal with computational hardness is parameterized complexity. The most natural parameter to study is $q:=AST_{LR}(\mathcal{T})$, the question being whether there exists an algorithm for AST-LR that runs in time $O(f(q)poly(n))$ for some function $f$ depending only on $q$. In a previous version of this work, we proposed an FPT algorithm that turned out to be inaccurate. This was observed by Chen et al. [5], who proposed a fix through an alternate proof of FPT membership. We invite the interested reader to consult the second arXiv version [V2] of this paper in Section 4 for the details of the inaccurate algorithm.

We consider an alternative parameter $d$, and show that finding a tree $T^{*}$, if it exists, such that $d_{LR}(T_{i},T^{*})\leq d$ for every input tree $T_{i}$, can be done in $O(c^{d}d^{3d}(n^{3}+tn\log n))$ time for some constant $c$.

4.1 Parameterization by maximum distance $d$

We now describe an algorithm for the AST-LR-d problem, running in time $O(c^{d}d^{3d}(n^{3}+tn\log n))$ that, if it exists, finds a solution (where here $c$ is a constant not depending on $d$ nor $n$).

We employ the following branch-and-bound strategy, keeping a candidate solution at each step. Initially, the candidate solution is the input tree $T_{1}$ and, if $T_{1}$ is indeed a solution, we return it. Otherwise (in particular if $d_{LR}(T_{1},T_{i})>d$ for some input tree $T_{i}$), we branch on a set of “leaf-prune-and-regraft” operations on $T_{1}$. In such an operation, we prune one leaf from $T_{1}$ and regraft it somewhere else. If we have not produced a solution after $d$ such operations, then we halt this branch of the algorithm (as any solution must be reachable from $T_{1}$ by at most $d$ operations). The resulting search tree has depth at most $d$. In order to bound the running time of the algorithm, we need to bound the number of “leaf-prune-and-regraft” operations to try at each branching step. There are two steps to this: first, we bound the set of candidate leaves to prune, second, given a leaf, we bound the number of places where to regraft it. To bound the candidate set of leaves to prune, let us call a leaf $x$ interesting if there is a solution $T^{*}$, and minimal sets $X_{1},X_{i}\subseteq\mathcal{X}$ of size at most $d$, such that

(a) $T_{1}-X_{1}=T^{*}-X_{1}$,

(b) $T_{i}-X_{i}=T^{*}-X_{i}$, and

(c) $x\in X_{1}\setminus X_{i}$,

where $T_{i}$ is an arbitrary input tree for which $d_{LR}(T_{1},T_{i})>d$. It can be shown that an interesting leaf $x$ must exist if there is a solution. Moreover, though we cannot identify $x$ before we know $T^{*}$, we can nevertheless construct a set $S$ of size $O(d^{2})$ containing all interesting leaves. Thus, in our branching step, it suffices to consider leaves in $S$.

Assuming we have chosen the correct $x$, we then bound the number of places to try regrafting $x$. Because of the way we chose $x$, we may assume there is a solution $T^{*}$ and $X_{i}\subseteq\mathcal{X}$ such that $|X_{i}|\leq d$, $T_{i}-X_{i}=T^{*}-X_{i}$ and $x\notin X_{i}$. Thus we may treat $T_{i}$ as a “guide” on where to regraft $x$. Due to the differences between $T_{1}$, $T_{i}$ and $T^{*}$, this guide does not give us an exact location in $T_{1}$ to regraft $x$. Nevertheless, we can show that the number of candidate locations to regraft $x$ can be bounded by $O(d)$. Thus, in total we have $O(d^{3})$ branches at each step in our search tree of depth $d$, and therefore have to consider $O((O(3^{d}))^{d})=O(c^{d}d^{3d})$ subproblems.

Theorem 4.1

AST-LR-d can be solved in time $O(c^{d}d^{3d}(n^{3}+tn\log n))$, where $c$ is a constant not depending on $d$ or $n$.

5 Conclusion

To conclude, we introduced a new supertree/consensus problem, based on a simple combinatorial operator acting on trees, the Leaf-Removal. We showed that, although this supertree problem is NP-hard, it admits interesting tractability results, that compare well with existing algorithms. Future research should explore if various simple combinatorial operators, that individually define relatively tractable supertree problems (for example LR and EC) can be combined into a unified supertree problem while maintaining approximability and fixed-parameter tractability.

**Acknowledgement: MJ was partially supported by Labex NUMEV (ANR-10-LABX-20) and Vidi grant 639.072.602 from The Netherlands Organization for Scientific Research (NWO). CC was supported by NSERC Discovery Grant 249834. CS was partially supported by the French Agence Nationale de la Recherche Investissements d’Avenir/Bioinformatique (ANR-10-BINF-01-01, ANR-10-BINF-01-02, Ancestrome). ML was supported by NSERC PDF Grant. MW was supported by the Institut de Biologie Computationnelle. **

Appendix 0.A Omitted proofs

Here we give proofs for several results whose proofs were omitted in the main paper. Note that the proof of Theorem 4.1 is deferred to its own section.

Lemma 1 * (restated). Let $\mathcal{T}=\{T_{1},\ldots,T_{t}\}$ be a set of trees on the same label set $\mathcal{X}$. Then, given a supertree $T$ such that $v:=\sum_{T_{i}\in\mathcal{T}}d_{LR}(T,T_{i})$, one can compute in time $O(tn\log n)$ a leaf-disagreement $\mathcal{C}$ of size at most $v$, where $n=|\mathcal{X}|$. Conversely, given a leaf-disagreement $\mathcal{C}$ for $\mathcal{T}$ of size $v$, one can compute in time $O(tn\log^{2}(tn))$ a supertree $T$ such that $\sum_{T_{i}\in\mathcal{T}}d_{LR}(T,T_{i})\leq v$. *

Proof

In the first direction, for each $T_{i}\in\mathcal{T}$, there is a set $X_{i}\subseteq\mathcal{X}$ of size $d_{LR}(T,T_{i})$ such that $T_{i}-X_{i}=T-X_{i}$. Moreover, $X_{i}$ can be found in time $O(n\log n)$. Thus $(X_{1},\ldots,X_{t})$ is a leaf-disagreement of the desired size and can be found in time $O(tn\log n)$. Conversely, let $\mathcal{C}=(X_{1},\ldots,X_{t})$ be a leaf-disagreement of size $v$. As $\mathcal{T}^{\prime}=\{T_{1}-X_{1},\ldots,T_{t}-X_{t}\}$ is compatible, there is a tree $T$ that displays $\mathcal{T}^{\prime}$, and it is easy to see that the sum of distances between $T$ and $\mathcal{T}^{\prime}$ is at most the size of $\mathcal{C}$. As for the complexity, it is shown in [7] how to compute in time $O(tn\log^{2}(tn))$, given a set of trees $\mathcal{T}^{\prime}$, a tree $T$ displaying $\mathcal{T}^{\prime}$ if one exists. ∎

We next consider the case where $\mathcal{T}$ consists only of two trees.

Lemma 2

Let $T_{1},T_{2}$ be two trees on the same label set $\mathcal{X}$. Then $AST_{LR}(T_{1},T_{2})=d_{LR}(T_{1},T_{2})$. Moreover, every optimal leaf-disagreement $\mathcal{C}=(\mathcal{X}_{1}^{\prime},\mathcal{X}_{2}^{\prime})$ for $T_{1}$ and $T_{2}$ can be obtained in the following manner: for every label-disagreement $\mathcal{X}^{\prime}$ of size $d_{LR}(T_{1},T_{2})$, partition $\mathcal{X}^{\prime}$ into $\mathcal{X}^{\prime}_{1},\mathcal{X}^{\prime}_{2}$.

Proof

Let $\mathcal{X}^{\prime}\subset\mathcal{X}$ such that $|\mathcal{X}^{\prime}|=d_{LR}(T_{1},T_{2})$ and $T_{1}-\mathcal{X}^{\prime}=T_{2}-\mathcal{X}^{\prime}$. Then clearly, for any bipartition $(\mathcal{X}^{\prime}_{1},\mathcal{X}^{\prime}_{2})$ of $\mathcal{X}^{\prime}$, $T_{1}^{\prime}:=T_{1}-\mathcal{X}^{\prime}_{1}$ and $T_{2}^{\prime}:=T_{2}-\mathcal{X}^{\prime}_{2}$ are compatible, since the leaves that $T_{1}^{\prime}$ and $T_{2}^{\prime}$ have in common yield the same subtree, and leaves that appear in only one tree cannot create incompatibility. In particular, $AST_{LR}(T_{1},T_{2})\leq d_{LR}(T_{1},T_{2})$.

Conversely, let $\mathcal{C}=(\mathcal{X}^{\prime}_{1},\mathcal{X}^{\prime}_{2})$ be a minimum leaf-disagreement. We have $\mathcal{X}^{\prime}_{1}\cap\mathcal{X}^{\prime}_{2}=\emptyset$, for if there is some $\ell\in\mathcal{X}^{\prime}_{1}\cap\mathcal{X}^{\prime}_{2}$, then $\ell$ could be reinserted into one of the two trees without creating incompatibility. Thus $\mathcal{C}$ is a bipartition of $\mathcal{X}^{\prime}=\mathcal{X}^{\prime}_{1}\cup\mathcal{X}^{\prime}_{2}$. Moreover, we must have $T_{1}-\mathcal{X}^{\prime}=T_{2}-\mathcal{X}^{\prime}$, implying $|\mathcal{X}^{\prime}|\geq d_{LR}(T_{1},T_{2})$. Combined with the above inequality, $|\mathcal{X}^{\prime}|=d_{LR}(T_{1},T_{2})$, and the Lemma follows.

It follows from Lemma 2 that any optimal label-disagreement $\mathcal{X}^{\prime}$ can be turned into an optimal leaf-disagreement, which is convenient as $\mathcal{X}^{\prime}$ can be found in polynomial time. We will make heavy use of this property later on.

Note that the same type of equivalence does not hold when $3$ or more trees are given, i.e. computing a MAST of three trees does not necessarily yield a leaf-disagreement of minimum size. Consider for example the instance $\mathcal{T}=\{T_{1},T_{2},T_{3}\}$ in Figure 1. An optimal leaf-disagreement for $\mathcal{T}$ has size $2$ and consists of any pair of distinct leaves. On the other hand, an optimal leaf-disagreement for $\mathcal{T}$ has size $3$, and moreover each leaf corresponds to a different label.

Theorem 3.1 * (restated). The AST-LR and LR-Consensus problems are NP-hard. *

Proof

We begin by restating the reduction from MinRTI to AST-LR.

Let $\mathcal{R}=\{R_{1},\ldots,R_{t}\}$ be an instance of MinRTI, with the label set $L:=\bigcup_{i=1}^{t}\mathcal{X}(R_{i})$. For a given integer $m$, we construct an AST-LR instance $\mathcal{T}=\{T_{1},\ldots,T_{t}\}$ which is such that $MINRTI(\mathcal{R})\leq m$ if and only if $AST_{LR}(\mathcal{T})\leq t(|L|-3)+m$.

We first construct a tree $Z$ with additional labels which will serve as our main gadget. Let $\{L_{i}\}_{1\leq i\leq t}$ be a collection of $t$ new label sets, each of size $(|L|t)^{10}$, all disjoint from each other and all disjoint from $L$. Each tree in our AST-LR instance will be on label set $\mathcal{X}=L\cup L_{1}\cup\ldots\cup L_{t}$. For each $i\in[t]$, let $X_{i}$ be any tree with label set $L_{i}$. Obtain $Z$ by taking any tree on $t$ leaves $l_{1},\ldots,l_{t}$, then replacing each leaf $l_{i}$ by the $X_{i}$ tree (i.e. $l_{i}$ is replaced by $r(X_{i})$). Denote by $r_{Z}(X_{i})$ the root of the $X_{i}$ subtree in $Z$.

Then for each $i\in[t]$, we construct $T_{i}$ from $R_{i}$ as follows. Let $L^{\prime}=L\setminus\mathcal{X}(R_{i})$ be the set of labels not appearing in $R_{i}$, noting that $|L^{\prime}|=|L|-3$. Let $T_{L^{\prime}}$ be any tree with label set $L^{\prime}$, and obtain the tree $Z_{i}$ by grafting $T_{L^{\prime}}$ on the edge between $r_{Z}(X_{i})$ and its parent. Finally, $T_{i}$ is obtained by grafting $R_{i}$ above $Z_{i}$. See Figure 2 for an example. Note that each tree $T_{i}$ has label set $\mathcal{X}$ as desired. Also, it is not difficult to see that this reduction can be carried out in polynomial time.

We now show that $MINRTI(\mathcal{R})\leq m$ if and only if $AST_{LR}(\mathcal{T})\leq t(|L|-3)+m$.

($\Rightarrow$) Let $\mathcal{R}^{\prime}\subset\mathcal{R}$ such that $|\mathcal{R}^{\prime}|\leq m$ and $\mathcal{R}^{*}:=\mathcal{R}\setminus\mathcal{R}^{\prime}$ is compatible, and let $T(\mathcal{R}^{*})$ be a tree displaying $\mathcal{R}^{*}$. Note that $|\mathcal{R}^{*}|\geq t-m$. We obtain a AST-LR solution by first deleting, in each $T_{i}\in\mathcal{T}$, all the leaves labeled by $L\setminus\mathcal{X}(R_{i})$ (thus $T_{i}$ becomes the tree obtained by grafting $R_{i}$ above $Z$). Then for each deleted triplet $R_{i}\in\mathcal{R}^{\prime}$, we remove any single leaf of $T_{i}$ labeled by some element in $\mathcal{X}(R_{i})$. In this manner, no more than $t(|L|-3)+m$ leaves get deleted. Moreover, grafting $T(\mathcal{R}^{*})$ above $Z$ yields a tree displaying the modified set of trees, showing that they are compatible.

($\Leftarrow$) We first argue that if $\mathcal{T}$ admits a leaf-disagreement $\mathcal{C}=(\mathcal{X}_{1},\ldots,\mathcal{X}_{t})$ of size at most $t(|L|-3)+m$, then there is a better or equal solution that removes, in each $T_{i}$, all the leaves labeled by $L\setminus\mathcal{X}(R_{i})$ (i.e. those grafted in the $Z_{i}$ tree). For each $i\in[t]$, let $T_{i}^{\prime}=T_{i}-\mathcal{X}_{i}$, and denote $\mathcal{T}^{\prime}=\{T_{1}^{\prime},\ldots,T_{t}^{\prime}\}$. Suppose that there is some $i\in[t]$ and some $\ell\in L\setminus\mathcal{X}(R_{i})$ such that $\ell\in\mathcal{X}(T_{i}^{\prime})$.

We claim that $\ell\notin\mathcal{X}(T_{j}^{\prime})$ for every $i\neq j\in[t]$. Suppose otherwise that $\ell\in\mathcal{X}(T_{j}^{\prime})$ for some $j\neq i$. Consider first the case where $\ell\notin\mathcal{X}(R_{j})$. Note that by the construction of $Z_{i}$ and $Z_{j}$, for every $x_{i}\in\mathcal{X}(X_{i})\cap\mathcal{X}(T_{i}^{\prime})\cap\mathcal{X}(T_{j}^{\prime})$ and every $x_{j}\in\mathcal{X}(X_{j})\cap\mathcal{X}(T_{i}^{\prime})\cap\mathcal{X}(T_{j}^{\prime})$, $T_{i}^{\prime}$ contains the $\ell x_{i}|x_{j}$ triplet whereas $T_{j}^{\prime}$ contains the $\ell x_{j}|x_{i}$ triplet. Since these triplets are conflicting, no supertree can contain both and so no such $x_{i},x_{j}$ pair can exist, as we are assuming that a supertree for $T^{\prime}_{i}$ and $T^{\prime}_{j}$ exists. This implies that one of $\mathcal{X}(X_{i})\cap\mathcal{X}(T_{i}^{\prime})\cap\mathcal{X}(T_{j}^{\prime})$ or $\mathcal{X}(X_{j})\cap\mathcal{X}(T_{i}^{\prime})\cap\mathcal{X}(T_{j}^{\prime})$ must be empty. Suppose without loss of generality that the former is empty. Then each $x_{i}\in X_{i}$ must have been deleted in at least one of $T_{i}$ or $T_{j}$. As $|\mathcal{X}(X_{i})|=(|L|t)^{10}>t(|L|-3)+m$, this contradicts the size of the solution $\mathcal{C}$. In the second case, we have $\ell\in\mathcal{X}(R_{j})$. But this time, if there are $x_{i}\in\mathcal{X}(X_{i})\cap\mathcal{X}(T_{i}^{\prime})\cap\mathcal{X}(T_{j}^{\prime})$ and $x_{j}\in\mathcal{X}(X_{j})\cap\mathcal{X}(T_{i}^{\prime})\cap\mathcal{X}(T_{j}^{\prime})$, then $T_{j}^{\prime}$ contains the $x_{i}x_{j}|\ell$ triplet, again conflicting with the $\ell x_{i}|x_{j}$ triplet found in $T_{i}$. As before, we run into a contradiction since too many $X_{i}$ or $X_{j}$ leaves need to be deleted. This proves our claim.

We thus assume that $\ell$ only appears in $T_{i}^{\prime}$. Let $R_{j}\in\mathcal{R}$ such that $\ell\in\mathcal{X}(R_{j})$, noting that $\ell$ does not appear in $T^{\prime}_{j}$. Consider the solution $\mathcal{T}^{\prime\prime}$ obtained from $\mathcal{T}^{\prime}$ by removing $\ell$ from $T_{i}^{\prime}$, and placing it back in $T_{j}^{\prime}$ where it originally was in $T_{j}$. Formally this is achieved by replacing, in the leaf-disagreement $\mathcal{C}$, $\mathcal{X}_{i}$ by $\mathcal{X}_{i}\cup\{\ell\}$ and $\mathcal{X}_{j}$ by $\mathcal{X}_{j}\setminus\{\ell\}$. Since $\ell$ still appears only in one tree, no conflict is created and we obtain another solution of equal size. By repeating this process for every such leaf $\ell$, we obtain a solution in which every leaf labeled by $L\setminus\mathcal{X}(R_{i})$ is removed from $T_{i}^{\prime}$. We now assume that the solution $\mathcal{T}^{\prime}$ has this form.

Consider the subset $\mathcal{R}^{\prime}=\{R_{i}\in\mathcal{R}:|\mathcal{X}(T_{i}^{\prime})\cap\mathcal{X}(R_{i})|<3\}$, that is those triplets $R_{i}$ for which the corresponding tree $T_{i}$ had a leaf removed outside of the $Z_{i}$ tree. By the form of the $\mathcal{T}^{\prime}$ solution, at least $t(|L|-3)$ removals are done in the $Z_{i}$ trees, and as only $m$ removals remain, $\mathcal{R}^{\prime}$ has size at most $m$. We show that $\mathcal{R}\setminus\mathcal{R}^{\prime}$ is a compatible set of triplets. Since $\mathcal{T}^{\prime}$ is compatible, there is a tree $T$ that displays each $T_{i}^{\prime}\in\mathcal{T}^{\prime}$, and since each triplet of $\mathcal{R}\setminus\mathcal{R}^{\prime}$ belongs to some $T_{i}^{\prime}$, $T$ also displays $\mathcal{R}\setminus\mathcal{R}^{\prime}$. This concludes the proof. ∎

Theorem 3.2 * (restated). The following is a factor $2$ approximation algorithm for LR-Consensus: return the tree $T\in\mathcal{T}$ that minimizes $\sum_{T_{i}\in\mathcal{T}}d_{LR}(T,T_{i})$. *

Proof

Let $T^{*}$ be an optimal solution for LR-Consensus, i.e. $T^{*}$ is a tree minimizing $\sum_{T_{i}\in\mathcal{T}}d_{LR}(T_{i},T^{*})$, and let $T$ be chosen as described in the theorem statement. Moreover let $T^{\prime}$ be the tree of $\mathcal{T}$ minimizing $d_{LR}(T^{\prime},T^{*})$. By the triangle inequality,

[TABLE]

where the last inequality is due to the fact that $d_{LR}(T^{\prime},T^{*})\leq d_{LR}(T^{*},T_{i})$ for all $i$, by our choice of $T^{\prime}$. Our choice of $T$ implies $\sum_{T_{i}\in\mathcal{T}}d_{LR}(T,T_{i})\leq\sum_{T_{i}\in\mathcal{T}}d_{LR}(T^{\prime},T_{i})\leq 2\sum_{T_{i}\in\mathcal{T}}d_{LR}(T_{i},T^{*})$. ∎

Corollary 1 * (restated). There are instances of AST-LR in which $\Omega(t(n-\sqrt{n}))$ leaves need to be deleted. *

Proof

Let $\mathcal{T}=\{T_{1},\ldots,T_{t}\}$ be a random set of $t$ trees chosen uniformly and independently. For large enough $n$, the expected sum of distances between each pair of trees is

[TABLE]

for some constant $c$, by Theorem 3.3. Let $S:=\min_{T}\sum_{i=1}^{t}d_{LR}(T,T_{i})$ be the random variable corresponding to the minimum sum of distances. By Theorem 3.2, there is a tree $T^{\prime}\in\mathcal{T}$ such that $\sum_{i=1}^{t}d_{LR}(T^{\prime},T_{i})\leq 2S$. We have

[TABLE]

Since, in general for two random variables $X$ and $Y$, always having $X\leq Y$ implies $\mathbb{E}[X]\leq\mathbb{E}[Y]$, we get

[TABLE]

yielding $\mathbb{E}[S]\geq t/4(n-c\sqrt{n})=\Omega(t(n-\sqrt{n}))$, and so there must exist an instance $\mathcal{T}$ satisfying the statement.

Appendix 0.B Leaf Prune-and-Regraft Moves

Here we introduce the notion of leaf prune-and-regraft (LPR) moves, which will be used in the proof of Theorem 4.1, and which may be of independent interest. In an LPR move, we prune a leaf from a tree and then regraft it another location (formal definitions below). LPR moves provide an alternate way of characterizing the distance function $d_{LR}$ - indeed, we will show that $d_{LR}(T_{1},T_{2})\leq k$ if and only if there is a sequence of at most $k$ LPR moves transforming $T_{1}$ into $T_{2}$.

Definition 1

Let $T$ be a tree on label set $\mathcal{X}$. A LPR move on $T$ is a pair $(\ell,e)$ where $\ell\in\mathcal{X}$ and $e\in\{E(T-\{\ell\}),\perp\}$. Applying $(\ell,e)$ consists in grafting $\ell$ on the $e$ edge of $T-\{\ell\}$ if $e\neq\perp$, and above the root of $T-\{\ell\}$ if $e=\perp$.

An LPR sequence $L=((\ell_{1},e_{1}),\ldots,(\ell_{k},e_{k}))$ is an ordered tuple of LPR moves, where for each $i\in[k]$, $(\ell_{i},e_{i})$ is an LPR move on the tree obtained after applying the first $i-1$ LPR moves of $L$. We may write $L=(\ell_{1},\ldots,\ell_{k})$ if the location at which the grafting takes place does not need to be specified. We say that $L$ turns $T_{1}$ into $T_{2}$ if, by applying each LPR move of $L$ in order on $T_{1}$, we obtain $T_{2}$.

See Figure 4 for an example of an LPR sequence.

In the following statements, we assume that $T_{1}$ and $T_{2}$ are two trees on label set $\mathcal{X}$. We exhibit an equivalence between leaf removals and LPR sequences, then show that the order of LPR moves in a sequence do not matter in terms of turning one tree into another - in particular any leaf can be displaced first.

Lemma 3

There is a subset $X\subseteq\mathcal{X}$ such that $T_{1}-X=T_{2}-X$ if and only if there is an LPR sequence $(x_{1},x_{2},\ldots,x_{k})$ turning $T_{1}$ into $T_{2}$ such that $X=\{x_{1},\ldots,x_{k}\}$.

Proof

If $T_{1}=T_{2}$ then the proof is trivial, so we will assume this is not the case. We prove the lemma by induction on $|X|$.

For the base case, suppose that $X=\{x\}$. If $T_{1}-X=T_{2}-X$, then let $T_{m}=T_{1}-X=T_{2}-X$. We find an LPR move $(x,e)$ with $e\in E(T_{m})\cup\{\perp\}$ turning $T_{1}$ into $T_{2}$. Observe that $T_{2}$ can be obtained by grafting $x$ on $T_{m}$, either on an edge $uv$, in which case we set $e=uv$, or above the root, in which case we set $e=\perp$. Since $T_{m}=T_{1}-\{x\}$, it follows that $(x,e)$ is an LPR move turning $T_{1}$ into $T_{2}$. In the other direction, assume there is an LPR move $(x,e)$ turning $T_{1}$ into $T_{2}$. Observe that for any tree $T^{\prime}$ derived from $T_{1}$ by an LPR move using $x$, $T^{\prime}-\{x\}=T_{1}-\{x\}$. In particular, $T_{2}-\{x\}=T_{1}-\{x\}$ and we are done.

For the induction step, assume that $|X|>1$ and that the claim holds for any $X^{\prime}$ such that $|X^{\prime}|<|X|$. If $T_{1}-X=T_{2}-X$, then define $T_{m}=T_{1}-X$, and let $x$ be an arbitrary element of $X$. We will first construct a tree $T_{1}^{\prime}$ such that $T_{1}-\{x\}=T_{1}^{\prime}-\{x\}$ and $T_{1}^{\prime}-(X\setminus\{x\})=T_{2}-(X\setminus\{x\})$.

Observe that $T_{2}-(X\setminus\{x\})$ can be obtained by grafting $x$ in $T_{m}$. Let $e=uv$ if this grafting takes place on an edge of $T_{m}$ with $v$ being the child of $u$, or $e=\perp$ if $x$ is grafted above $T_{m}$, and in this case let $v=r(T_{m})$. Let $v^{\prime}=v_{T_{1}-\{x\}}$ be the node in $T_{1}-\{x\}$ corresponding to $v$.

Let $T_{1}^{\prime}$ be derived from $T_{1}-\{x\}$ by grafting $x$ onto the edge between $v^{\prime}$ and its parent if $v^{\prime}$ is non-root, and grafting above $v^{\prime}$ otherwise. It is clear that $T_{1}-\{x\}=T_{1}^{\prime}-\{x\}$. Furthermore, by our choice of $v^{\prime}$ we have that $T_{1}^{\prime}-(X\setminus\{x\})=T_{2}-(X\setminus\{x\})$.

Now that we have $T_{1}-\{x\}=T_{1}^{\prime}-\{x\}$ and $T_{1}^{\prime}-(X\setminus\{x\})=T_{2}-(X\setminus\{x\})$, by the inductive hypothesis there is an LPR sequence turning $T_{1}$ into $T_{1}^{\prime}$ consisting of a single move $(x,e)$, and an LPR sequence $(x_{1},x_{2},\ldots,x_{k-1})$ turning $T_{1}^{\prime}$ into $T_{2}$ such that $\{x_{1},\ldots,x_{k^{\prime}}\}=(X\setminus\{x\})$. Then by concatenating these two sequences, we have an LPR sequence $(x_{1},x_{2},\ldots,x_{k})$ turning $T_{1}$ into $T_{2}$ such that $X=\{x_{1},\ldots,x_{k}\}$.

For the converse, suppose that there is an LPR sequence $(x_{1},x_{2},\ldots,x_{k})$ turning $T_{1}$ into $T_{2}$ such that $X=\{x_{1},\ldots,x_{k}\}$. Let $T_{1}^{\prime}$ be the tree derived from $T_{1}$ by applying the first move in this sequence. That is, there is an LPR move $(x_{1},e)$ turning $T_{1}$ into $T_{1}^{\prime}$, and there is an LPR sequence $(x_{2},\ldots,x_{k})$ turning $T_{1}^{\prime}$ into $T_{2}$. Then by the inductive hypothesis $T_{1}-\{x_{1}\}=T_{1}^{\prime}-\{x_{1}\}$ and $T_{1}^{\prime}-\{x_{2},\ldots,x_{k}\}=T_{2}-\{x_{2},\ldots,x_{k}\}$. Thus, $T_{1}-X=T_{1}^{\prime}-X=T_{2}-X$, as required. ∎

Lemma 4

If there is an LPR sequence $L=(x_{1},\ldots,x_{k})$ turning $T_{1}$ into $T_{2}$, then for any $i\in[k]$, there is an LPR sequence $L^{\prime}=(x^{\prime}_{1},\ldots,x^{\prime}_{k})$ turning $T_{1}$ into $T_{2}$ such that $x^{\prime}_{1}=x_{i}$ and $\{x_{1},\ldots,x_{k}\}=\{x^{\prime}_{1},\ldots,x^{\prime}_{k}\}$.

Proof

Consider again the proof that if $T_{1}-X=T_{2}-X$ then there is an LPR sequence $(x_{1},\dots x_{k})$ turning $T_{1}$ into $T_{2}$ such that $X=\{x_{1},\dots,x_{k}\}$ (given in the proof of Lemma 3). When $|X|>1$, we construct this sequence by concatenating the LPR move $(x,e)$ with an LPR sequence of length $|X|-1$, where $x$ is an arbitrary element of $X$. As we could have chosen any element of $X$ to be $x$, we have the following: If $T_{1}-X=T_{2}-X$ then for each $x\in X$, there is an LPR sequence $(x_{1},\dots,x_{k})$ turning $T_{1}$ into $T_{2}$ such that $X=\{x_{1},\dots,x_{k}\}$ and $x_{1}=x$.

Thus our proof is as follows: Given an LPR sequence $L=(x_{1},\ldots,x_{k})$ turning $T_{1}$ into $T_{2}$ and some $i\in[k]$, Lemma 3 implies that $T_{1}-\{x_{1},\ldots,x_{k}\}=T_{2}-\{x_{1},\ldots,x_{k}\}$. By the observation above, this implies that there is an LPR sequence $(x^{\prime}_{1},\dots,x^{\prime}_{k})$ turning $T_{1}$ into $T_{2}$ such that $\{x_{1},\ldots,x_{k}\}=\{x^{\prime}_{1},\ldots,x^{\prime}_{k}\}$ and $x^{\prime}_{1}=x$. ∎

Appendix 0.C Proof of Theorem 4.1

This section makes use of the concept of LPR moves, which are introduced in the previous section. As discussed in the main paper, we employ a branch-and-bound style algorithm, in which at each step we alter a candidate solution by pruning and regrafting a leaf. That is, we apply an LPR move.

The technically challenging part is bound the number of possible LPR moves to try. To do this, we will prove Lemma 7, which provides a bound on the number of leaves to consider, and Lemma 10, which bounds the number of places a leaf may be regrafted to.

Denote by $tr(T)$ the set of rooted triplets of a tree $T$. Two triplets $R_{1}\in tr(T_{1})$ and $R_{2}\in tr(T_{2})$ are conflicting if $R_{1}=ab|c$ and $R_{2}\in\{ac|b,bc|a\}$. We denote by $conf(T_{1},T_{2})$ the set of triplets of $T_{1}$ for which there is a conflicting triplet in $T_{2}$. That is, $conf(T_{1},T_{2})=\{ab|c\in tr(T_{1}):ac|b\in tr(T_{2})$ or $bc|a\in tr(T_{2})\}$. Finally we denote by $confset(T_{1},T_{2})=\{\{a,b,c\}:ab|c\in conf(T_{1},T_{2})\}$, i.e. the collection of 3-label sets formed by conflicting triplets. Given a collection $C=\{S_{1},\ldots,S_{|C|}\}$ of sets, a hitting set of $C$ is a set $S$ such that $S\cap S_{i}\neq\emptyset$ for each $S_{i}\in C$.

Lemma 5

Let $X\subseteq\mathcal{X}$. Then $T_{1}-X=T_{2}-X$ if and only if $X$ is a hitting set of $confset(T_{1},T_{2})$.

Proof

It is known that for two rooted trees $T_{1},T_{2}$ that are leaf-labelled and binary, $T_{1}=T_{2}$ if and only if $tr(T_{1})=tr(T_{2})$ [2]. Note also that $tr(T-X)=\{ab|c\in tr(T_{1}):X\cap\{a,b,c\}=\emptyset\}$ for any tree $T$ and $X\subseteq\mathcal{X}$.

Therefore we have that $T_{1}-X=T_{2}-X$ if and only if $tr(T_{1}-X)=tr(T_{2}-X)$, which holds if and only if for every $a,b,c\in\mathcal{X}\setminus X$, if $ab|c\in tr(T_{1})$ then $ab|c\in tr(T_{2})$. This in turn occurs if and only if $X$ is a hitting set for $confset(T_{1},T_{2})$. ∎

In what follows, we call $X\subseteq\mathcal{X}$ a minimal disagreement between $T_{1}$ and $T_{2}$ if $T_{1}-X=T_{2}-X$ and for any $X^{\prime}\subset X$, $T_{1}-X^{\prime}\neq T_{2}-X^{\prime}$.

Lemma 6

Suppose that $d<d_{LR}(T_{1},T_{2})\leq d^{\prime}+d$ with $d^{\prime}\leq d$, and that there is a tree $T^{*}$ and subsets $X_{1},X_{2}\subseteq\mathcal{X}$ such that $T_{1}-X_{1}=T^{*}-X_{1}$, $T_{2}-X_{2}=T^{*}-X_{2}$ and $|X_{1}|\leq d^{\prime},|X_{2}|\leq d$. Then, there is a minimal disagreement $X$ between $T_{1}$ and $T_{2}$ of size at most $d+d^{\prime}$ and $x\in X$ such that $x\in X_{1}\setminus X_{2}$.

Proof

Let $X^{\prime}=X_{1}\cup X_{2}$. Observe that $T_{1}-X^{\prime}=T^{*}-X^{\prime}=T_{2}-X^{\prime}$ and $|X^{\prime}|\leq d+d^{\prime}$. Letting $X$ be the minimal subset of $X^{\prime}$ such that $T_{1}-X=T_{2}-X$, we have that $X$ is a minimal disagreement between $T_{1}$ and $T_{2}$ and $|X|\leq d+d^{\prime}$. Furthermore as $|X|\geq d_{LR}(T_{1},T_{2})>d$, $|X\setminus X_{2}|>0$, and so there is some $x\in X$ with $x\in X\setminus X_{2}=X_{1}\setminus X_{2}$. ∎

We are now ready to state and prove Lemma 7.

Lemma 7

Suppose that $d_{LR}(T_{1},T_{2})\leq d$ for some integer $d$. Then, there is some $S\subseteq\mathcal{X}$ such that $|S|\leq 8d^{2}$, and for any minimal disagreement $X$ between $T_{1}$ and $T_{2}$ with $|X|\leq d$, $X\subseteq S$. Moreover $S$ can be found in time $O(n^{2})$.

We will call $S$ as described in Lemma 7 a $d$-disagreement kernel between $T_{1}$ and $T_{2}$. Thus Lemma 6 essentially states that if $T_{1}$ isn’t a solution and $d_{LR}(T_{1},T_{2})>d$, then for $T_{1}$ to get closer to a solution, there is a leaf $x$ in the $d_{LR}(T_{1},T_{2})$-disagreement kernel that needs to be removed and regrafted in a location that $T_{2}$ ‘agrees with’. Lemma 7 in turn gives us a set $S$ of size at most $8d^{2}$ such that the desired $x$ must be contained in $S$.

Proof

By Lemma 5, it is enough to find a set $S$ such that $S$ contains every minimal hitting set of $confset(T_{1},T_{2})$ of size at most $d$.

We construct $S$ as follows.

Let $X$ be a subset of $\mathcal{X}$ of size at most $d$ such that $T_{1}-X=T_{2}-X$. As previously noted, this can found in time $O(n\log n)$ [6].

For notational convenience, for each $x\in X$ we let $x_{1},x_{2}$ be two new labels, and set $X_{1}=\{x_{1}:x\in X\}$, $X_{2}=\{x_{2}:x\in X\}$. Thus, $X_{1},X_{2}$ are disjoint “copies” of $X$. Let $T_{1}^{\prime}$ be derived from $T_{1}$ by replacing every label from $X$ with the corresponding label in $X_{1}$, and similarly let $T_{2}^{\prime}$ be derived from $T_{2}$ by replacing every label from $X$ with the corresponding label in $X_{2}$.

Let $T_{J}$ be a tree with label set $(\mathcal{X}\setminus X)\cup X_{1}\cup X_{2}$ such that $T_{J}-X_{2}=T_{1}^{\prime}$ and $T_{J}-X_{1}=T_{2}^{\prime}$. The tree $T_{J}$ always exists and can be found in polynomial time. Intuitively, we can start from $T_{1}^{\prime}$, and graft the leaves of $X_{2}$ where $T_{2}$ “wants” them to be. See Figure 4 for an example. Algorithm 1 gives a method for constructing $T_{J}$, and takes $O(n^{2})$ time.

In addition, let $L$ be the set of all labels in $\mathcal{X}\setminus X$ that are descended in $T_{J}$ from $\textsc{lca}_{T_{J}}(X_{1}\cup X_{2})$, and let $R=\mathcal{X}\setminus(L\cup X)$. Thus, $L,X,R$ form a partition of $\mathcal{X}$, and $L,X_{1},X_{2},R$ form a partition of the labels of $T_{J}$.

For the rest of the proof, we call $\{x,y,z\}$ a conflict triple if $\{x,y,z\}\in confset(T_{1},T_{2})$.

We first observe that no triple in $confset(T_{1},T_{2})$ contains a label in $R$. Indeed, consider a triple $\{x,y,z\}$. Any conflict triple must contain a label from $X$, so assume without loss of generality that $x\in X,z\in R$. If $x\in X,y\in L,z\in R$, then we have that $T_{J}$ contains the triplets $x_{1}y|z,x_{2}y|z$, and so $T_{1}$ and $T_{2}$ both contain $xy|z$, and $\{x,y,z\}$ is not a conflict triple. Similarly if $x,y\in X,z\in R$, then $T_{J}$ contains the triplets $x_{1}y_{1}|z,x_{2}y_{2}|z$, and again $\{x,y,z\}$ is not a conflict triple. If $x\in X$ and $y,z\in R$, then the triplet on $\{x_{1},y,z\}$ in $T_{J}$ depends only on the relative positions in $T_{J}$ of $y,z$ and $\textsc{lca}_{T_{J}}(X_{1}\cup X_{2})$. Thus we get the same triplet if we replace $x_{1}$ with $x_{2}$, and so $\{x,y,z\}$ is not a conflict triple.

This concludes the proof that no triple in $confset(T_{1},T_{2})$ contains a label in $R$. Having shown this, we may conclude that any minimal disagreement between $T_{1}$ and $T_{2}$ is disjoint from $R$, and so our returned set $S$ only needs to contain labels in $L\cup X$.

Now consider the tree $T^{*}=T_{J}|_{X_{1}\cup X_{2}}$, i.e. the subtree of $T_{J}$ restricted to the labels in $X_{1}\cup X_{2}$. Thus in the example of Figure 4, $T^{*}$ is the subtree of $T_{J}$ spanned by $\{x_{1},x_{2},y_{1},y_{2}\}$. We will now use the edges of $T^{*}$ to form a partition of $L$, as follows. For any edge $uv$ in $T^{*}$ with $u$ the parent of $v$, let $s(uv)$ denote the set of labels $y\in\mathcal{X}$ such that $y$ has an ancestor which is an internal node on the path from $u$ to $v$ in $T_{J}$, but $y$ is not a descendant of $v$ itself. For example in Figure 4, if $u$ is the least common ancestor of $x_{1},y_{1}$ and $v$ is the least common ancestor of $x_{1},y_{2}$, then $uv$ is an edge in $T^{*}$ and $s(uv)=\{c,d,e\}$.

Observe that $\{s(uv):uv\in E(T^{*})\}$ forms a partition of $L$. (Indeed, for any $l\in L$, let $u$ be the minimal element in $T^{*}$ on the path in $T_{J}$ between $l$ and $\textsc{lca}_{T_{J}}(X_{1}\cup X_{2})$ (note that $u$ exists as $\textsc{lca}_{T_{J}}(X_{1}\cup X_{2})$ itself is in $T^{*}$). As $u$ is in $T^{*}$, both of its children are on paths in $T_{J}$ between $u$ and a child of $u$ in $T^{*}$. In particular, the child of $u$ which is an ancestor of $l$ is an internal node on the path between $u$ and $v$ in $T_{J}$, for some child $v$ of $u$ in $T^{*}$, and $l$ is not descended from $v$ by construction. It is clear by construction that all $s(uv)$ are disjoint.)

The main idea behind the construction of $S$ is that we will add $X$ to $S$, together with $O(d)$ labels from $s(uv)$ for each edge $uv$ in $T^{*}$. As the number of edges in $T^{*}$ is $2(|X_{1}\cup X_{2}|-1)=O(d)$, we have the required bound of $O(d^{2})$ on $|S|$.

So now consider $s(uv)$ for some edge $uv$ in $T^{*}$. In order to decide which labels to add to $S$, we need to further partition $s(uv)$. Let $u=u_{0}u_{1}\dots u_{t}=v$ be the path in $T_{J}$ from $u$ to $v$. For each $i\in[t-1]$ (note that this does not include $i=0$), we call the set of labels descended from $u_{i}$ but not $u_{i+1}$ a dangling clade. Observe that the dangling clades form a partition of $s(uv)$. Thus in the example of Figure 4, if $u$ is the least common ancestor of $x_{1},y_{1}$ and $v$ is the least common ancestor of $x_{1},y_{2}$, then for the edge $uv$ the dangling clades are $\{c\}$ and $\{d,e\}$f.

We now make the following observations about the relation between $s(uv)$ and triples in $confset(T_{1},T_{2})$.

Observation 1: if $\{x,y,z\}$ is a conflict triple and $x\in s(uv),y,z\notin s(uv)$, then $\{x^{\prime},y,z\}$ is also a conflict triple for any $x^{\prime}\in s(uv)$. (The intuition behind this is that there are no labels appearing ’between’ $x$ and $x^{\prime}$ that are not in $s(uv)$.)

Observation 2: for any triple $\{x,y,z\}$ with $x,y\in s(uv)$, $\{x,y,z\}$ is a conflict triple if and only if $x,y$ are in different dangling clades and $z\in X$ with $z_{i}$ descended from $v$, $z_{3-i}$ not descended from $u_{1}$ for some $i\in[2]$ (recall that $z_{1}\in X_{1}$ and $z_{2}\in X_{2}$). To prove one direction, it is easy to see that if the conditions hold, then $T_{i}$ displays either $xz|y$ or $yz|x$ (depending on which dangling clade appears ’higher’), and $T_{3-i}$ displays $xy|z$. For the converse, observe first that $z\in X$ as $X$ is a hitting set for $confset(T_{1},T_{2})$ and $x,y\notin X$. Then if $xy$ are in the same dangling clade, we have that both $T_{1}$ and $T_{2}$ display $xy|z$. So $x,y$ must be in different dangling clades. Next observe that each of $z_{1},z_{2}$ must either be descended from $v$ or not descended from $u_{1}$, as otherwise $v$ would not be the child of $u$ in $T^{*}$. If $z_{1},z_{2}$ are both descended from $v$ or neither are descended from $u_{1}$, then $T_{1}$ and $T_{2}$ display the same triplet on $\{x,y,z\}$. So instead one must be descended from $v$ and one not descended from $u_{1}$, as required.

Using Observations 1 and 2, we now prove the following:

Observation 3: for any minimal disagreement $X^{\prime}$ between $T_{1}$ and $T_{2}$, one of the following holds:

• •

$X^{\prime}\cap s(uv)=\emptyset$;

• •

$s(uv)\subseteq X^{\prime}$;

• •

$s(uv)\setminus X^{\prime}$ forms a single dangling clade.

To see this, let $X^{\prime}$ be any minimal hitting set of $confset(T_{1},T_{2})$ with $s(uv)\cap X^{\prime}\neq\emptyset$ and $s(uv)\setminus X^{\prime}\neq\emptyset$. As $X^{\prime}$ is minimal, any $x\in s(uv)\cap X^{\prime}$ must be in a conflict triple $\{x,y,z\}$ with $y,z\notin X^{\prime}$. As $X$ is a hitting set for $confset(T_{1},T_{2})$, at least one of $y,z$ must be in $X$. If $y,z\notin s(uv)$, then by Observation 1 $\{x^{\prime},y,z\}$ is also a conflict triple for any $x^{\prime}\in s(uv)\setminus X^{\prime}$. But this is a contradiction as $\{x^{\prime},y,z\}$ has no elements in $X^{\prime}$. Then one of $y,z$ must also be in $s(uv)$. Suppose without loss of generality that $y\in s(uv)$. We must also have that $z\in X$, as $X$ is a hitting set for $confset(T_{1},T_{2})$ and $x,y\notin X$. By Observation 2, we must have that one of $z_{1},z_{2}$ is descended from $v$, and the other is not descended from $u_{1}$. This in turn implies (again by Observation 2) that for any $x^{\prime}\in s(uv)\setminus X^{\prime}$, if $x^{\prime}$ and $y$ are in different dangling clades then $\{x^{\prime},y,z\}$ is a conflict triple. Again this is a contradiction as $\{x^{\prime},y,z\}$ has no elements of $X^{\prime}$, and so we may assume that all elements of $s(uv)\setminus X^{\prime}$ are in the same dangling clade.

It remains to show that every element of this dangling clade is in $s(uv)\setminus X^{\prime}$. To see this, suppose there exists some $x\in X^{\prime}$ in the same dangling clade as the elements of $s(uv)\setminus X^{\prime}$. Once again we have that $x$ is in some conflict triple $\{x,y,z\}$ with $y,z\notin X^{\prime}$, and if $y,z\notin s(uv)$ then $\{x^{\prime},y,z\}$ is also a conflict triple for any $x^{\prime}\in s(uv)\setminus X^{\prime}$, a contradiction. So we may assume that one of $y,z$ is in $s(uv)\setminus X^{\prime}$. But all elements of $s(uv)\setminus X^{\prime}$ are in the same dangling clade as $x$, and so by Observation 2 $\{x,y,z\}$ cannot be a conflict triple, a contradiction. So finally we have that all elements of $s(uv)\setminus X^{\prime}$ are in the same dangling clade and all elements of this clade are in $s(uv)\setminus X^{\prime}$, as required.

With the proof of Observation 3 complete, we are now in a position to construct $S$. For any minimal hitting set $X^{\prime}$ of $confset(T_{1},T_{2})$ with size at most $d$, by Observation 3 either $X^{\prime}\cap s(uv)=\emptyset$, or $s(uv)\subseteq X^{\prime}$ (in which case $|s(uv)|\leq d$), or $s(uv)\setminus X^{\prime}$ forms a single dangling clade $C$ (in which case $|s(uv)\setminus C|\leq d$).

So add all elements of $X$ to $S$. For all $uv\in E(T_{J})$ and any dangling clade $C$ of labels in $s(uv)$, add $s(uv)\setminus C$ to $S$ if $|s(uv)\setminus C|\leq d$. Observe that this construction adds at most $2d$ labels from $s(uv)$ to $S$.

Thus, in total, we have that the size of $S$ is at most $|X|+2d|E(T_{J})|\leq d+2d(2(|X_{1}\cup X_{2}|-1))\leq d+2d(4d-2)=8d^{2}-3d\leq 8d^{2}$.

Algorithm 2 describes the full procedure formally. The construction of $T_{J}$ occurs once and as noted above takes $O(n^{2})$ time. As each other line in the algorithm is called at most $n$ times and takes $O(n)$ time, the overall running time of the algorithm $O(n^{2})$. ∎

The last ingredient needed for Theorem 4.1 is Lemma 10, which shows that if a leaf $x$ of $T_{1}$ as described in Lemma 6 has to be moved, then there are not too many ways to regraft it in order to get closer to $T^{*}$.

In the course of the following proofs, we will want to take observations about one tree and use them to make statements about another. For this reason it’s useful to have a concept of one node ”corresponding” to another node in a different tree. In the case of leaf nodes this concept is clear - two leaf nodes are equivalent if they are assigned the same label- but for internal nodes there is not necessarily any such correspondence. However, in the case that one tree is the restriction of another to some label set, we can introduce a well-defined notion of correspondence:

Given two trees $T,T^{\prime}$ such that $T^{\prime}=T|_{X}$ for some $X\subseteq\mathcal{X}(T)$, and a node $u\in V(T^{\prime})$, define the node $u_{T}$ of $T$ by $u_{T}=\textsc{lca}_{T}(\mathcal{L}_{T^{\prime}}(u))$. That is, $u_{T}$ is the least common ancestor, in $T$, of the set of labels belonging to descendants of $u$ in $T^{\prime}$. We call $u_{T}$ the node corresponding to $u$ in $T$.

We note two useful properties of $u_{T}$ here:

Lemma 8

For any $T,T^{\prime},X\subseteq\mathcal{X}(T)$ such that $T^{\prime}=T|_{X}$ and any $u,v\in V(T^{\prime})$, $u_{T}$ is an ancestor of $v_{T}$ if and only if $u$ is an ancestor of $v$.

Proof

If $u$ is an ancestor of $v$ then $\mathcal{L}_{T^{\prime}}(v)\subseteq\mathcal{L}_{T^{\prime}}(u)$, which implies that $u_{T}$ is an ancestor of $v_{T}$. For the converse, observe that for any $Z\subseteq X$, any label in $X$ descending from $\textsc{lca}_{T}(Z)$ in $T$ is also descending from $\textsc{lca}_{T^{\prime}}(Z)$ in $T^{\prime}$. In particular letting $Z=\mathcal{L}_{T^{\prime}}(u)$, we have $\mathcal{L}_{T}(u_{T})\cap X=\mathcal{L}_{T}(\textsc{lca}_{T}(Z))\cap X\subseteq\mathcal{L}_{T^{\prime}}(\textsc{lca}_{T^{\prime}}(Z))=\mathcal{L}_{T^{\prime}}(\textsc{lca}_{T^{\prime}}(\mathcal{L}_{T^{\prime}}(u)))=\mathcal{L}_{T^{\prime}}(u)\subseteq\mathcal{L}_{T}(u_{T})\cap X$. Thus $\mathcal{L}_{T^{\prime}}(u)=\mathcal{L}_{T}(u_{T})\cap X$ and similarly $\mathcal{L}_{T^{\prime}}(v)=\mathcal{L}_{T}(v_{T})\cap X$. Then we have that $u_{T}$ being an ancestor of $v_{T}$ implies $\mathcal{L}_{T}(v_{T})\subseteq L_{T}(u_{T})$, which implies that $\mathcal{L}_{T^{\prime}}(v)=\mathcal{L}_{T}(v_{T})\cap X\subseteq L_{T}(u_{T})\cap X=\mathcal{L}_{T^{\prime}}(u)$, which implies that $u$ is an ancestor of $v$. ∎

Lemma 9

For any $T^{\prime\prime},T^{\prime},T$ and $Y\subseteq X\subseteq\mathcal{X}(T)$ such that $T^{\prime}=T|_{X}$ and $T^{\prime\prime}=T^{\prime}|_{Y}$, $(u_{T^{\prime}})_{T}=u_{T}$.

Proof

It is sufficient to show that any node in $V(T)$ is a common ancestor of $\mathcal{L}_{T^{\prime}}(\textsc{lca}_{T^{\prime}}(Z))$ if and only if it is a common ancestor of $Z$, where $Z=\mathcal{L}_{T^{\prime\prime}}(u)$ (as this implies that the least common ancestors of these two sets are the same). It is clear that if $v\in V(T)$ is a common ancestor of $\mathcal{L}_{T^{\prime}}(\textsc{lca}_{T^{\prime}}(Z))$ then it is also a common ancestor of $Z$, as $Z\subseteq\mathcal{L}_{T^{\prime}}(\textsc{lca}_{T^{\prime}}(Z))$. For the converse, observe that as $T^{\prime}=T|_{X}$ and $Z\subseteq X$, any label in $X$ descended from $\textsc{lca}_{T^{\prime}}(Z)$ in $T^{\prime}$ is also descended from $\textsc{lca}_{T}(Z)$ in $T$. This implies $\mathcal{L}_{T^{\prime}}(\textsc{lca}_{T^{\prime}}(Z))\subseteq\mathcal{L}_{T}(\textsc{lca}_{T}(Z))$, and so any common ancestor of $Z$ in $T$ is also a common ancestor of $\mathcal{L}_{T^{\prime}}(\textsc{lca}_{T^{\prime}}(Z))$. ∎

We are now ready to state and prove Lemma 10

Lemma 10

Suppose that $d<d_{LR}(T_{1},T_{2})\leq d^{\prime}+d$ with $d^{\prime}\leq d$, and that there are $X_{1},X_{2}\subseteq\mathcal{X}$, and a tree $T^{*}$ such that $T_{1}-X_{1}=T^{*}-X_{1},T_{2}-X_{2}=T^{*}-X_{2},|X_{1}|\leq d^{\prime},|X_{2}|\leq d$, and let $x\in X_{1}\setminus X_{2}$. Then, there is a set $P$ of trees on label set $\mathcal{X}$ that satisfies the following conditions:

• •

for any tree $T^{\prime}$ such that $d_{LR}(T^{\prime},T^{*})<d_{LR}(T_{1},T^{*})$ and $T^{\prime}$ can be obtained from $T_{1}$ by pruning a leaf $x$ and regrafting it, $T^{\prime}\in P$;

• •

$|P|\leq 18(d+d^{\prime})+8$;

• •

$P$* can be found in time $O(n(\log n+18(d+d^{\prime})+8))$.*

The idea behind the proof is as follows: by looking at a subtree common to $T_{1}$ and $T_{2}$, we can identify the location that $T_{2}$ “wants” $x$ to be positioned. This may not be the correct position for $x$, but we can show that if $x$ is moved too far from this position, we will create a large number of conflicting triplets between $T_{2}$ and the solution $T^{*}$. As a result, we can create all trees in $P$ by removing $x$ from $T_{1}$ and grafting it on one of a limited number of edges.

Proof

For the purposes of this proof, we will treat each tree $T$ as “planted”, i.e. as having an additional root of degree $1$, denoted $r(T)$, as the parent of what would normally be considered the “root” of the tree. (That is, $r(T)$ is the parent of $\textsc{lca}_{T}(\mathcal{X}(T))$. Note that trees are otherwise binary. We introduce $r(T)$ as a notational convenience to avoid tedious repetition of proofs - grafting a label above a tree $T$ can instead be represented as grafting it on the edge between $r(T)$ and its child. For the purposes of corresponding nodes, if $T^{\prime}=T-X$ then $(r(T^{\prime}))_{T}=r(T)$. This allows us to assume that every node in $T$ is a descendant of $u_{T}$ for some node $u$ in $T^{\prime}$.

A naive method for constructing a tree in $P$ is the following: Apply an LPR move $(x,e)$ on $T_{1}$, such that $x$ is moved to a position that $T_{2}$ “wants” $x$ to be in. There are at least two problems with this method. The first is that, since $T_{1}$ and $T_{2}$ have different structures, it is not clear where in $T_{1}$ it is that $T_{2}$ “wants” $x$ to be. We can partially overcome this obstacle by initially considering a subtree common to both $T_{1}$ and $T_{2}$. However, because $T_{2}$ will want to move leafs that will not be moved in $T_{1}$, it can still be the case that even though $T_{2}$ “agrees” with $T^{*}$ on $x$, $T_{2}$ may want to put $x$ in the “wrong” place, when viewed from the perspective of $T_{1}$. For this reason we have to give a counting argument to show that if $x$ is moved “too far” from the position suggested by $T_{2}$, it will create too many conflicting triplets, which cannot be covered except by moving $x$. We make these ideas precise below.

Let $P^{*}$ be the set of all trees $T^{\prime}$ such that $d_{LR}(T^{\prime},T^{*})<d_{LR}(T_{1},T^{*})$ and $T^{\prime}$ can be obtained from $T_{1}$ by an LPR move on $x$. Thus, it is sufficient to construct a set $P$ such that $|P|\leq 18(d+d^{\prime})+8$ and $P^{*}\subseteq P$.

We first construct a set $X_{m}\subseteq\mathcal{X}$ such that $|X_{m}|\leq d+d^{\prime},x\in X_{m}$, and $T_{1}-X_{m}=T_{2}-X_{m}$. Note that the unknown set $(X_{1}\cup X_{2})$ satisfies these properties, as $T_{1}-(X_{1}\cup X_{2})=T^{*}-(X_{1}\cup X_{2})=T_{2}-(X_{1}\cup X_{2})$, and so such a set $X_{m}$ must exist. We can find $X_{m}$ in time $O(n\log n)$ by applying MAST on $(T_{1}-\{x\},T_{2}-\{x\})$ [6].

Now let $T_{m}$ be the tree with labelset $\mathcal{X}\setminus{X_{m}}$ such that $T_{m}=T_{1}-X_{m}=T_{2}-X_{m}$. Note that for any $T^{\prime}$ in $P^{*}$, we have that $T^{\prime}-\{x\}=T_{1}-\{x\}$ and therefore $T^{\prime}-X_{m}=T_{1}-X_{m}=T_{m}$.

Informally, we now have a clear notion of where $T_{2}$ “wants” $x$ to go, relative to $T_{m}$. There is a unique edge $e$ in $T_{m}$ such that grafting $x$ on $e$ will give the tree $T_{2}-(X_{m}\setminus\{x\})$. If we assume that this is the “correct” position to add $x$, then it only remains to add the remaining labels of $X_{m}$ back in a way that agrees with $T_{1}$ (we will describe how this can be done at the end of the proof). Unfortunately, grafting $x$ onto the obvious choice $e$ does not necessarily lead to a graph in $P^{*}$. This is due to the fact that $T_{2}$ can be “mistaken” about labels outside of $X_{m}$.

To address this, we have try grafting $x$ on other edges of $T_{m}$. There are too many edges to try them all. We therefore need the following claim, which allows us to limit the number of edges to try.

Claim: In $O(n)$ time, we can find $y\in V(T_{m})$ and $Z\subseteq V(T_{m}),|Z|\leq 4$, such that:

• •

For any $T^{\prime}$ in $P^{*}$, $x\in\mathcal{L}_{T^{\prime}}(y_{T^{\prime}})\setminus\bigcup_{z^{\prime}\in Z}\mathcal{L}_{T^{\prime}}(z^{\prime}_{T^{\prime}})$

• •

$|\mathcal{L}_{T_{m}}(y)\setminus\bigcup_{z^{\prime}\in Z}\mathcal{L}_{T_{m}}(z^{\prime})|\leq 8(d+d^{\prime})$**

Informally, the claim identifies a node $y$ and set of nodes $Z$ in $T_{m}$, such that $x$ should be added as a descendant of $y$ but not of any node in $Z$, and the number of such positions is bounded. Algorithm 3 describes the formal procedure to produce $y$ and $Z$. The proof of the claim takes up most of the remainder of our proof; the reader may wish to skip it on their first readthrough.

Proof

Let $T_{m}^{\prime}=T_{2}-(X_{m}\setminus\{x\})$. Note that $T_{m}^{\prime}-\{x\}=T_{m}$. We will use the presence of $x$ in $T_{m}^{\prime}$ to identify the node $y$ and set $Z$. (Technically, this means the nodes we find are nodes in $T_{m}^{\prime}$ rather than $T_{m}$. However, we note that apart the parent of $x$ and $x$ itself, neither of which will be added to $\{y\}\cup Z$, every node in $T_{m}^{\prime}$ is the node $v_{T_{m}^{\prime}}$ corresponding to some node $v$ in $T_{m}$. For the sake of clarity, we ignore the distinction and write $v$ to mean $v_{T_{m}^{\prime}}$ throughout this proof. The nodes in $\{y\}\cup Z$ should ultimately be replaced with the nodes in $T_{m}$ to which they correspond.)

We first identify two nodes $z,y$ of $T_{m}$ as follows:

• •

Let $z$ be the least ancestor of $x$ in $T_{m}^{\prime}$ such that $|\mathcal{L}_{T_{m}^{\prime}}(z)\setminus\{x\}|\geq d+d^{\prime}$. If no such $x$ exists, then $\mathcal{X}(T_{m}^{\prime})\leq d+d^{\prime}$ and we may return $y=r(T_{m}^{\prime}),Z=\emptyset$.

• •

Let $y$ be the least ancestor of $z$ in $T_{m}^{\prime}$ such that $|\mathcal{L}_{T_{m}^{\prime}}(y)\setminus\mathcal{L}_{T_{m}^{\prime}}(z)|\geq d+d^{\prime}$. If no such $y$ exists, set $y=r(T_{m}^{\prime})$.

Using this definition, we will show that $x$ must be a descendant of $y_{T^{\prime}}$ for any $T^{\prime}\in P^{*}$. We first describe a general tactic for restricting the position of $x$ in $T^{\prime}$, as this tactic will be used a number of times.

Suppose that for some $T^{\prime}\in P^{*}$ there is a set of $d+d^{\prime}$ triplets in $confset(T^{\prime},T_{2})$ whose only common element is $x$. Then let $X^{\prime}\subseteq\mathcal{X}$ be a set of labels such that $T^{\prime}-X^{\prime}=T^{*}-X^{\prime}$ and $|X^{\prime}|=d_{LR}(T^{\prime},T^{*})\leq d_{LR}(T_{1},T^{*})-1\leq d^{\prime}-1$. Note that $T_{2}-(X^{\prime}\cup X_{2})=T^{*}-(X^{\prime}\cup X_{2})=T^{\prime}-(X^{\prime}\cup X_{2})$, and therefore $(X^{\prime}\cup X_{2})$ is a hitting set for $confset(T^{\prime},T_{2})$. As $|X^{\prime}\cup X_{2}|\leq d+d-1$ and there are $d+d^{\prime}$ triplets in $confset(T^{\prime},T_{2})$ whose only common element is $x$, it must be the case that $x\in X^{\prime}\cup X_{2}$. As $x\notin X_{2}$, we must have $x\in X^{\prime}$. But this implies that $T_{1}-X^{\prime}=T^{\prime}-X^{\prime}=T^{*}-X^{\prime}$ and therefore $d_{LR}(T_{1},T^{*})\leq|X^{\prime}|=d_{LR}(T^{\prime},T^{*})\leq d_{LR}(T_{1},T^{*})-1$, a contradiction. Thus we may assume that such a set of triplets does not exist.

We now use this idea to show that $x\in\mathcal{L}_{T^{\prime}}(y_{T^{\prime}})$, for any $T^{\prime}\in P^{*}$. Indeed, suppose $x\notin\mathcal{L}_{T^{\prime}}(y_{T^{\prime}})$. We may assume $y\neq r(T_{m}^{\prime})$ as otherwise $y_{T^{\prime}}=r(T^{\prime})$ by definition and so $\mathcal{L}_{T^{\prime}}(y_{T^{\prime}})=\mathcal{X}(T^{\prime})$. Then let $z_{1},\ldots,z_{d+d^{\prime}}$ be $d+d^{\prime}$ labels in $\mathcal{L}_{T_{m}^{\prime}}(z)\setminus\{x\}$. Let $y_{1},\ldots,y_{d+d^{\prime}}$ be $d+d^{\prime}$ labels in $\mathcal{L}_{T_{m}^{\prime}}(y)\setminus\mathcal{L}_{T_{m}^{\prime}}(z)$. Observe that for each $i\in[d+d^{\prime}]$, $T_{m}^{\prime}$ (and therefore $T_{2}$) contains the triplet $(z_{i}x|y_{i})$, but $T^{\prime}$ contains the triplet $(z_{i}y_{i}|x)$. Therefore $confset(T^{\prime},T_{2})$ contains $d+d^{\prime}$ sets whose only common element is $x$. As this implies a contradiction, we must have $x\in\mathcal{L}_{T^{\prime}}(y_{T^{\prime}})$.

Note however that $|\mathcal{L}_{T_{m}^{\prime}}(y)|$ maybe be very large. In order to provide a bounded range of possible positions for $x$, we still need to find a set $Z$ of nodes such that $|\mathcal{L}_{T_{m}^{\prime}}(y)\setminus\bigcup_{z^{\prime}\in Z}\mathcal{L}_{T_{m}^{\prime}}(z^{\prime}))|$ is bounded, and such that we can show $x\notin\mathcal{L}_{T^{\prime}}(z^{\prime}_{T^{\prime}})$ for any $z^{\prime}\in Z$.

We now construct a set $Z_{1}$ of descendants of $z$ as follows:

• •

Let $z_{1},z_{2}$ be the children of $z$ in $T_{m}^{\prime}$ such that $x$ is descended from $z_{1}$.

• •

If $|\mathcal{L}_{T_{m}^{\prime}}(z_{2})|\leq 3(d+d^{\prime})$ then set $Z_{1}=\emptyset$.

• •

Otherwise, let $Z_{1}$ be the set of highest descendants $z^{\prime}$ of $z_{2}$, such that $|\mathcal{L}_{T_{m}^{\prime}}(z^{\prime})|\geq d+d^{\prime}$ and $|\mathcal{L}_{T_{m}^{\prime}}(z_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(z^{\prime})|\geq d+d^{\prime}$ (i.e. by highest descendant we mean such that $z^{\prime}$ has no ancestor $z^{\prime\prime}$ with the same properties).

Note that $|\mathcal{L}_{T_{m}^{\prime}}(z_{1})|\leq d+d^{\prime}$ by our choice of $z$. It follows that if $|\mathcal{L}_{T_{m}^{\prime}}(z_{2})|\leq 3(d+d^{\prime})$ then $|\mathcal{L}_{T_{m}^{\prime}}(z)|\leq 4(d+d^{\prime})$. If on the other hand $|\mathcal{L}_{T_{m}^{\prime}}(z_{2})|>3(d+d^{\prime})$ then $Z_{1}$ is non-empty. Indeed, let $z^{\prime}$ be a lowest descendant of $z_{2}$ with $|\mathcal{L}_{T_{m}^{\prime}}(z^{\prime})|\geq d+d^{\prime}$, and observe that $|\mathcal{L}_{T_{m}^{\prime}}(z^{\prime})|\leq 2(d+d^{\prime})$. Then either $z^{\prime}\in Z_{1}$, or $|\mathcal{L}_{T_{m}^{\prime}}(z_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(z^{\prime})|\leq d+d^{\prime}$, in which case $|\mathcal{L}_{T_{m}^{\prime}}(z_{2})|\leq|\mathcal{L}_{T_{m}^{\prime}}(z_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(z^{\prime})|+|\mathcal{L}_{T_{m}^{\prime}}(z^{\prime})|\leq d+d^{\prime}+2(d+d^{\prime})=3(d+d^{\prime})$.

We also have that $|Z_{1}|\leq 2$. Indeed, let $z_{1}^{\prime},z_{2}^{\prime},z_{3}^{\prime}$ be three distinct nodes in $Z_{1}$, and suppose without loss of generality that $(z_{1}^{\prime}z_{2}^{\prime}|z_{3}^{\prime})\in tr(T_{m}^{\prime})$. Then setting $z^{\prime}=\textsc{lca}_{T_{m}^{\prime}}(z_{1}^{\prime},z_{2}^{\prime})$, we have that $z^{\prime}$ is an ancestor of $z_{1}^{\prime}$ such that $|\mathcal{L}_{T_{m}^{\prime}}(z^{\prime})|\geq d+d^{\prime}$ and $|\mathcal{L}_{T_{m}^{\prime}}(z_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(z^{\prime})|\geq|\mathcal{L}_{T_{m}^{\prime}}(z_{3}^{\prime})|\geq d+d^{\prime}$, a contradiction by minimality of $z_{1}$.

We have that $|\mathcal{L}_{T_{m}^{\prime}}(z)\setminus\bigcup_{z^{\prime}\in Z_{1}}\mathcal{L}_{T_{m}^{\prime}}(z^{\prime}))|\leq 4(d+d^{\prime})$. Indeed, if $Z_{1}=\emptyset$ then $|\mathcal{L}_{T_{m}^{\prime}}(z)|\leq 4(d+d^{\prime})$ as described above. Otherwise, let $z^{\prime}$ be an element of $Z_{1}$ and $z_{p}$ its parent, $z_{s}$ its sibling in $T_{m}^{\prime}$. Clearly $|\mathcal{L}_{T_{m}^{\prime}}(z_{p})|\geq|\mathcal{L}_{T_{m}^{\prime}}(z^{\prime})|\geq d+d^{\prime}$, and so as $z_{p}\notin Z_{1}$ we have $|\mathcal{L}_{T_{m}^{\prime}}(z_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(z_{p})|<d+d^{\prime}$. If $|\mathcal{L}_{T_{m}^{\prime}}(z_{s})|\geq d+d^{\prime}$ then $z_{s}\in Z_{1}$ (since $|\mathcal{L}_{T_{m}^{\prime}}(z_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(z_{s})|\geq|\mathcal{L}_{T_{m}^{\prime}}(z^{\prime})|\geq d+d^{\prime}$), and so $|\mathcal{L}_{T_{m}^{\prime}}(z)\setminus\bigcup_{z^{\prime}\in Z_{1}}\mathcal{L}_{T_{m}^{\prime}}(z^{\prime}))|\leq|\mathcal{L}_{T_{m}^{\prime}}(z_{1})|+|\mathcal{L}_{T_{m}^{\prime}}(z_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(z_{p})|\leq 2(d+d^{\prime})$. Otherwise, $|\mathcal{L}_{T_{m}^{\prime}}(z)\setminus\bigcup_{z^{\prime}\in Z_{1}}\mathcal{L}_{T_{m}^{\prime}}(z^{\prime}))|\leq|\mathcal{L}_{T_{m}^{\prime}}(z_{1})|+|\mathcal{L}_{T_{m}^{\prime}}(z_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(z_{p})|+|\mathcal{L}_{T_{m}^{\prime}}(z_{s})|\leq 3(d+d^{\prime})$.

We have now shown that $|Z_{1}|\leq 2$ and that $|\mathcal{L}_{T_{m}^{\prime}}(z)\setminus\bigcup_{z^{\prime}\in Z_{1}}\mathcal{L}_{T_{m}^{\prime}}(z^{\prime}))|\leq 4(d+d^{\prime})$. The final property of $Z_{1}$ we wish to show is that for any $z^{\prime}\in Z_{1}$ and any $T^{\prime}\in P$, $x\notin\mathcal{L}_{T^{\prime}}(z^{\prime}_{T^{\prime}})$.

So suppose $x\in\mathcal{L}_{T^{\prime}}(z^{\prime}_{T^{\prime}})$. Let $\hat{z}_{1},\dots,\hat{z}_{d+d^{\prime}}$ be $d+d^{\prime}$ labels in $\mathcal{L}_{T_{m}^{\prime}}(z_{2})\setminus L_{T_{m}^{\prime}}(z^{\prime})$. Also, $z_{1}$ and $z_{2}$ were already taken. Let $w_{1},\dots,w_{d+d^{\prime}}$ be $d+d^{\prime}$ labels in $\mathcal{L}_{T_{m}^{\prime}}(z^{\prime})$. Then for each $i\in[d+d^{\prime}]$, $T_{m}^{\prime}$ (and therefore $T_{2}$) contains the triplet $(\hat{z}_{i}w_{i}|x)$, but $T^{\prime}$ contains the triplet $(xw_{i}|\hat{z}_{i})$. Therefore $confset(T^{\prime},T_{2})$ contains $d+d^{\prime}$ sets whose only common element is $x$. As this implies a contradiction, we must have $x\notin\mathcal{L}_{T^{\prime}}(z^{\prime}_{T^{\prime}})$.

We now define a set $Z_{2}$ of descendants of $y$:

• •

If $y=r(T_{m}^{\prime})$, set $Z_{2}=\emptyset$.

• •

Otherwise, let $y_{1},y_{2}$ be the children of $y$ in $T_{m}^{\prime}$ such that $z$ is descended from $y_{1}$.

• •

If $|\mathcal{L}_{T_{m}^{\prime}}(y_{2})|\leq 3(d+d^{\prime})$ then set $Z_{2}=\emptyset$.

• •

Otherwise, let $Z_{2}$ be the set of highest descendants $y^{\prime}$ of $y_{2}$, such that $|\mathcal{L}_{T_{m}^{\prime}}(y^{\prime})|\geq d+d^{\prime}$ and $|\mathcal{L}_{T_{m}^{\prime}}(y_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(y^{\prime})|\geq d+d^{\prime}$ (i.e. such that $y^{\prime}$ has no ancestor $y^{\prime\prime}$ with the same properties).

In a similar way to the proofs for $Z_{1}$, we can show that $|Z_{2}|\leq 2$, that $|(\mathcal{L}_{T_{m}^{\prime}}(y)\setminus\mathcal{L}_{T_{m}^{\prime}}(z))\setminus\bigcup_{y^{\prime}\in Z_{2}}\mathcal{L}_{T_{m}^{\prime}}(y^{\prime}))|\leq 4(d+d^{\prime})$, and that $x\notin\mathcal{L}_{T^{\prime}}(y^{\prime}_{T^{\prime}})$ for any $y^{\prime}\in Z_{2}$ and any $T^{\prime}\in P^{*}$.

Note that $|\mathcal{L}_{T_{m}^{\prime}}(y_{1})\setminus\mathcal{L}_{T_{m}^{\prime}}(z)|\leq d+d^{\prime}$ by our choice of $y$. It follows that if $|\mathcal{L}_{T_{m}^{\prime}}(y_{2})|\leq 3(d+d^{\prime})$ then $|\mathcal{L}_{T_{m}^{\prime}}(y)\setminus\mathcal{L}_{T_{m}^{\prime}}(z)|\leq 4(d+d^{\prime})$. If on the other hand $|\mathcal{L}_{T_{m}^{\prime}}(y_{2})|>3(d+d^{\prime})$, then $Z_{2}$ is non-empty. Indeed, let $y^{\prime}$ be a lowest descendant of $y_{2}$ with $|\mathcal{L}_{T_{m}^{\prime}}(y^{\prime})|\geq d+d^{\prime}$, and observe that $|\mathcal{L}_{T_{m}^{\prime}}(y^{\prime})|\leq 2(d+d^{\prime})$. Then either $y^{\prime}\in Z_{2}$, or $|\mathcal{L}_{T_{m}^{\prime}}(y_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(y^{\prime})|\leq d+d^{\prime}$, in which case $|\mathcal{L}_{T_{m}^{\prime}}(y_{2})|\leq|\mathcal{L}_{T_{m}^{\prime}}(y_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(y^{\prime})|+|\mathcal{L}_{T_{m}^{\prime}}(y^{\prime})|\leq d+d^{\prime}+2(d+d^{\prime})=3(d+d^{\prime})$.

We also have that $|Z_{2}|\leq 2$. Indeed, let $y_{1}^{\prime},y_{2}^{\prime},y_{3}^{\prime}$ be three distinct nodes in $Z_{2}$, and suppose without loss of generality that $(y_{1}^{\prime}y_{2}^{\prime}|y_{3}^{\prime})\in tr(T_{m}^{\prime})$. Then setting $y^{\prime}=\textsc{lca}_{T_{m}^{\prime}}(y_{1}^{\prime},y_{2}^{\prime})$, we have that $y^{\prime}$ is an ancestor of $y_{1}^{\prime}$ such that $|\mathcal{L}_{T_{m}^{\prime}}(y^{\prime})|\geq d+d^{\prime}$ and $|\mathcal{L}_{T_{m}^{\prime}}(y_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(y^{\prime})|\geq|\mathcal{L}_{T_{m}^{\prime}}(y_{3}^{\prime})|\geq d+d^{\prime}$, a contradiction by minimality of $y_{1}$.

We have that $|(\mathcal{L}_{T_{m}^{\prime}}(y)\setminus\mathcal{L}_{T_{m}^{\prime}}(z))\setminus\bigcup_{y^{\prime}\in Z_{2}}\mathcal{L}_{T_{m}^{\prime}}(y^{\prime}))|\leq 4(d+d^{\prime})$. Indeed, if $y=r(T_{m}^{\prime})$ then by construction $|\mathcal{L}_{T_{m}^{\prime}}(\hat{y})\setminus\mathcal{L}_{T_{m}^{\prime}}(z)|<d+d^{\prime}$ for any ancestor $\hat{y}$ of $z$ (noting that otherwise there would be no reason to set $y$ as $r(T_{m}^{\prime})$ rather than the child of $r(T_{m}^{\prime})$), and so in particular $|\mathcal{L}_{T_{m}^{\prime}}(y)\setminus\mathcal{L}_{T_{m}^{\prime}}(z)|<d+d^{\prime}$. If $y\neq r(T_{m}^{\prime})$ and $Z_{2}=\emptyset$ then $|\mathcal{L}_{T_{m}^{\prime}}(y)\setminus\mathcal{L}_{T_{m}^{\prime}}(z)|\leq 4(d+d^{\prime})$ as described above. Otherwise, let $y^{\prime}$ be an element of $Z_{2}$ and $y_{p}$ its parent, $y_{s}$ its sibling in $T_{m}^{\prime}$. Clearly $|\mathcal{L}_{T_{m}^{\prime}}(y_{p})|\geq|\mathcal{L}_{T_{m}^{\prime}}(y^{\prime})|\geq d+d^{\prime}$, and so as $y_{p}\notin Z_{2}$ we have $|\mathcal{L}_{T_{m}^{\prime}}(y_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(y_{p})|<d+d^{\prime}$. If $|\mathcal{L}_{T_{m}^{\prime}}(y_{s})|\geq d+d^{\prime}$ then $y_{s}\in Z_{2}$ (since $|\mathcal{L}_{T_{m}^{\prime}}(y_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(y_{s})|\geq|\mathcal{L}_{T_{m}^{\prime}}(y^{\prime})|\geq d+d^{\prime}$), and so $|(\mathcal{L}_{T_{m}^{\prime}}(y)\setminus\mathcal{L}_{T_{m}^{\prime}}(z))\setminus\bigcup_{y^{\prime}\in Z_{2}}\mathcal{L}_{T_{m}^{\prime}}(y^{\prime}))|\leq|\mathcal{L}_{T_{m}^{\prime}}(y_{1})\setminus\mathcal{L}_{T_{m}^{\prime}}(z)|+|\mathcal{L}_{T_{m}^{\prime}}(y_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(y_{p})|\leq 2(d+d^{\prime})$. Otherwise, $|(\mathcal{L}_{T_{m}^{\prime}}(y)\setminus\mathcal{L}_{T_{m}^{\prime}}(z))\setminus\bigcup_{y^{\prime}\in Z_{2}}\mathcal{L}_{T_{m}^{\prime}}(y^{\prime}))|\leq|\mathcal{L}_{T_{m}^{\prime}}(y_{1})\setminus\mathcal{L}_{T_{m}^{\prime}}(z)|+|\mathcal{L}_{T_{m}^{\prime}}(y_{2})\setminus\mathcal{L}_{T_{m}^{\prime}}(y_{p})|+|\mathcal{L}_{T_{m}^{\prime}}(y_{s})|\leq 3(d+d^{\prime})$.

We have now shown that $|Z_{2}|\leq 2$ and $|(\mathcal{L}_{T_{m}^{\prime}}(y)\setminus\mathcal{L}_{T_{m}^{\prime}}(z))\setminus\bigcup_{y^{\prime}\in Z_{2}}\mathcal{L}_{T_{m}^{\prime}}(y^{\prime}))|\leq 4(d+d^{\prime})$. The final property of $Z_{2}$ we wish to show is that for any $y^{\prime}\in Z_{2}$ and any $T^{\prime}\in P^{*}$, we have that $x\notin\mathcal{L}_{T^{\prime}}(y^{\prime}_{T^{\prime}})$.

So suppose $x\in\mathcal{L}_{T^{\prime}}(y^{\prime}_{T^{\prime}})$. Let $\hat{y}_{1},\dots,\hat{y}_{d+d^{\prime}}$ be $d+d^{\prime}$ labels in $\mathcal{L}_{T_{m}^{\prime}}(y_{2})\setminus L_{T_{m}^{\prime}}(y^{\prime})$. Let $w_{1},\dots,w_{d+d^{\prime}}$ be $d+d^{\prime}$ labels in $\mathcal{L}_{T_{m}^{\prime}}(y^{\prime})$. Then for each $i\in[d+d^{\prime}]$, $T_{m}^{\prime}$ (and therefore $T_{2}$) contains the triplet $(\hat{y}_{i}w_{i}|x)$, but $T^{\prime}$ contains the triplet $(xw_{i}|\hat{y}_{i})$. Therefore $confset(T^{\prime},T_{2})$ contains $d+d^{\prime}$ sets whose only common element is $x$. As this implies a contradiction, we must have $x\notin\mathcal{L}_{T^{\prime}}(y^{\prime}_{T^{\prime}})$.

Now that $Z_{1}$ and $Z_{2}$ have been constructed, let $Z=Z_{1}\cup Z_{2}$. Note that $|Z|\leq 4$. Algorithm 3 describes the construction of $y$ and $Z$ formally (see Figure 5).

We have shown above that for any $T^{\prime}\in P^{*}$, $x$ is descended from $y_{T^{\prime}}$ in $T^{\prime}$ and not from $z^{\prime}_{T^{\prime}}$ for any $z^{\prime}\in Z$, and so $x\in\mathcal{L}_{T^{\prime}}(y_{T^{\prime}})\setminus\bigcup_{z^{\prime}\in Z}\mathcal{L}_{T^{\prime}}(z^{\prime}_{T^{\prime}})$. As $|\mathcal{L}_{T_{m}^{\prime}}(z)\setminus\bigcup_{z^{\prime}\in Z_{1}}\mathcal{L}_{T_{m}^{\prime}}(z^{\prime}))|\leq 4(d+d^{\prime})$ and $|(\mathcal{L}_{T_{m}^{\prime}}(y)\setminus\mathcal{L}_{T_{m}^{\prime}}(z))\setminus\bigcup_{y^{\prime}\in Z_{2}}\mathcal{L}_{T_{m}^{\prime}}(y^{\prime}))|\leq 4(d+d^{\prime})$, we have $|\mathcal{L}_{T_{m}^{\prime}}(y)\setminus\bigcup_{z^{\prime}\in Z}\mathcal{L}_{T_{m}^{\prime}}(z^{\prime}))|\leq 8(d+d^{\prime})$.

To analyze the complexity, note that we can calculate the value of $|\mathcal{L}_{T_{m}^{\prime}}(u)|$ for all $u$ in $O(n)$ time using a depth-first search approach, together with the fact that $|\mathcal{L}_{T_{m}^{\prime}}(u)|=|\mathcal{L}_{T_{m}^{\prime}}(u_{1})|+|\mathcal{L}_{T_{m}^{\prime}}(u_{2})|$ for any node $u$ with children $u_{1},u_{2}$. Then we can find $z$ in $O(n)$ time, and once we have found $z$ we can find $y$, and thence $z_{1},z_{2},y_{1},y_{2}$, in $O(n)$ time. Similarly, once these nodes are found we can find the members of $Z$ in $O(n)$ time. ∎

Using the claim, we may now construct a set $P^{\prime}$ of $O\leq 16(d+d^{\prime})+8$ trees on $\mathcal{X}\setminus(X_{m}\setminus\{x\})$, such for any $T^{\prime}\in P^{*}$, $P^{\prime}$ contains the tree $T^{\prime\prime}=T^{\prime}-(X_{m}\setminus\{x\})$. Indeed, let $F$ be the set of arcs $uv$ in $T_{m}$ that exist on a path from $y$ to a node in $(\mathcal{L}_{T_{m}}(y)\setminus\bigcup_{z\in Z}\mathcal{L}_{T_{m}}(z^{\prime}))\cup Z$. As $|\mathcal{L}_{T_{m}}(y)\setminus\bigcup_{z^{\prime}\in Z}\mathcal{L}_{T_{m}}(z^{\prime})|\leq 8(d+d^{\prime})$, $|Z|\leq 4$ and $T_{m}$ is a binary tree, we have $|F|\leq 16(d+d^{\prime})+8$. For each $e\in F$, let $T_{e}$ be the tree obtained from $T_{m}$ by grafting $x$ onto the arc $e$ . Let $P^{\prime}=\{T_{e}:e\in F\}$.

Let $T^{\prime}$ be a tree in $P^{*}$ and consider $T^{\prime\prime}=T^{\prime}-(X_{m}\setminus\{x\})$. Note that $T^{\prime\prime}-\{x\}=T^{\prime}-X_{m}=T_{m}$. Therefore $u_{T^{\prime\prime}}$ is well-defined for every node $u\in V(T_{m})$, and every node in $T^{\prime\prime}$ is equal to $u_{T^{\prime\prime}}$ for some $u\in V(T_{m})$, except for $x$ and its parent in $T^{\prime\prime}$. So let $w$ be the parent of $x$ in $T^{\prime\prime}$, $u_{T^{\prime\prime}}$ the parent of $w$ in $T^{\prime\prime}$, and $v_{T^{\prime\prime}}$ the child of $w$ in $T^{\prime\prime}$ that is not $x$. Observe that $T^{\prime\prime}$ can be obtained from $T_{m}$ by grafting $x$ onto the arc $uv$. Then it is enough to show that $uv\in F$.

To see that $uv\in F$, first note that for each $z^{\prime}\in\{y\}\cup Z$, $z^{\prime}_{T^{\prime\prime}}$ is well-defined and $(z^{\prime}_{T^{\prime\prime}})_{T^{\prime}}=z^{\prime}_{T^{\prime}}$ (see Lemma 9). Then as $x$ is descended from $(y_{T^{\prime\prime}})_{T^{\prime}}=y_{T^{\prime}}$ in $T^{\prime}$, $x$ is descended from $y_{T^{\prime\prime}}$ in $T^{\prime\prime}$ (Lemma 8). Similarly, as $x$ is not descended from $(z^{\prime}_{T^{\prime\prime}})_{T^{\prime}}=z^{\prime}_{T^{\prime}}$ in $T^{\prime}$ for any $z^{\prime}\in Z$, $x$ is not descended from $z_{T^{\prime\prime}}$ in $T^{\prime\prime}$. Thus $x\in\mathcal{L}_{T^{\prime\prime}}(y_{T^{\prime\prime}})\setminus\bigcup_{z^{\prime}\in Z}\mathcal{L}_{T^{\prime\prime}}(z^{\prime}_{T^{\prime\prime}})$. It follows that $u_{T^{\prime\prime}}$ is a descendant of $y_{T^{\prime\prime}}$ in $T^{\prime\prime}$ (note that $y_{T^{\prime\prime}}\neq w$, as $w$ is not the least common ancestor of any set of labels in $\mathcal{X}(T_{m})$). Also, $v_{T^{\prime\prime}}$ is not a descendant of $z_{T^{\prime\prime}}$ for any $z^{\prime}\in Z$, unless $v_{T^{\prime\prime}}\in\bigcup_{z\in Z}z_{T^{\prime\prime}}$, as otherwise $x$ would be a descendant of such a $z_{T^{\prime\prime}}$. Thus, $v_{T^{\prime\prime}}$ is either a member or an ancestor of $(\mathcal{L}_{T^{\prime\prime}}(y_{T^{\prime\prime}})\setminus\bigcup_{z^{\prime}\in Z}\mathcal{L}_{T^{\prime\prime}}(z^{\prime}_{T^{\prime\prime}})))\cup\bigcup_{z^{\prime}\in Z}z^{\prime}_{T^{\prime\prime}}$. It follows using Lemma 8 that $u$ is a descendant in $T_{m}$ of $y$, and $v$ is an ancestor of $(\mathcal{L}_{T_{m}}(y)\setminus\bigcup_{z^{\prime}\in Z}\mathcal{L}_{T_{m}}(z^{\prime})))\cup\bigcup_{z^{\prime}\in Z}z^{\prime}$. Then $uv\in F$, as required.

Now that we have constructed our set $P^{\prime}$, it remains to find, for each $T_{e}\in P^{\prime}$, every tree $T^{\prime}$ on $\mathcal{X}$ such that $T^{\prime}-(X_{m}\setminus\{x\})=T_{e}$ and $T^{\prime}-\{x\}=T_{1}-\{x\}$. This will give us our set $P$, as for every $T^{\prime}\in P^{*}$, $T^{\prime}-(X_{m}\setminus\{x\})$ is a tree $T_{e}$ in $P^{\prime}$, and $T^{\prime}-\{x\}=T_{1}-\{x\}$.

Let $e=uv$, where $u,v\in V(T_{m})$, and let $T_{1e}$ be the subtree of $T_{1}-\{x\}$ whose root is $v$, and has as its label set $v$ together with all labels in $X_{m}\setminus\{x\}$ descended from $u$ but not $v$. Then we have to try every way of adding $x$ into this tree. If $T_{1e}$ contains $t$ labels from $X_{m}$, then there are $2t-1$ places to try adding $x$. Therefore $P$ will have at most $2|X_{m}|\leq 2(d+d^{\prime})$ additional trees compared to $P^{\prime}$, and so $|P|\leq 18(d+d^{\prime})+8$. Algorithm 4 gives the full procedure to construct $P$.

To analyze the complexity, recall that we find $X_{m}$, and therefore construct $T_{m}$ and $T_{m}^{\prime}$, in $O(n\log n)$ time. As shown above, we can find the node $y$ and set $Z$ in $O(n)$ time. Given $y$ and $Z$, the set of arcs $F$ can be found in $O(n)$ time using a depth-first search approach. For each $e\in F$ it takes $O(n)$ time to construct $T_{e}$, and so the construction of $P^{\prime}$ takes $O(|F|n)=O((16(d+d^{\prime})+8)n)$ time. Finally, the construction of of $P$ from $P^{\prime}$ takes $O(|P|n)=O((18(d+d^{\prime})+8)n)$ time. Putting it all together, we have that the construction of $P$ takes $O(n(\log n+18(d+d^{\prime})+8))$ time. ∎

We will call the set of trees $P$ described in Lemma 10 the set of candidate trees for $(T_{1},T_{2},x)$.

We are finally ready to give the proof of Theorem 4.1

Theorem 4.1 * (restated). AST-LR-d can be solved in time $O(c^{d}d^{3d}(n^{3}+tn\log n))$, where $c$ is a constant not depending on $d$ or $n$. *

Proof

The outline for our algorithm is as follows. We employ a branch-and-bound algorithm, in which at each step we attempt to modify the input tree $T_{1}$ to become close to a solution. We keep track of an integer $d^{\prime}$, representing the maximum length of an LPR sequence between $T_{1}$ and a solution. Initially set $d^{\prime}=d$. At each step, if $d_{LR}(T_{1},T_{i})\leq d$ for each $T_{i}\in\mathcal{T}$ then $T_{1}$ is a solutioon, and we are done. Otherwise, there must exist sime $T_{i}$ for which $d_{LR}(T_{1},T_{i})\geq d+d^{\prime}$. In this case, we calculate the $(d+d^{\prime})$ disagreement kernel $S$ between $T_{1}$ and $T_{i}$ (using the procedure of Lemma 7), and for each $x\in S$, attempt to construct a set $P$ of trees as in Lemma 10. For each $T^{\prime}\in P$, we try replacing $T_{1}$ with $T^{\prime}$, reducing $d^{\prime}$ by $1$, and repeating the procedure. Algorithm 5 describes the full procedure formally.

We claim that Algorithm 5 is a correct algorithm for AST-LR-d, and runs in time $O(c^{d}d^{3d}(n^{2}+tn\log n))$, for some constant $c$ not depending on $n$ or $d$.

First notice that if, in a leaf node of the branch tree created by Algorithm 5, a tree $T^{*}$ is returned, this occurs at line 4 in which case it has been verified that $T^{*}$ is indeed a solution. As an internal node of the branch tree returns a tree if and only if a child recursive call also returns a tree (the for loop on line 10), this shows that when the algorithm outputs a tree $T^{*}$, it is indeed a solution.

We next show that if a solution exists, then Algorithm 5 will return one. Suppose that $\mathcal{T}$ admits a solution, and let $T^{*}$ be a solution that minimizes $d_{1}:=d_{LR}(T_{1},T^{*})$, with $d_{1}\leq d^{\prime}$. We show that one leaf of the branch tree created by the algorithm returns $T^{*}$ (and thus the root of the branch tree also returns a solution, albeit not necessarily $T^{*}$). This is done by proving that in one of the recursive calls made to mastrl-distance on line 14, the tree $T^{\prime}$ obtained from $T_{1}$ satisfies $d_{LR}(T^{\prime},T^{*})=d_{1}-1$. By applying this argument inductively, this shows that the algorithm will find $T^{*}$ at some node of depth $d_{1}$ in the branch tree of the algorithm.

First notice that since $d_{LR}$ is a metric, for each $T_{i}\in\mathcal{T}$, $d_{LR}(T_{1},T_{i})\leq d_{LR}(T_{1},T^{*})+d_{LR}(T^{*},T_{i})\leq d^{\prime}+d$, and so the algorithm will not return $FALSE$ on line 6.

If $T_{1}$ isn’t a solution, then there is a tree of $\mathcal{T}$, say $T_{2}$ w.l.o.g., such that $d_{LR}(T_{1},T_{2})>d$. Notice that in this case, all the conditions of Lemma 6 are satisfied, i.e. $d_{LR}(T_{1},T_{2})>d$, and there are sets $X_{1},X_{2}\subseteq\mathcal{X}$ both of size at most $d$ such that $T_{1}-X_{1}=T^{*}-X_{1}$, $T_{2}-X_{2}=T^{*}-X_{2}$. Thus there is a minimal disagreement $X$ between $T_{1}$ and $T_{2}$, $|X|\leq d^{\prime}+d$, and $x\in X$ such that $x\in X_{1}\setminus X_{2}$. By Lemma 3, there is an LPR sequence $L=(x_{1},\ldots,x_{k})$ turning $T_{1}$ into $T^{*}$, where $\{x_{1},\ldots,x_{k}\}=X_{1}$. As $x\in X_{1}$, by Lemma 4, the leaves appearing in $L$ can be reordered, and we may assume that $x=x_{1}$. Finally by Lemma 10, if $T^{\prime}$ satisfies $d_{LR}(T^{\prime},T^{*})\leq d_{1}-1$ and $T^{\prime}$ can be obtained from $T_{1}$ by an LPR move on $x$, then $T^{\prime}\in P$. As we are making one recursive call to mastrl-distancefor each tree in $P$, this proves that one such call replaces $T_{1}$ by $T^{\prime}$ such that $d_{LR}(T^{\prime},T^{*})=d_{1}-1$.

As for the complexity, recall from Lemma 7 that the $(d+d^{\prime})$-disagreement kernel $S$ computed in line 8 contains at most $8d^{2}$ labels.Therefore when Algorithm 5 enters the ’for’ loop of line 9, it branches into at most $8d^{2}$ cases, one for each $x\in S$. Within each of these cases, the algorithm enters at most $|P|$ recursive calls, each of which decrements $d^{\prime}$. As $|P|\leq 18(d+d^{\prime})+8\leq 36d+8$ by Lemma 10, a single call of the algorithm splits into at most $8d^{2}(36d+8)=O(d^{3})$, each of which decrements $d^{\prime}$. Therefore, the branching tree created by the algorithm has degree at most $cd^{3}$ (for some constant $c$) and depth at most $d$, and so $O(c^{d}d^{3d})$ cases are considered.

As $d_{LR}(T_{1},T_{i})$ can be calculated in $O(n\log n)$ time for each $T_{i}$, a single call of lines 2-5 of the algorithm takes $O(tn\log n)$ time. A single call of lines 6-8 takes $O(n^{2})$ time by Lemma 10. Thus the total time for all calls of lines 2-8 is $O(c^{d}d^{3d}n(n^{2}+t\log n)$. Each call of line 10 occurs just before a recursive call to the algorithm, as so line 10 is called at most $O(c^{d}d^{3d})$ times. A single call of line 10 takes $O(n(\log n+18(d+d^{\prime})+8))=O(n(\log n+36d))$ time by Lemma 10, and so the total time for all calls of line 10 is $O(c^{d}d^{3d}n(\log n+36d))$. Thus in total, we have that the running time of the algorithm is $O(c^{d}d^{3d}(n^{2}+n(t\log n+36d))$. As we may assume $d\leq n$, this simplifies to $O(c^{d}d^{3d}(n^{2}+tn\log n))$. ∎