Phase Transitions in the Coloring of Random Graphs

Lenka Zdeborov\'a; Florent Krzakala

arXiv:0704.1269·cond-mat.dis-nn·November 9, 2011

Phase Transitions in the Coloring of Random Graphs

Lenka Zdeborov\'a, Florent Krzakala

PDF

TL;DR

This paper analyzes the phase transitions in the solution space of graph coloring problems on random graphs, revealing critical thresholds and their implications for computational hardness and algorithms.

Contribution

It provides a systematic analytical study of the solution space structure and phase transitions in graph coloring, using the cavity method, and discusses algorithmic implications.

Findings

01

Identification of clustering, condensation, and freezing transitions in solution space.

02

Critical connectivities for phase transitions in Erdős-Rényi and regular graphs.

03

Freezing transition may be linked to computational hardness.

Abstract

We consider the problem of coloring the vertices of a large sparse random graph with a given number of colors so that no adjacent vertices have the same color. Using the cavity method, we present a detailed and systematic analytical study of the space of proper colorings (solutions). We show that for a fixed number of colors and as the average vertex degree (number of constraints) increases, the set of solutions undergoes several phase transitions similar to those observed in the mean field theory of glasses. First, at the clustering transition, the entropically dominant part of the phase space decomposes into an exponential number of pure states so that beyond this transition a uniform sampling of solutions becomes hard. Afterward, the space of solutions condenses over a finite number of the largest states and consequently the total entropy of solutions becomes smaller than the…

Figures26

Click any figure to enlarge with its caption.

Tables2

Table 1. Table 1: Left: The transition thresholds for regular random graphs: c SP subscript 𝑐 SP c_{\rm SP} is the smallest connectivity with a nontrivial solution at m = 0 𝑚 0 m=0 ; the clustering threshold c d subscript 𝑐 𝑑 c_{d} is the smallest connectivity with a nontrivial solution at m = 1 𝑚 1 m=1 ; the rigidity threshold c r subscript 𝑐 𝑟 c_{r} is the smallest connectivity at which hard fields are present in the dominant states, the condensation c c subscript 𝑐 𝑐 c_{c} is the smallest connectivity for which the complexity at m = 1 𝑚 1 m=1 is negative and c s subscript 𝑐 𝑠 c_{s} the smallest UNCOL connectivity. Note that 3 − limit-from 3 3- coloring of 5 − limit-from 5 5- regular graphs is exactly critical for that c d = 5 + subscript 𝑐 𝑑 superscript 5 c_{d}=5^{+} . The rigidity transition may not exist due to the discreteness of the connectivities. Right: Values of m ∗ superscript 𝑚 m^{*} (corresponding to the dominating clusters), and in the range of [ − ∞ , m r ] subscript 𝑚 𝑟 [-\infty,m_{r}] the hard-field solution exists, in the range [ m s , ∞ ] subscript 𝑚 𝑠 [m_{s},\infty] the soft-field solution exists.

q	$c_{S P}$ ColoringFlo	$c_{d}$ MM05	$c_{r}$	$c_{c}$	$c_{s}$ ColoringFlo
3	5	$5^{+}$	-	6	6
4	9	9	-	10	10
5	13	14	14	14	15
6	17	18	19	19	20
7	21	23	-	25	25
8	26	29	30	31	31
9	31	34	36	37	37
10	36	39	42	43	44
20	91	101	105	116	117

Table 2. Table 2: Critical connectivities c d subscript 𝑐 𝑑 c_{d} (dynamical, clustering), c r subscript 𝑐 𝑟 c_{r} (rigidity, rearrangments), c c subscript 𝑐 𝑐 c_{c} (condensation, Kauzmann) and c s subscript 𝑐 𝑠 c_{s} (COL/UNCOL) for the phase transitions in the coloring problem on Erdős-Rényi graphs. The connectivities c S P subscript 𝑐 𝑆 𝑃 c_{SP} (where the first non trivial solution of SP appears) and c r ( m = 1 ) subscript 𝑐 𝑟 𝑚 1 c_{r(m=1)} (where hard fields appear at m = 1 𝑚 1 m=1 ) are also given. The error bars consist of the numerical precision on evaluation of the critical connectivities by the population dynamics technique, details are given in appendix D .

q	$c_{d}$	$c_{r}$	$c_{c}$	$c_{s}$	$c_{SP}$	$c_{r (m = 1)}$
3	4	4.66(1)	4	4.687(2)	4.42(1)	4.911
4	8.353(3)	8.83(2)	8.46(1)	8.901(2)	8.09(1)	9.267
5	12.837(3)	13.55(2)	13.23(1)	13.669(2)	12.11(2)	14.036
6	17.645(5)	18.68(2)	18.44(1)	18.880(2)	16.42(2)	19.112
7	22.705(5)	24.16(2)	24.01(1)	24.455(5)	20.97(2)	24.435
8	27.95(5)	29.93(3)	29.90(1)	30.335(5)	25.71(2)	29.960
9	33.45(5)	35.658	36.08(5)	36.490(5)	30.62(2)	35.658
10	39.0(1)	41.508	42.50(5)	42.93(1)	35.69(3)	41.508

Equations201

H ({s}) = (i, j) \in E \sum δ (s_{i}, s_{j}) .

H ({s}) = (i, j) \in E \sum δ (s_{i}, s_{j}) .

μ ({s}) = \frac{1}{Z _{0}} e^{- β H ({s})},

μ ({s}) = \frac{1}{Z _{0}} e^{- β H ({s})},

Q_{1} (k) = \frac{( k + 1 ) Q ( k + 1 )}{c} .

Q_{1} (k) = \frac{( k + 1 ) Q ( k + 1 )}{c} .

ψ_{s_{i}}^{i \to j} = \frac{1}{Z _{0}^{i \to j}} k \in i - j \prod s_{k} \sum e^{- β δ_{s_{i} s_{k}}} ψ_{s_{k}}^{k \to i} = \frac{1}{Z _{0}^{i \to j}} k \in i - j \prod (1 - (1 - e^{- β}) ψ_{s_{i}}^{k \to i}) .

ψ_{s_{i}}^{i \to j} = \frac{1}{Z _{0}^{i \to j}} k \in i - j \prod s_{k} \sum e^{- β δ_{s_{i} s_{k}}} ψ_{s_{k}}^{k \to i} = \frac{1}{Z _{0}^{i \to j}} k \in i - j \prod (1 - (1 - e^{- β}) ψ_{s_{i}}^{k \to i}) .

Z_{0}^{i \to j} = e^{- β Δ F^{i \to j}} .

Z_{0}^{i \to j} = e^{- β Δ F^{i \to j}} .

e^{- β Δ F^{i}} = Z_{0}^{i} = s \sum k \in i \prod (1 - (1 - e^{- β}) ψ_{s}^{k \to i}) .

e^{- β Δ F^{i}} = Z_{0}^{i} = s \sum k \in i \prod (1 - (1 - e^{- β}) ψ_{s}^{k \to i}) .

ψ_{s_{i}}^{i} = \frac{1}{Z _{0}^{i}} k \in i \prod (1 - (1 - e^{- β}) ψ_{s_{i}}^{k \to i}) .

ψ_{s_{i}}^{i} = \frac{1}{Z _{0}^{i}} k \in i \prod (1 - (1 - e^{- β}) ψ_{s_{i}}^{k \to i}) .

e^{- β Δ F^{ij}} = Z_{0}^{ij} = 1 - (1 - e^{- β}) s \sum ψ_{s}^{j \to i} ψ_{s}^{i \to j} .

e^{- β Δ F^{ij}} = Z_{0}^{ij} = 1 - (1 - e^{- β}) s \sum ψ_{s}^{j \to i} ψ_{s}^{i \to j} .

f (β) = \frac{1}{N} (i \sum Δ F^{i} - ij \sum Δ F^{ij}) .

f (β) = \frac{1}{N} (i \sum Δ F^{i} - ij \sum Δ F^{ij}) .

- β f (β) = - β e + s (e),

- β f (β) = - β e + s (e),

e^{- β h_{s_{i}}^{i \to j}} \equiv Z_{0}^{i \to j} ψ_{s_{i}}^{i \to j} \equiv k \in i - j \prod e^{- β u_{s_{i}}^{k \to i}} .

e^{- β h_{s_{i}}^{i \to j}} \equiv Z_{0}^{i \to j} ψ_{s_{i}}^{i \to j} \equiv k \in i - j \prod e^{- β u_{s_{i}}^{k \to i}} .

P (ψ) = k \sum Q_{1} (k) i = 1 \prod k - 1 \int d ψ^{i} P (ψ^{i}) δ (ψ - F ({ψ^{i}})),

P (ψ) = k \sum Q_{1} (k) i = 1 \prod k - 1 \int d ψ^{i} P (ψ^{i}) δ (ψ - F ({ψ^{i}})),

- β f_{RS} = lo g q + \frac{c}{2} lo g (1 - \frac{1 - e ^{- β}}{q}) .

- β f_{RS} = lo g q + \frac{c}{2} lo g (1 - \frac{1 - e ^{- β}}{q}) .

s_{RS} = lo g q + \frac{c}{2} lo g (1 - \frac{1}{q}) .

s_{RS} = lo g q + \frac{c}{2} lo g (1 - \frac{1}{q}) .

E [{s_{l}}, {s_{l}^{'}} sup s_{i} = 1 \sum q ∣ μ (s_{i} ∣ {s_{l}}) - μ (s_{i} ∣ {s_{l}^{'}}) ∣] \to_{l \to \infty} 0,

E [{s_{l}}, {s_{l}^{'}} sup s_{i} = 1 \sum q ∣ μ (s_{i} ∣ {s_{l}}) - μ (s_{i} ∣ {s_{l}^{'}}) ∣] \to_{l \to \infty} 0,

E {s_{l}} \sum μ ({s_{l}}) s_{i} = 1 \sum q ∣ μ (s_{i} ∣ {s_{l}}) - μ (s_{i}) ∣ \to_{l \to \infty} 0 .

E {s_{l}} \sum μ ({s_{l}}) s_{i} = 1 \sum q ∣ μ (s_{i} ∣ {s_{l}}) - μ (s_{i}) ∣ \to_{l \to \infty} 0 .

χ_{SG} = \frac{1}{N} i, j \sum ⟨ s_{i} s_{j} ⟩_{c}^{2} .

χ_{SG} = \frac{1}{N} i, j \sum ⟨ s_{i} s_{j} ⟩_{c}^{2} .

c_{RS}^{reg} = q^{2} - 2 q + 2, c_{RS}^{ER} = q^{2} - 2 q + 1,

c_{RS}^{reg} = q^{2} - 2 q + 2, c_{RS}^{ER} = q^{2} - 2 q + 1,

\tilde{μ} ({ψ}) = \frac{Z _{0} ({ ψ } ) ^{m}}{Z _{1}} = \frac{1}{Z _{1}} e^{- β m N f ({ψ})},

\tilde{μ} ({ψ}) = \frac{Z _{0} ({ ψ } ) ^{m}}{Z _{1}} = \frac{1}{Z _{1}} e^{- β m N f ({ψ})},

P^{i \to j} (ψ_{s_{i}}^{i \to j})

P^{i \to j} (ψ_{s_{i}}^{i \to j})

Φ (β, m) \equiv - \frac{1}{β m N} lo g (Z_{1}) = \frac{1}{N} (i \sum Δ Φ^{i} - ij \sum Δ Φ^{ij}),

Φ (β, m) \equiv - \frac{1}{β m N} lo g (Z_{1}) = \frac{1}{N} (i \sum Δ Φ^{i} - ij \sum Δ Φ^{ij}),

e^{- β m Δ Φ^{i}} = \int_{POP} e^{- β m Δ F^{i}}, e^{- β m Δ Φ^{ij}} = \int_{POP} e^{- β m Δ F^{ij}} .

e^{- β m Δ Φ^{i}} = \int_{POP} e^{- β m Δ F^{i}}, e^{- β m Δ Φ^{ij}} = \int_{POP} e^{- β m Δ F^{ij}} .

Z_{1} = e^{- β m N Φ (β, m)} = {ψ} \sum e^{- β m N f ({ψ})} = \int_{f} d f e^{- N [β m f (β) - Σ (f)]},

Z_{1} = e^{- β m N Φ (β, m)} = {ψ} \sum e^{- β m N f ({ψ})} = \int_{f} d f e^{- N [β m f (β) - Σ (f)]},

- β m Φ (β, m) = - β m f (β) + Σ (f) .

- β m Φ (β, m) = - β m f (β) + Σ (f) .

Σ = β m^{2} \partial_{m} Φ (β, m), f = \partial_{m} [m Φ (β, m)], β m = \partial_{f} Σ (f) .

Σ = β m^{2} \partial_{m} Φ (β, m), f = \partial_{m} [m Φ (β, m)], β m = \partial_{f} Σ (f) .

f (β) = i \sum \frac{\int _{POP} Δ F ^{i} e ^{- β m Δ F^{i}}}{\int _{POP} e ^{- β m Δ F^{i}}} - ij \sum \frac{\int _{POP} Δ F ^{ij} e ^{- β m Δ F^{ij}}}{\int _{POP} e ^{- β m Δ F^{ij}}} .

f (β) = i \sum \frac{\int _{POP} Δ F ^{i} e ^{- β m Δ F^{i}}}{\int _{POP} e ^{- β m Δ F^{i}}} - ij \sum \frac{\int _{POP} Δ F ^{ij} e ^{- β m Δ F^{ij}}}{\int _{POP} e ^{- β m Δ F^{ij}}} .

Φ (β, 1) = e - \frac{Σ + s}{β} = e - T s_{tot},

Φ (β, 1) = e - \frac{Σ + s}{β} = e - T s_{tot},

P [P (ψ)] = k \sum Q_{1} (k) i = 1 \prod k - 1 \int d P^{i} (ψ^{i}) P [P^{i} (ψ^{i})] δ (P (ψ) - F_{2} ({P^{i} (ψ^{i})})),

P [P (ψ)] = k \sum Q_{1} (k) i = 1 \prod k - 1 \int d P^{i} (ψ^{i}) P [P^{i} (ψ^{i})] δ (P (ψ) - F_{2} ({P^{i} (ψ^{i})})),

- y Φ_{e} (y) = - y e + Σ (e) .

- y Φ_{e} (y) = - y e + Σ (e) .

Φ_{s} (m) = m s + Σ (s) .

Φ_{s} (m) = m s + Σ (s) .

Peer Reviews

No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.

Videos

No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.

Full text

Phase Transitions in the Coloring of Random Graphs

Lenka Zdeborová1 and Florent Krza̧kała2

1 LPTMS, UMR 8626 CNRS et Univ. Paris-Sud, 91405 Orsay CEDEX, France

2 PCT, UMR Gulliver 7083 CNRS-ESPCI, 10 rue Vauquelin, 75231 Paris, France

Abstract

We consider the problem of coloring the vertices of a large sparse random graph with a given number of colors so that no adjacent vertices have the same color. Using the cavity method, we present a detailed and systematic analytical study of the space of proper colorings (solutions).

We show that for a fixed number of colors and as the average vertex degree (number of constraints) increases, the set of solutions undergoes several phase transitions similar to those observed in the mean field theory of glasses. First, at the clustering transition, the entropically dominant part of the phase space decomposes into an exponential number of pure states so that beyond this transition a uniform sampling of solutions becomes hard. Afterward, the space of solutions condenses over a finite number of the largest states and consequently the total entropy of solutions becomes smaller than the annealed one. Another transition takes place when in all the entropically dominant states a finite fraction of nodes freezes so that each of these nodes is allowed a single color in all the solutions inside the state. Eventually, above the coloring threshold, no more solutions are available. We compute all the critical connectivities for Erdős-Rényi and regular random graphs and determine their asymptotic values for large number of colors.

Finally, we discuss the algorithmic consequences of our findings. We argue that the onset of computational hardness is not associated with the clustering transition and we suggest instead that the freezing transition might be the relevant phenomenon. We also discuss the performance of a simple local Walk-COL algorithm and of the belief propagation algorithm in the light of our results.

pacs:

89.20.Ff, 75.10.Nr, 05.70.Fh, 02.70.-c

I Introduction

Graph coloring is a famous yet basic problem in combinatorics. Given a graph and $q$ colors, the problem consists in coloring the vertices in such a way that no connected vertices have the same color GaJo . The celebrated four-colors theorem assures that this is always possible for planar graphs using only four colors 4colors . For general graphs, however, the problem can be extremely hard to solve and is known to be NP-complete NPcol , so that it is widely believed that no algorithm can decide in a polynomial time (with respect to the size of the graph) if a given arbitrary instance is colorable or not. Indeed, the problem is often taken as a benchmark for the evaluation of the performance of algorithms in computer science. It has also important practical application as timetabling, scheduling, register allocation in compilers or frequency assignment in mobile radios.

In this paper, we study colorings of sparse random graphs Bo ; Janson . Random graphs are one of the most fundamental source of challenging problems in graph theory since the seminal work of Erdős and Rényi Erdos in 1959. Concerning the coloring problem, a crucial observation was made by focusing on typical instances drawn from the ensemble of random graphs with a given average vertex connectivity $c$ , as $c$ increases a threshold phenomenon is observed. Bellow a critical value $c_{s}$ a proper coloring of the graph with $q$ colors exists with a probability going to one in the large size limit, while beyond $c_{s}$ it does not exist in the same sense. This sharp transition also appears in other Constraint Satisfaction Problems (CSPs) such as the satisfiability of Boolean formulae GaJo . The existence of the sharp COLorable/UNCOLorable (COL/UNCOL) transition was partially 111More precisely Friedgut proves the existence of a sharp threshold for a possibly size dependent sequence of thresholds, whose convergence is not proven. proven in Friedgut , and computing rigorously its precise location is a major open problem in graph theory. Many upper and lower bounds were established Lu ; Nature ; DuboisMandler ; AchlioptasMoore03 ; AchlioptasRegular ; Wormlad1 ; Wormlad2 ; Diaz for Erdős-Rényi and regular random graphs.

It was also observed empirically CKT91 ; Selman that deciding colorability becomes on average much harder near to the coloring threshold $c_{s}$ than far away from it. This onset of computational hardness cannot be explained only by the simple fact that near to the colorable threshold the number of proper colorings is small Hardness . Some progress in the theoretical understanding has been done by the analysis of search algorithms Hogg ; Cocco . For the coloring problem, it was proven AcMo that a simple algorithm $q$ -colors almost surely in linear time random graphs of average connectivity $c\leq q\log{q}-3q/2$ for all $q\geq 3$ (see AcMo for references on previous works). For 3-coloring the best algorithmic lower bound is $c=4.03$ AchlioptasMoore03 . An important and interesting open question FrMcDi is the existence of an $\epsilon>0$ and of a polynomial algorithm which $q$ -colors almost surely a random graph of connectivity $c=(1+\varepsilon)q\log q$ for arbitrary large $q$ .

The sharp coloring threshold and the onset of hardness in its vicinity has also triggered a lot of interest within the statistical physics community following the discovery of a close relation between constraint satisfaction problems and spin glasses NatureMonasson ; Science . In physical terms, solving a CSP consists in finding groundstates of zero energy. The limit of infinitely large graphs corresponds to the thermodynamic limit. In the case of the $q-$ coloring problem, for instance, one studies the zero temperature behavior of the anti-ferromagnetic Potts model WU . Using this correspondence, a powerful heuristic tool called the cavity method has been developed Science ; MP99 ; MP03 ; MeZe ; it allows an exact analytical study of the CSPs on sparse random graphs. Unfortunately, some pieces are still missing to make the cavity method a rigorous tool although many of its results were rigorously proven. The cavity method is equivalent to the famous replica method MPVkniha (and unfortunately for clarity, it has also inherited some of its notations, as we shall see).

The cavity method and the statistical physics approach have been used to study the $q$ -coloring of random graphs in Coloring ; ColoringFlo ; ColoringSaad . The coloring threshold $c_{s}$ was calculated Coloring , the self-consistency of the solution checked ColoringFlo and the large $q$ asymptotics of the coloring threshold computed ColoringFlo . These results are believed to be exact although proving their validity rigorously remains a major subject in the field. Nevertheless —as the results obtained for the random satisfiability problem MeZe ; MeZeII ; MPR03 — they agree perfectly with rigorous mathematical bounds Lu ; Nature ; DuboisMandler ; AchlioptasMoore03 ; AchlioptasRegular ; Wormlad1 ; Wormlad2 , and with numerical simulations. The coloring threshold is thus fairly well understood, at least at the level of cavity method.

A maybe more interesting outcome of the statistical physics analysis of the CSPs was the identification of a new transition which concerns the structure of the set of solutions, and that appears before the coloring threshold GiulioRemiMartin ; Science ; MeZe : while at low connectivities all solutions are in a single pure state (cluster) 222In this paper we use the words ”cluster” and ”state” as synonyms., the set of solutions splits in an exponential number of different states (clusters) at a connectivity strictly smaller than $c_{s}$ . Roughly speaking, clusters are groups of near-by solutions that are in some sense disconnected from each other. Recently, the existence of the clustered phase was proven rigorously in some cases for the satisfiability problem Mora ; Fede . A major step was made by applying the cavity equations on a single instance: this led to the development of a very efficient message-passing algorithm called Survey Propagation (SP) that was originally used for the satisfaction problem in Science and later adapted for the coloring problem in Coloring . Survey propagation allows one to find solutions of large random instances even in the clustered phase and very near to the coloring threshold.

Despite all this success, the cavity description of the clustered phase was not complete in many aspects, and a first improvement has been made with the introduction of a refined zero temperature cavity formalism that allows a more detailed description of the geometrical properties of the clusters ColoringMarc ; OlivierThesis . We pursue in this direction and give for the first time a detailed characterization of the structure of the set of solutions. We observed in particular that the clustering threshold was not correctly computed, that other important transitions were overlooked and the global picture was mixed up. The corrected picture that we describe in this paper is the following: when the connectivity is increased, the set of solutions undergoes several phase transitions similar to those observed in mean field structural glasses (we sketch these successive transitions in fig. 1). First, the phase space of solutions decomposes into an exponential number of states which are entropically negligible with respect to one large cluster. Then, at the clustering threshold $c_{d}$ , even this large state decomposes into an exponential number of smaller states. Subsequently, above the condensation threshold $c_{c}$ , most of the solutions are found in a finite number of the largest states. Eventually, the connectivity $c_{s}$ is reached beyond which no more solutions exist. Another important transition, that we refer to as rigidity, takes place at $c_{r}$ when a finite fraction of frozen variables appears inside the dominant pure states (those containing almost all the solutions). All those transitions are sharp and we computed the values of the corresponding critical connectivities.

A nontrivial ergodicity breaking takes place at the clustering transition, in consequence uniform sampling of solutions becomes hard. On the other hand, clustering itself is not responsible for the hardness of finding a solution. Moreover, until the condensation transition many results obtained by neglecting the clustering effect are correct. In particular for all $c<c_{c}$ : i) the number of solutions is correctly given by the annealed entropy (and, for general CSP, by the replica symmetric entropy), and ii) simple message passing algorithm such as Belief Propagation (BP) Yedidia ; BP-Pearl converges to a set of exact marginals (i.e. the probability that a given node takes a given color). In consequence we can use BP plus a decimation-like strategy to find proper colorings on a given graph. Finally we give some arguments to explain why the rigidity transition is a better candidate for the onset of computational hardness in finding solutions.

Our results are obtained within the one-step replica symmetry breaking approach, and we believe (and argue partially in section IV), that our results would not be modified by considering further steps of RSB (as opposed to previous conclusions ColoringFlo ).

A shorter and partial version of our results, together with a study of similar issues in the satisfiability problem, was already published in US . We refer to THEM for a detailed discussion of the satisfiability problem. The paper is organized as follows: In section II, we present the model. In section III, we introduce the cavity formalism at the so-called replica symmetric level, and discuss in detail why and where this approach fails. In section IV we take into account the existence of clusters of solutions and employ the so called one-step replica-symmetry breaking formalism to describe the properties of clusters. The results for several ensembles of random graphs are then presented in section V. We finally discuss the algorithmic implications of our findings in section VI and conclude by a general discussion. Some appendixes with detailed computations complete the paper.

II The Model

II.1 Definition of the model

For the statistical physics analysis of the $q-$ coloring problem ColoringSaad ; Kanter ; Coloring we consider a Potts WU spin model with anti-ferromagnetic interactions where each variable $s_{i}$ (spin, node, vertex) is in one of the $q$ different states (colors) $s=1,\dots,q$ . Consider a graph $G=({\cal V,E})$ defined by its vertices ${\cal V}=\{1,\dots,N\}$ and edges $(i,j)\in{\cal E}$ which connect pairs of vertices $i,j\in{\cal V}$ ; we write the Hamiltonian as

[TABLE]

With this choice there is no energy contribution for neighbors with different colors, but a positive contribution otherwise. The ground state energy is thus zero if and only if the graph is $q$ -colorable. The Hamiltonian leads to a Gibbs measure GIBBS over configurations (where $\beta$ is the inverse temperature) :

[TABLE]

In the zero temperature limit, where $\beta\to\infty$ , this measure becomes uniform over all the proper colorings of the graph.

II.2 Ensembles of Random Graphs

We will consider ensembles of graphs that are given by a degree distribution ${\cal Q}(k)$ . The required property of ${\cal Q}(k)$ is that its parameters should not depend on the size of the graph. All the analytical results will concern only very large sparse graphs ( $N\to\infty$ ). Provided the second and higher moments of ${\cal Q}(k)$ do not diverge, such graphs are locally tree-like in this limit Bo ; Janson . More precisely, call a $d$ -neighborhood of a node $i$ the set of nodes which are at distance at most $d$ from $i$ . For $d$ arbitrary but finite the $d$ -neighborhood is almost surely a tree graph when $N\to\infty$ . This property is connected with the fact that the length $\ell$ of the shortest loops (up to a finite number of them) scales with the graph size as $\ell\sim\log(N)$ . We will consider the two following canonical degree distribution functions:

(i)

Uniform degree distribution, ${\cal Q}(k)=\delta(k-c)$ , corresponding to the $c$ -regular random graphs, where every vertex has exactly $c$ neighbors. 2. (ii)

Poissonian degree distribution, ${\cal Q}(k)={\rm e}^{-c}c^{k}/k!$ , corresponding to the Erdős and Rényi (ER) random graphs Erdos . A simple way to generate graphs with $N$ vertices from this ensemble is to consider that each link is present with probability $c/(N-1)$ . The binomial degree distribution converges to the Poissonian in the large size limit.

It will turn out that the cavity technics simplify considerably for the regular graphs. However, ER graphs have the advantage that their average connectivity is a real number that can be continuously tuned, which is obviously very convenient when one wants to study phase transitions. It is thus useful to introduce a third ensemble, which still has the computational advantage of regular graphs, but that at the same time gives more freedom to vary the connectivity:

(iii)

bi-regular random graphs, where nodes with connectivity $c_{1}$ are all connected to nodes with connectivity $c_{2}$ , and vice-versa. There are thus two sets of nodes with degree distributions ${\cal Q}(k)=\delta(k-c_{1})$ and ${\cal Q}(k)=\delta(k-c_{2})$ .

Notice that these graphs are bipartite by definition and therefore have always a trivial $2$ -coloring which we will have to dismiss in the following. This can be easily done within the cavity formalism (it is equivalent to neglecting the ordered “crystal” phase in glass models GLASSFERRO ).

In the first two cases, the parameter $c$ plays the role of the average connectivity, $c=\sum_{k}k{\cal Q}(k)$ . In the cavity approach, a very important quantity is the excess degree distribution ${\cal Q}_{1}(k)$ , i.e. the distribution of the number of neighbors, different from $j$ , of a vertex $i$ adjacent to a random edge $(ij)$ :

[TABLE]

This distribution remains Poissonian for Erdős-Rényi graphs, whereas the excess degree is equal to $c-1$ in the case of regular graphs. In the bi-regular case, there are two sets of nodes with excess degrees $c1-1$ and $c2-1$ .

III The cavity formalism at the replica symmetric level

We start by reviewing the replica symmetric (RS) version of the cavity method MP99 ; MP03 and its implications for the coloring problem. In the last part of the section we show when, and why, the RS approach fails.

III.1 The replica symmetric cavity equations

The coloring problem on a tree is solved exactly by an iterative method called the belief propagation algorithm BP-Pearl (note some boundary conditions have to be imposed, otherwise a tree is always 2-colorable) that is equivalent to the replica symmetric cavity method Yedidia . At this level, the method is in fact a classical tool in statistical physics to deal with that tree structure that dates back to the original ideas of Bethe, Peierls and Onsager BetheOnsager . It allows one to compute the marginal probabilities that a given node takes a given color and, in the language of statistical physics, observables like energy, entropy, average magnetization, etc. The applicability of the method goes however beyond tree graphs and we will discuss when it is correct for random locally tree-like graphs in section III.3.

Let us now describe the RS cavity equations. Denote $\psi^{i\to j}_{s_{i}}$ the probability that the spin $i$ has color $s_{i}$ when the edge $(ij)$ is not present and consider the iterative construction of a tree in fig. 2. The probability follows a recursion:

[TABLE]

where $Z_{0}^{i\to j}$ is a normalization constant (the cavity partition sum) and $\beta$ the inverse temperature. The notation $k\in i-j$ means the set of neighbors of $i$ except $j$ . The normalization $Z_{0}^{i\to j}$ is related to the free energy shift after the addition of the node $i$ and the edges around it, except $(ij)$ , as

[TABLE]

In the same way, the free energy shift after the addition of the node $i$ and all the edges around it is

[TABLE]

The $Z_{0}^{i}$ is at the same time the normalization of the total probability that a node $i$ takes color $s_{i}$ (the marginal of $i$ ):

[TABLE]

The free energy shift after the addition of an edge $(ij)$ is

[TABLE]

The free energy density in the thermodynamic limit is then given by (see for instance MP99 )

[TABLE]

Note that this relation for the free energy is variational, i.e. that if one differentiates with respect to $\beta$ , then only the explicit dependence needs to be considered (see MP99 for details). The energy (the number of contradictions) and the corresponding entropy (the logarithm of the number of solutions with a given energy) densities can be then computed from the Legendre transform

[TABLE]

were $f=F/N$ , $e=E/B$ and $s=S/N$ are intensive variables.

The learned reader will notice that in some previous works using the cavity method Coloring ; ColoringFlo , these equations were often written for a different object than the probabilities $\psi$ . Instead, the so-called cavity magnetic fields $h$ and biases $u$ where considered. The two approaches are related via

[TABLE]

Strictly speaking the $\psi$ are “cavity probabilities” while $h$ and $u$ are “cavity fields”, however $\psi$ are sometimes also referred to as cavity fields (or messages) in the literature, and the reader will thus forgive us if we do so.

Note that each of the two notations is suitable for performing the zero temperature limit in a different way: in the first one we fix the energy zero and we obtain the zero temperature BP recursion which gives the marginal probabilities for each variable, while in the second case, we obtain a simpler recursion called Warning Propagation (49) that deals with the energetic contributions but neglects the entropic ones MeZe ; Coloring . This is the origin of the discrepancy between the RS results of refs. ColoringSaad and Coloring . As we shall see, the differences between these two limits will be an important point in the paper.

III.2 Average over the ensemble of graphs and the RS solution

To compute (quenched) averages of observables over the considered ensemble of random graphs given by the degree distribution ${\cal Q}(k)$ we need to solve the self-consistent cavity functional equation

[TABLE]

where ${\cal Q}_{1}(k)$ is the distribution of the number of neighbors given that there is already one neighbor, and the function ${\cal F}(\{\psi^{i}\})$ is given by eq.(4). Beware that $\psi$ here is a $q$ -components vector while we omit the vector notation to lighten the reading. This equation is quite complicated since the order parameter is nontrivial, but we can solve it numerically using the population dynamics method described in MP99 ; MP03 .

Throughout this paper we will search only for the color symmetric functions ${\cal P}(\psi)$ , i.e. invariant under permutation of colors. Clearly with this assumption we might miss some solutions of (12). Consider for example $q>2$ colors and the ensemble of random bi-regular graphs. Since every bipartite graph is 2-colorable there are $q(q-1)/2$ corresponding color asymmetric solutions for ${\cal P}(\psi)$ . For the ensemble of random graphs considered here, we later argue that this assumption is however justified.

Another important observation is that for regular graphs the equation (12) crucially simplifies: the solution factorizes Factorizes in the sense that the order parameter $\psi$ is the same for each edge. This is due to the fact that, locally, every edge in such a regular graph has the same environment. All edges are therefore equivalent and thus the distribution ${\cal P}(\psi)$ has to be a delta function. For the bi-regular graphs, the solution of (12) also factorizes, but the two sets of nodes of connectivity $c_{1}$ and $c_{2}$ (each of them being connected to the other) have to be considered separately.

It is immediate to observe that ${\cal P}(\psi)=\delta(\psi-1/q)$ (i.e. each of the $q$ components of each cavity field $\psi$ equals $1/q$ ), is always a solution of (12). By analogy with magnetic systems we shall call this solution paramagnetic. Numerically, we do not find any other solution in the colorable phase. For regular random graphs the paramagnetic solution is actually the only factorized one. The number of proper colorings predicted by the RS approach is thus easy to compute. Since all messages are of the type ${\cal P}(\psi)=\delta(\psi-1/q)$ , the free energy density simply reads

[TABLE]

The entropy density at zero temperature thus follows

[TABLE]

It coincides precisely with the annealed (first moment) entropy. We will see in the following that, surprisingly, the validity of this formula goes well beyond the RS phase (actually until the so-called condensation transition).

III.3 Validity conditions of the replica symmetric solution

We used the main assumption of the replica symmetric approach when we wrote eq. (4): we supposed that the cavity probabilities $\psi_{s_{i}}^{k\to i}$ for the neighbors $k$ of the node $i$ are “sufficiently” independent in absence of the node $i$ , because only then the joint probability factorizes. This assumption would be true if the lattice were a tree with non-correlated boundary conditions, but loops, or correlations in the boundaries, may create correlations between the neighbors of node $i$ (in absence of $i$ ) and the RS cavity assumption might thus cease to be valid in a general graph. The aim here is to precise and quantify this statement both from a rigorous and heuristic point of view.

III.3.1 The Gibbs measure uniqueness condition

Proving rigorously the correctness of the RS cavity assumption for random graphs is a crucial step that has not yet been successfully overcome in most cases. The only success so far was obtained by proving a far too strong condition: the Gibbs measure uniqueness Aldous01 ; BG05 ; Jon ; MSh06 . Roughly speaking: the Gibbs measure (2) is unique if the behavior of a spin $i$ is totally independent from the boundary conditions (i.e. very distant spins) for any possible boundary conditions. More precisely, let us define $\{s_{l}\}$ colors of all the spins at distance at least $l$ from the spin $i$ . The Gibbs measure $\mu$ is unique if and only if the following condition holds for every $i$ (and in the limit $N\to\infty$ )

[TABLE]

where the average is over the ensemble of graphs. In BG05 ; Jon , it was proven that the Gibbs measure in the coloring problem on random regular graphs is unique only for graphs of degree $c<q$ . To the best of our knowledge, this has not been computed for Erdős-Rényi graphs (later in this section, we argue on the basis of a physical argument that it should be $c<q-1$ in this case).

III.3.2 The Gibbs measure extremality condition

In many cases, the RS approach is observed to be correct beyond the uniqueness threshold. It was thus suggested in MM05 (see also US ) that the Gibbs measure extremality provides a proper criterion for the correctness of the replica symmetric assumption. Roughly speaking, the difference between uniqueness and extremality of a Gibbs measure is that although there may exist boundary conditions for which the spin $i$ is behaving differently than for others, such boundary conditions have a null measure if the extremality condition is fulfilled. Formally (and keeping the notations from the previous section), the extremality corresponds to

[TABLE]

In mathematics the “extremal Gibbs measure” is often used as a synonym for a “pure state”. Recently, the authors of MM05 provided rigorous bounds for the Gibbs measure extremality of the coloring problem on trees.

There exist two heuristic equivalent approaches to check the extremality condition. In the first one, one studies the divergence of the so called “point-to-set” correlation length BiroliBouchaud ; MS05 . The second one is directly related to the cavity formalism: one should check for the existence of a nontrivial solution of the one-step replica symmetry breaking equations (1RSB) at $m=1$ (see section V). Both these analogies were remarked in MM05 and exploited in US . We will show in section V that the extremality condition ceases to be valid at the clustering threshold $c_{d}$ , beyond which the 1RSB formalism will be needed.

III.3.3 The local stability: a simple self-consistency check

A necessary, simple to compute but not sufficient, validity condition for the RS assumption is the non-divergence of the spin glass susceptibility (see for instance Rivoire ). If it diverges, a spin glass transition occurs, and the replica symmetry has to be broken MPVkniha . The local stability analysis thus gives an upper bound to the Gibbs extremality condition, which remarkably coincides with the rigorous upper bound of KS66 . This susceptibility is defined as

[TABLE]

The connectivities above which it diverges at zero temperature can be computed exactly within the cavity formalism (we refer to appendix A for the derivation). It follows for regular and Erdős-Rényi graphs:

[TABLE]

while the stability of the bi-regular graphs of connectivities $c_{1},c_{2}$ is equivalent to regular graphs with $c=1+\sqrt{(c_{1}-1)(c_{2}-1)}$ .

Note that for regular and ER graphs the RS instability threshold is in the colorable phase only for $q=3$ . Indeed the 5-regular graphs are 3-colorable ColoringFlo (and rigorous results in Diaz ; Wormlad2 ) and exactly critical since $c^{\rm reg}_{\rm RS}(3)=5$ , and for ER graphs the COL/UNCOL transition appears at $c_{s}\approx 4.69$ Coloring while $c^{\rm ER}_{\rm RS}(3)=4$ . This means that the replica symmetry breaking transition appears continuously at the point $c_{\rm RS}$ so that above it the RS approach is not valid anymore. For all $q\geq 4$ , however, the local stability point is found beyond the best upper bound on the coloring threshold for both regular and ER graphs. In this case, the extremality condition will not be violated by the continuous mechanism, but we will see that, instead, a discontinuous phase transition, as happens in mean field structural glasses, will take place.

Interestingly enough, a similar computation can be made for the ferromagnetic susceptibility $\chi_{F}=\frac{1}{N}\sum_{i,j}\langle s_{i}s_{j}\rangle_{c}$ (see again appendix A). It diverges at $c=q$ for regular graph and $c=q-1$ for Erdős-Rényi graphs. This divergence (called modulation instability in Rivoire ) would announce the transition towards an anti-ferromagnetic ordering on a tree, which is however incompatible with the frustrating loops in a random graph (although such order might exist on the bi-regular graphs). This is precisely the solution we dismiss when considering only the color symmetric solution of (12). Note however that the presence of this instability shows that the problem ceases to a have a unique Gibbs state (although it is still extremal) as for some specific (and well-chosen) boundary conditions, an anti-ferromagnetic solution may appear. Indeed it coincides perfectly with the rigorous uniqueness condition $c=q$ for regular graph, and suggests strongly that the uniqueness threshold (or at least an upper bound) for Erdős-Rényi graphs is $c=q-1$ .

IV One-step replica symmetry breaking framework

So far we described the RS cavity method for coloring random graphs and explained that the extremality of the Gibbs measure gives a validity criterion. We now describe the one-step replica symmetry breaking cavity solution MP99 ; MP03 . In this approach, the non-extremality of the Gibbs measure is cured, by decomposing it into several parts (pure states, clusters) in such a way that within each of the states the Gibbs measure becomes again extremal.

This decomposition has many elements/states, not just a finite numbers like the $q$ -states of the usual ferromagnetic Potts models. It is actually found that the number of pure states is growing exponentially with the size of the system. Let us define the state-entropy function $\Sigma(f)$ —called the complexity— which is just the logarithm of the number of states with internal free energy density $f$ , i.e. ${\cal{N}}(f)=\exp[N\Sigma(f)]$ . In the glass transition formalism, this complexity is usually referred to as the configurational entropy. Dealing with exponentially many pure states is obviously a nightmare for all known rigorous approaches to the thermodynamic limit. The heuristic cavity method overcomes this problem elegantly, as was shown originally in the seminal work of MP99 ; MP03 .

Another very useful intuition about the 1RSB cavity method comes from the identification of states $\alpha$ with the fixed points $\{\psi\}$ of the belief propagation equations (4). The goal is thus to compute the statistical properties of these fixed points. Each of the states is weighted by the corresponding free energy (9) to the power $m$ , where $m$ is just a parameter analogous to the inverse temperature $\beta$ (in the literature $m$ is often called the Parisi replica symmetry breaking parameter Parisi80 ; MPVkniha ). The probability measure over states $\{\psi\}$ is then

[TABLE]

where $Z_{1}$ is just the normalization constant. To write the analog of the belief propagation equations we need to define the probability (distribution) $P^{i\to j}(\psi^{i\to j})$ of the fields $\psi^{i\to j}$ . This can be obtained from those of incoming fields as

[TABLE]

The function ${\cal F}$ is given by eq. (4) and the delta function ensures that the set of fields $\psi^{i\to j}$ is a fixed point of the belief propagation (4). The re-weighting term $\left(Z_{0}^{i\to j}\right)^{m}$ takes into account the change of the free energy of a state after the addition of a cavity spin $i$ and its adjacent edges except $(ij)$ , as defined in eq. (5). This term appears for the same reason as a Boltzmann factor $e^{-\beta\delta_{s_{i},s_{k}}}$ in eq. (4): it ensures that the state $\alpha$ is weighted by $(Z_{\alpha})^{m}$ in the same way a configuration $\{s\}$ is weighted by $e^{-\beta{\cal H}(\{s\})}$ in (2). Finally $Z_{1}^{i\to j}$ is a normalization constant. In the second line of (20) we introduced an abbreviation that will be used from now on to make the equations more easily readable. The notation POP comes from “population dynamics” which refers to the numerical method we use to solve eq. (20). The probability distribution $P(\psi)$ can be represented numerically by a set of fields taken from $P(\psi)$ , and then the probability measure $P(\psi){\rm d}\psi$ becomes uniform sampling from this set, for more details see appendix D.

We define the “replicated free energy” and compute it in analogy with eq. (9) as

[TABLE]

where

[TABLE]

Putting together (19) and (21) we have

[TABLE]

where the sum over $\{\psi\}$ is over all states (or BP fixed points). In the interpretation of Remi $m$ is the number of replicas of the system, thus the name “replicated free energy” for $\Phi(\beta,m)$ . Note that we are using the word “replica” only to refer to the established terminology as no replicas are needed within the cavity formalism. From the saddle point method, it follows that the Legendre transform of complexity function $\Sigma(f)$ gives the replicated free energy $\Phi(m)$

[TABLE]

Notice that this equation is correct only in the highest order in the system size $N$ , i.e. in densities and at the thermodynamic limit. From the properties of the Legendre transform we have

[TABLE]

Thus, from eq.(21), the free energy reads

[TABLE]

When the parameter $m$ is equal to one (the number of replicas is actually one in the approach of Remi ), then $-\beta\Phi(\beta,1)=-\beta f(\beta)+\Sigma(f)$ reduces to the usual free energy function considered in the RS approximation

[TABLE]

where $s$ in the internal entropy density of the corresponding clusters and $s_{\rm tot}$ the total entropy density of the system.

IV.1 Analyzing the 1RSB equations

Combining (21) and (26) we can compute $\Sigma$ and $f$ for each value $m$ , that gives us implicitly $\Sigma(f)$ . To compute the thermodynamic observables in the model we have to minimize the total free energy $f_{\rm tot}=f(\beta)-\Sigma/\beta$ over such values of $f$ where the complexity $\Sigma(f)$ is non-negative (so that the states exist in the thermodynamic limit). The minimum of the total free energy corresponds to a value of parameter $m=m^{*}$ and states with the free energy $f^{*}$ dominate the thermodynamics. Three different cases are then observed:

a)

If there is only the trivial (replica symmetric) solution at $m=1$ , then the Gibbs measure (2) is extremal and the replica symmetric approach is correct. If at the same time a nontrivial solution exists for some $m\neq 1$ , then the clusters corresponding to this solution have no influence on the thermodynamics. 2. b)

If there is a nontrivial solution at $m=1$ with a positive complexity, then $m^{*}=1$ minimizes the total free energy. The system is in a clustered phase with an exponential number of dominating states. 3. c)

If however the complexity is negative at $m=1$ , then the corresponding states are absent with probability one in the thermodynamic limit. Instead the total entropy is dominated by clusters corresponding to $m^{*}$ such that $\Sigma(m^{*})=0$ : the system is in a condensed phase. Note that the condition $\Sigma(m^{*})=0$ corresponds to the maximum of the replicated free energy (21).

The transitions between these cases are well known in structural glass phenomenology where they appear when the temperature is lowered REM ; GLASSTHEORY . The transition from the paramagnetic phase to the clustered one is usually referred to as the dynamical transition MS06 or the clustering transition. It is not a true thermodynamic transition as the total free energy of the system at $m^{*}=1$ is still equal to the replica symmetric one (9) (see appendix C) and thus is an analytical function of connectivity. However, the phase space is broken into exponentially many components and, as a consequence, the dynamics fall out-of-equilibrium beyond this transition.

The second transition from the clustered to the condensed phase is, however, a genuine thermodynamic transition (the free energy has a discontinuity in the second derivative at $c_{c}$ ) and is called the replica symmetry breaking transition, or the static glass transition. At this point the measure condenses into few clusters, and we shall call it the condensation transition. In structural glasses, it corresponds to the well known Kauzmann transition Kauzmann . The sizes of the clusters in the condensed phase follow the so called Poisson-Dirichlet process which is discussed shortly in appendix B.

The procedure to compute the replicated free-energy (21) and the related observables was described above for a single large random graph. To compute the averages over the ensemble of random graphs, we need to solve an equation analogous to eq. (12)

[TABLE]

where the functional ${\cal F}_{2}$ is given by eq. (20). Solving this equation for a general ensemble of random graphs and a general parameter $m$ is a numerically quite tedious problem. In the population dynamics algorithm MP99 ; MP03 we need to deal with a population of populations of q-components fields. It is much more convenient to look at the ensemble of random regular graphs where a factorized solution ${\cal P}[P(\psi)]=\delta(P(\psi)-P_{0}(\psi))$ must exists. Then we are left with only one functional equation (20).

Before discussing the zero temperature limit, we would like to point out that there exists another very important case in which eq. (28) simplifies. For $m=1$ , as first remarked and proved in MM05 , when dealing with the problem of reconstruction on trees, the equations can be written (and numerically solved) in a much simpler way. We again refer to the appendix C for details. Especially for the Poissonian random graphs this simplification is very useful.

IV.2 Zero temperature limit

We now consider the zero temperature limit $\beta\to\infty$ of the 1RSB equations (24)-(28) to study the coloring problem. In most of the previous works MP03 ; MeZe ; Coloring the energetic zero temperature limit was employed. The $\beta\to\infty$ limit of eq. (24) was taken in such a way that $m\beta=y$ remains constant. The replicated free energy (24) then becomes

[TABLE]

It is within this approach that the survey propagation (SP) algorithm was derived. The connectivity at which the complexity function $\Sigma(e=0)$ becomes negative is the coloring threshold. Above this connectivity $\Sigma(e)$ was used to compute the minimal number of violated constraints (the ground state energy). The reweighting in eq. (20) becomes ${\rm e}^{-y\Delta E^{i\to j}}$ , and when $y\to\infty$ all the configurations with positive energy are forbidden.

In this paper we adopt the entropic zero temperature limit, suggested originally in ColoringMarc ; OlivierThesis . The difference in the two approaches was already underlined in sec. III.1. We want to study the structure of proper colorings, i.e. the configurations of zero energy, and we thus fix the energy to zero. Then we obtain the entropy by considering $-\beta f=s$ and introduce a free entropy —or in replica term a “replicated entropy”— as $\Phi_{s}(m)=-\beta m\Phi(\beta,m)|_{\beta\to\infty}$ . Eq. (24) then becomes

[TABLE]

The belief propagation update (4) becomes

[TABLE]

while the 1RSB equation (20) keeps the same expression (and thus the same computational complexity).

The partition sum $Z_{0}$ in (2) becomes in this limit the number of proper colorings or solutions. The clusters are now sets of such solutions, and are weighted by their size to the power $m$ . The free entropy $\Phi_{s}(m)$ is then computed as

[TABLE]

where $\Delta Z^{i}$ and $\Delta Z^{ij}$ are given by eqs. (5) and (6). The analysis from the previous section is valid also for the entropic zero temperature limit. The information extracted from the number of clusters of a given size, $\Sigma(s)$ , is one of the main results of this paper and will be discussed and interpreted in section V.

IV.3 The role of frozen variables

In this section we discuss the presence and the role of the frozen variables and explain the connection between the energetic and the entropic zero temperature limits. This allows us to revisit (and extend) the survey propagation equations. Remember that the components of the cavity field $\psi^{i\to j}_{s_{i}}$ are the probabilities that the node $i$ takes the color $s_{i}$ when the constraint on the edge $ij$ is not present. In the zero temperature limit we can classify them in two categories:

(i)

A hard field corresponds to the case when all components of $\psi^{i\to j}$ are zero except one, $s$ . Then only that color is allowed for the spin $i$ , in absence of edge $(ij)$ . 2. (ii)

A soft field corresponds to the case when more than one component of $\psi^{i\to j}_{s_{i}}$ is nonzero. The variable $i$ is thus not frozen in absence of edge $(ij)$ , and the colors of all the nonzero components are allowed.

This distinction is also meaningful for the full probabilities $\psi^{i}_{s_{i}}$ (4), if $\psi^{i}_{s_{i}}$ is a hard field then the variable $i$ is frozen. In the colorable region there cannot exist a finite fraction of frozen variables (even if we consider properly the permutational symmetry of colors) since by adding a link the connectivity changes by $1/N$ but the probability of becoming uncolorable would be finite. On the contrary, in the 1RSB picture, we observe that a finite fraction of variables can be frozen within a single cluster. In other words, in all the solutions that belong to this given cluster a finite fraction of variables can take one color only. By adding a link into the graph, the connectivity grows by $1/N$ , and there is a finite probability that a cluster with frozen variables disappears. The distinction between hard and soft fields is useful not only for the intuition about clusters, but also for the analysis of the cavity equations and it also leads to the survey propagation algorithm.

The distribution of fields over states $P^{i\to j}(\psi^{i\to j})$ (20) can be decomposed into the hard- and soft-field parts

[TABLE]

where $\tilde{P}^{i\to j}$ is the distribution of the soft fields and the normalization is $\sum_{s=0}^{q}\eta^{i\to j}_{s}=1$ .

Interestingly, the presence of frozen variables in the entropically dominating clusters is connected to the divergence of the size of average minimal rearrangement MS05 ; Guilhem_rear . Precisely, choose a random proper coloring $\{s\}$ and a random node $i$ in the graph. The average minimal rearrangement is the Hamming distance to the nearest solution in which node $i$ has a color different from $s_{i}$ averaged over the nodes $i$ , the proper colorings, and graphs in the ensemble.

Another interesting role of the frozen variables arises within the whitening procedure, introduced in ParisiWhit and studied, between others, for the satisfiability problem in Elitza ; Riccardo . This procedure is equivalent to the warning propagation (WP) update (49) which we outlined in sec. III.1. Whitening is able to identify if a solution belongs to a cluster with frozen variables or not. Particularly, the result of the whitening is a set of hard cavity fields.

Since the survey propagation algorithm is computing statistics over the states that contain hard fields, then the solution found after decimating the survey propagation result should a priori also contain hard fields. However, recent works show that if one applies the whitening procedure starting from solutions found by SP on large graphs, whitening converges every time to the trivial fixed point (see detailed studies for K-SAT in Elitza ; Riccardo ). A possible solution to this apparent paradox is discussed in sec. VI.

IV.3.1 Hard fields in the simplest case, $m=0$

Let us now consider the survey propagation equations originally derived in Coloring from the energetic zero temperature limit (29) when $y\to\infty$ . For simplicity we will write them only for the 3-coloring. We consider the 1RSB cavity equation (20) for $m\to 0$ , then the reweighting factor $(Z_{0}^{i\to j})^{m}$ is equal to zero when the arriving hard fields are contradictory, and equal to one otherwise. The update of probability $\eta_{s}$ that a field is frozen in direction $s$ is then written from eq.(20):

[TABLE]

In the numerator there is a telescopic sum counting the probability that color $s$ and only color $s$ is not forbidden by the incoming fields. In the denominator the telescopic sum is counting the probability that there is at least one color which is not forbidden. If we do not want to actually find a proper coloring on a single graph but just to compute the replicated free energy/entropy, we can further simplify eq. (34) by imposing the color symmetry. Indeed, the probability that in a given state a field is hard in direction of a color $s$ has to be independent of $s$ (except $s=0$ which corresponds to a soft field). Then (34) becomes, now for general number of colors $q$ :

[TABLE]

Note that since $\partial\Sigma(s)/\partial s=-m$ , the value $m=0$ corresponds to the point where the function $\Sigma(s)$ has a zero slope. If a nontrivial solution of (35) exists, then $\Sigma(s)|_{m=0}$ is the maximum of the curve $\Sigma(s)$ and is counting the total log-number of clusters of size $s$ , which is due to the exponential dependence, also the total log-number of all clusters, regardless their size. There are two points that we want to emphasize:

•

Suppose that a nontrivial solution of (35) exists, i.e. many clusters exist and their number can be computed with the energetic zero temperature limit calculations. Then the clusters might be very small and contain very few solution in comparison to bigger less numerous clusters; or in comparison to a giant single cluster which might still exist. This situation cannot be decided by the energetic formalism that weights clusters equally independently of their size.

•

Suppose, on contrary, that a nontrivial solution of (35) does not exist. It might still well be that many clusters exist, but the $\Sigma(s)$ curve has no part with zero slope.

We will see that these two cases are actually observed. The energetic method, that can locate the coloring threshold and from which the survey propagation can be derived, is therefore not a good tool to study the clustering transition.

IV.3.2 Generalized survey propagation recursion

Let us compute how the fraction of hard fields $\eta$ evolves after one iteration of equation (20) at general $m$ . There are two steps in each iterations of (20). In the first step, $\eta$ iterates via eq. (35). In the second step we re-weight the fields. Writing $P^{\rm hard}_{m}(Z)$ the —unknown— distribution of the reweightings $Z^{m}$ for the hard fields, one gets

[TABLE]

A similar equation can formally be written for the soft fields

[TABLE]

Writing explicitly the normalization ${\cal N}$ , we finally obtain the generalized survey propagation equations:

[TABLE]

In order to do this recursion, the only information needed is the ratio $r$ between between soft- and hard-field reweightings, which is in general difficult to compute since it depends on the full distribution of soft fields.

There are two cases where eq. (38) simplifies so that the hard-field recursion become independent from the soft-field distribution. The first case is, of course, $m=0$ then $r=1$ independently of the edge $(ij)$ , and the equation reduces to the original SP. The second case arise for $m=1$ , where one can use the so-called reconstruction formalism and obtain again a closed set of equations. The computation is done in appendix C, and the SP equations at $m=1$ read

[TABLE]

It would be interesting in the future to use eq. (38) in an algorithm to find proper graph colorings, as it has been done with the original SP equation Coloring . As an approximation one might also use a value $r$ independent of the edge $(ij)$ , but different from one.

For the purpose of the present work, it is important to notice that it is also possible to use eq. (38) in the population dynamics to simplify the numerical evaluation of the 1RSB solution by separating the hard-field and the soft-field contributions. Indeed, it gives the exact density of hard fields provided the ratio $r$ is calculated, which is doable numerically (see appendix D). This allows us to monitor precisely the hard-field density and only the soft-field part is given by the population dynamics. This turns out to greatly improve the precision of the numerical solution of the cavity equations and to considerably fasten the code.

IV.3.3 The presence of frozen variables

A natural question is: “When are the hard fields present?” or more precisely: “When does eq. (38) have a nontrivial solution $\eta>0$ ?” First notice that in order to constrain a node into one color, one needs at least $q-1$ incoming fields that forbids all the other colors. It means that function $w(\{\eta^{k\to i}\})$ defined in eq. (35) is identically zero for $k<q-1$ and might be non-zero only for $k\geq q-1$ , where $k$ is the number of incoming fields.

In the limit $r\to 0$ (which corresponds to $m\to-\infty$ ) eq. (38) gives $\eta=1/q$ if $w(\{\eta^{k\to i}\})$ is positive, and $\eta=0$ if $w(\{\eta^{k\to i}\})$ is zero. Updating eq. (38) on a given graph, from initial conditions $\eta=1/q$ everywhere, is equivalent to recursive removing of all the nodes of connectivity smaller than $q$ . This shows that the first nontrivial solution with hard fields exists if and only if the $q$ -core q_core of the graph is extensive. For regular graphs it is simply at connectivity $c=q$ while for Erdős-Rényi graphs these critical connectivities can be computed exactly and read, for small $q$ , $c_{3}=3.35$ , $c_{4}=5.14$ , $c_{5}=6.81$ q_core . Indeed we see that the first nontrivial solution to the 1RSB equation appears much before those of the original SP equation at $m=0$ .

On a regular graph, the equations further simplify as $\eta$ factorizes (is edge independent) and follows a simple self-consistent equation

[TABLE]

This equation can be solved for every possible ratio $r$ so that for all $c\geq q$ , we can compute and plot the curve $\eta(r)$ . We show the results in fig. 3 for different numbers of colors $q$ . On this plot, we observe that $\eta=1/q$ , as predicted, for $r=0$ . It then gets smaller for larger value of the ratio and, at a critical value $r_{\rm crit}$ , the solution disappears discontinuously and only the (trivial) solution $\eta=0$ exists. The values $r_{\rm crit}$ correspond to a critical value of $m_{r}$ . For all $m>m_{r}$ no solution containing frozen variables can exist.

IV.4 Validity conditions of the 1RSB solution

Now that we have discussed in detail the 1RSB formalism the next question is: “Is this approach correct?” To answer this question, one has to test if the Gibbs measure is extremal within the thermodynamically dominating pure states. This is equivalent to checking if the two-step Replica Symmetry Breaking (2RSB) solution is non trivial. Computing explicitly the 2RSB solution is however very complicated numerically, especially for Erdős-Rényi graphs. Instead, the local stability of the 1RSB solution towards 2RSB should be checked, in analogy with the RS stability in sec. III.3.3. It is indeed a usual feature in spin glass physics to observe that the 1RSB glass phase become unstable at low temperatures towards a more complex RSB phase and this phenomenon is called the Gardner transition Gardner .

To perform the stability analysis MR03 ; MPR03 ; ColoringFlo ; Rivoire , one first writes the 2RSB recursion —where the order parameter is a distribution of distributions of fields on every edge $P_{1}(P_{2}(\psi))$ — and then two types of 1RSB instabilities have to be considered depending on the way the 2RSB arises from the 1RSB solution. The first type of instability —called states aggregation— corresponds to $\delta(P(\psi))\rightarrow P_{1}(P_{2}(\psi))$ while the second type —called states splitting— corresponds to $P(\delta(\psi))\rightarrow P_{1}(P_{2}(\psi))$ . A complete stability analysis is left for future works, but it is worth to discuss the relevance of the results derived over the last few years MR03 ; MPR03 ; ColoringFlo .

The 1RSB stability was studied for the coloring problem in ColoringFlo but only for the energetic zero temperature limit (29). In this case the parameter $y=\beta m$ is conjugated to the energy. The results derived in ColoringFlo —as well as those previously derived for other problems MR03 ; MPR03 — thus concern only the clusters of sizes corresponding to $m=0$ at zero (for $y=\infty$ ) or at positive (for finite $y$ ) energy. The main result of ColoringFlo ; MR03 ; MPR03 was that the 1RSB approach was stable in vicinity of the coloring threshold $c_{s}$ . As we shall see the clusters corresponding to $m=0$ are those dominating the total entropy at the coloring threshold and as a consequence its location is thus exact within the cavity approach. The states of the lowest energy (the ground states) in the uncolorable phase also correspond to $m=0$ , and thus the conclusions of ColoringFlo ; MR03 ; MPR03 concerning the uncolorable phase are also correct. In particular, a Gardner transition towards further steps of RSB appears in the uncolorable phase beyond a connectivity denoted $c_{G}$ in ColoringFlo .

On the other hand in the colorable phase the stability of the entropically dominating clusters that correspond to $m>0$ should be investigated. Some more relevant information can be, however, already drawn from known results. It was indeed found that the 1RSB approach at $m=0$ is type I stable for all $y$ , and type II stable for all $y>y_{I}$ in vicinity of the coloring threshold. These results concerns the states of positive energy, but keeping in mind the interpretation of $y$ as a slope in $T,m$ diagram, we see that the clusters of zero energy corresponding to small but nonzero positive $m$ and zero temperature are also stable with respect to both types of stabilities. Near the colorable threshold, the value of $m^{*}$ which describes the dominating clusters is close to zero and as a consequence all the dominating clusters are 1RSB stable in vicinity of the coloring threshold. Far from the coloring threshold, the stability analysis of MPR03 ; ColoringFlo is irrelevant. In particular, the predictions of a full-RSB colorable phase made in ColoringFlo ; MR03 ; MPR03 is not correct. Quite the contrary, our preliminary results indicate that all the dominating clusters are 1RSB stable for $q>3$ .

The $3-$ coloring is however a special case as the clustering transition is continuous. Although the type II instability seems irrelevant in this case as well, we cannot at the moment dismiss a type I instability close to the clustering transition. Indeed the entropically relevant clusters correspond to values of $m^{*}$ close to one in this case, and it is simple to show that the clusters at $m=1$ are type I unstable in the case there is a continuous transition: this is because the type I stability is equivalent to the convergence of the 1RSB update on a single graph. Since for $m=1$ the averages of the 1RSB fields satisfy the RS belief propagation equations, and since we know from the RS stability analysis in section III.3.3 that those equations do not converge in the RS unstable region (i.e. for $c>c_{\rm RS}=4$ in 3-coloring of Erdős-Rényi graphs), it then follows that the 3-coloring is unstable against state aggregation at $m=1$ for all connectivities $c>4$ . Therefore, it is possible that the 1RSB result for 3-coloring are only approximative for what concerns the number and the structure of solutions close to the clustering transition (note that the critical values for the phase transition are however correct and do not depend on that). This, and related issues SUSY , will be hopefully clarified in future works.

To conclude, we believe that all the transition points we discuss in this paper (and those computed in the K-SAT problem in US ) as well as the overall picture, are exact and would not be modified by considering further steps of replica symmetry breaking.

V The coloring of random graphs: cavity results

We now solve the 1RSB equations, discuss and interpret the results. We solve the equation (28) by the population dynamics technique, the technical difficulties and the precision of the method are discussed in appendix D. Let us stress at this point that the correctness of eq. (28) is guaranteed only in the limit of large graphs ( $N\to\infty$ ), unfortunately the cavity method does not give us any direct hint about the finite graph-size corrections. We start by the results for the regular random graphs, then consider ensemble of bi-regular graphs and after that we turn towards Erdős-Rényi graphs. Finally, we discuss the limit of large number of colors.

V.1 Regular random graphs

Let us fix the number of colors $q$ , vary the connectivity, and identify successively all the transitions that we shall encounter. For the sake of the discussion, we choose as a typical example the 6-coloring and we discuss later in details the cases, for different number of colors, where some transitions are missing or are arriving in a different order. We solved the 1RSB equation (20) for regular graphs, where the distribution $P^{i\to j}(\psi)$ is the same for every edge $(ij)$ (see appendix D) and plot the curves for $\Sigma(s)$ we obtained doing so in fig. 4. We now describe the phase space of solutions when the connectivity is increased:

At very low connectivities $c<q$ , only the paramagnetic RS solution is found at all $m$ . i.e. $P(\psi)=\delta(\psi-1/q)$ . The phase space is made of a single RS cluster. 2. 2)

For larger connectivities $c\geq q$ , we saw in section IV.3.3 that the 1RSB equations start to have nontrivial solutions with hard fields in an interval $[-\infty,m_{r}]$ . Interestingly, another nontrivial solution, without hard fields, can be found numerically in an interval $[m_{s},\infty]$ , and we shall call this one the soft-field solution. As the connectivity increases, we find that $m_{r}$ increases while $m_{s}$ decreases, so that the gap $[m_{r},m_{s}]$ where no nontrivial solution exists it getting continuously smaller.

However, there is no nontrivial solution at $m=1$ for connectivities smaller then $c_{d}$ (see fig. 4 for the example of the $6$ -coloring at $c=17$ ). This means that the Gibbs measure (2) is still extremal. In other words the large RS state still exists and is entropically dominant (its entropy (14) is noted by a circle in fig. 4). Despite the fact that an exponential number of clusters of solutions exist and that the SP equations converge to a nontrivial result, a random proper coloring will almost surely belong to the large RS cluster. 3. 3)

If the connectivity is increased at and above the clustering threshold $c_{d}$ , a nontrivial solution with positive complexity $\Sigma$ is found at $m=1$ . In fig. 4, we see that this happens at $c_{d}=18$ for the regular $6$ -coloring. At this point, the RS Gibbs measure (2) ceases to be extremal and the single large RS cluster splits into exponentially numerous components. To cover almost all proper colorings we need to consider exponentially many clusters ${\cal N}\sim{\rm e}^{N\Sigma(m^{*}=1)}$ . The probability that two random proper colorings belong to the same cluster is going exponentially to zero with the system size. The connectivity $c_{d}$ is thus the true clustering (dynamic) transition. This is not, however, a thermodynamic phase transition because the 1RSB total entropy reduces to the RS entropy (14) at $m=1$ which is analytical in $c$ . Thus the RS approach gives a correct number of solution and correct marginals as long as the complexity function at $m=1$ is non-negative. 4. 4)

For even larger connectivities $c\geq c_{c}$ , the complexity at $\Sigma(m=1)$ becomes negative, e.g. $c_{c}=19$ for 6-coloring. It means that the clusters corresponding to $m=1$ are absent with probability one. The total entropy is then smaller than the RS/annealed one and is dominated by clusters corresponding to $m^{*}<1$ such that $\Sigma(m^{*})=0$ . The ordered weights of the entropically dominating clusters follow the Poisson-Dirichlet process (explained in appendix B). As a consequence, the probability that two random proper colorings belong to the same cluster is finite in the thermodynamic limit. Another way to describe the situation is that the entropy condenses into a finite number of clusters. This condensation is a true thermodynamic transition, since the total entropy is non-analytical at $c_{c}$ (there is a discontinuity in its second derivative with respect to connectivity). The condensation is analogous to the static (Kauzmann) glass transition observed in mean field models of glasses REM ; GLASSTHEORY . 5. 5)

For connectivities $c\geq c_{s}$ ( $c_{s}=20$ for $6$ -coloring) even the maximum of the complexity $\Sigma(m=0)$ becomes negative. In this case proper colorings are absent with probability going to one exponentially fast with the size of the graph, and we are in the uncolorable phase.

It is useful to think of the growing connectivity as additions of the constraints into a fixed set of nodes. From this point of view the set of solutions which exists at connectivity $c$ gets smaller when new edges are introduced and the connectivity increased. This translates into the cartoon in the introduction (fig. 1) where all the successive transitions are represented. Finally, another important transition has to be considered:

There is a connectivity $c_{r}$ beyond which the measure is dominated by clusters that contain a finite fraction of frozen variables. For the regular $6-$ coloring, $c_{r}=19$ . We refer to this as the rigidity transition, by analogy with Jamming ; Jorge .

The presence or the absence of hard fields inside a given cluster is crucial: if a cluster contains only soft fields, then after the addition of a small but finite fraction of new constraints, its size will get smaller (or it will split). If, however, a cluster contains a finite fraction of frozen variable, then after adding a small but finite fraction of links the cluster will almost surely disappear.

Since the connectivities of regular graphs are integer numbers, we define the dynamical threshold $c_{d}$ as the smallest connectivity where a nontrivial 1RSB solution exists at $m=1$ , the condensation transition $c_{c}$ as the smallest connectivity where complexity at $m=1$ is negative, $c_{r}$ the smallest connectivity where hard fields are present at $m^{*}$ and the coloring threshold $c_{s}$ as the first uncolorable case. The scenario described here is observed for all cases of the regular ensemble, although, since connectivities are integer, the transitions are not very well separated at small $q$ . We summarize the results in table 1.

Note that for $q>3$ , the local RS stability discussed in section III.3.3 is irrelevant in the colorable regime. The only subtle case being for $3-$ coloring of 5-regular graphs where the RS solution is only marginally stable, i.e. the spin glass correlation function goes to zero only algebraically instead of exponentially (from this point of view $c=5$ would correspond to the critical point well known in the second order phase transitions). More interesting cases will arise in the other ensembles of random graphs.

V.2 Results for the bi-regular ensemble

The bi-regular ensemble allows us to fine-tune the connectivity while preserving the factorization of the 1RSB solution, which is crucial for the numerical precision. It is actually more correct to say that the solution is “bi-factorized”, as all the messages going from the nodes with connectivity $c_{1}$ to $c_{2}$ are the same and the other way around. The bi-regular ensemble allows us to describe with large precision two interesting cases, which reappear in the Erdős-Rényi ensemble and which are not present in the regular ensemble (again, due to the discrete nature of the connectivity). Let us remind here that bipartite graphs are always 2-colorable, but we consider only the color symmetric cavity solutions and that is why we get a nontrivial result from this ensemble.

In fig. 5, the left picture is the result for the complexity as a function of entropy $\Sigma(s)$ for 4-coloring of 5-21-bi-regular graphs. The replica symmetric solution on this case is locally stable. We see clearly the gap between the hard-field and the soft-field solution, and yet we are already beyond the clustering transition $c_{d}$ ; actually the system is in the condensed phase. This example is similar to what happens for the 4-coloring of Erdős-Rényi graphs.

The second interesting case, the right hand side of fig. 5, is given by the results for $\Sigma(s)$ for the 4-coloring of 4- $c$ -bi-regular graphs, which are RS unstable for $c>28$ . Both the clustering and the condensation transitions coincide with the RS instability $c_{d}=c_{c}=28$ . The survey propagation equations have a nontrivial solution starting from $c_{\rm SP}=37$ . The rigidity transition is at $c_{r}=49$ . Finally the coloring threshold is $c_{s}=57$ . Qualitatively, the results for this 4- $c$ -bi-regular ensemble are the same as those for the 3-coloring of Erdős-Rényi random graphs.

We see that for $c\leq 42$ the gap between the hard-field (full line) and soft-field (dotted line) solution exists. For $m>m_{s}$ there is a non-physical nontrivial soft-field solution, the convex part of the line in the figure, zoomed in the inset. It means that for $m<m_{r}$ we actually can find two solutions depending if we start or not with a population containing enough hard fields. The unphysical branch survives even when the gap $[m_{r},m_{s}]$ closes, see the example of $c=45$ in the figure.

We would like to stress at this point the enormous similarity of the soft-field part of the curve $\Sigma(s)$ to the one in fig. 4 in ref. GiulioRemiMartin . Actually the variational results of GiulioRemiMartin should be very precise and relevant near to the continuous clustering transition (this is also case for the 3-coloring of Erdős-Rényi graphs or for 3-SAT).

V.3 Results for Erdős-Rényi random graphs

For Erdős-Rényi random graphs obtaining the solution of eq. (28) is computationally more involved as the solution is no longer factorized. In the population dynamics a population of populations has to be updated, which is numerically possible only for small populations, and so one has to be careful that the finite population-size corrections are small enough, see details in appendix D. However, all the computations can be done with the same computational complexity as for the regular graphs for $m=0$ , the energetic zero temperature limit (section IV.3.1), and for $m=1$ (appendix C). That is enough to obtain the SP, clustering, condensation and COL/UNCOL transitions (from which the first and last one was computed in Coloring ). We can also compute exactly when hard fields appear for $m=1$ , eq. (72), this transition is further studied in Guilhem_rear . Finally, using the generalized survey propagation equation introduced in section IV.3.2, the rigidity transition can be computed quite precisely.

V.3.1 The general case for $q>3$ , discontinuous clustering transition

The phase transitions in $q$ -coloring of random Erdős-Rényi graphs for $q>3$ are qualitatively identical to those discussed in the case of random regular graphs. We plot the results for the total entropy (number of solutions) and complexity (number of clusters which dominate the entropy) in the $4-$ and $5-$ coloring in fig. 6.

At the clustering transition $c_{d}$ the complexity becomes discontinuously positive, the large RS cluster suddenly splits in an exponential number of smaller ones. The total entropy $\Sigma^{*}+s^{*}$ is given by the RS formula (14) up to the condensation transition $c_{c}$ . At the condensation transition the complexity of the dominating clusters becomes zero, the total entropy $s^{\rm tot}=s^{*}<s_{\rm RS}$ is given by the point where $\Sigma(s^{*})=0$ . The function $s^{\rm tot}(c)$ is non-analytical at the point $c_{c}$ , it has a discontinuity in the second derivative. At the coloring threshold $c_{s}$ all the clusters of solutions disappear, note, however, that the total entropy of the last existing clusters is strictly positive (about a half of the total entropy at the condensation transition). That means that the COL/UNCOL transition is not only sharp but also discontinuous in terms of entropy of solutions. Note that the positive entropy has two contributions: the trivial and smaller one coming from presence of leaves and other small subgraphs, and the nontrivial and more important one connected with the fact, that the ground state entropy is positive, even in the uncolorable phase or for the random regular graphs.

Finally we located the rigidity transition, when frozen variables appears in the dominating clusters. For $3\leq q\leq 8$ this transition appears in the condensed phase. As the number of colors grows it approaches the clustering transition. All the four critical values $c_{d}$ , $c_{r}$ , $c_{c}$ and $c_{s}$ are summarized in table 2, values of $c_{\rm SP}$ and $c_{r}(m=1)$ are given for comparison.

V.3.2 The special case of $3-$ coloring, continuous clustering transition

The only case which is left to be discussed is the 3-coloring of Erdős-Rényi graphs. It is different from $q>3$ because the replica symmetric solution is locally unstable in the colorable phase (see section III.3.3). The extremality condition underlying the RS assumption ceases to be valid because of the mechanism discussed in section III.3.3, with a divergence of the spin glass correlation length: the main difference with the previous cases is therefore that the clustering transition is continuous and coincide with the condensation transition.

However, the phenomenology does not differ too much from the other cases: $c_{\rm RS}=c_{d}=c_{c}=4$ ; the phase where the entropy is dominated by exponential number of states is thus missing and the complexity corresponding to $m=1$ is always negative (see fig. 7 left together with the dependence of the total entropy on the connectivity). Note that the curves $\Sigma(s)$ for the 3-coloring have been already studied in ColoringMarc ; OlivierThesis where the authors considered however only the range of connectivities $c=[4.42,4.69]=[c_{\rm SP},c_{s}]$ .

All the results derived for the 4-coloring of 4- $c$ -bi-regular bipartite graphs are quantitatively valid also here. We are thus not surprised by the fact that in interval $c=[4,4.42]$ the survey propagation algorithm gives us a trivial result: simply the maximum of the curve $\Sigma(s)$ does not exist yet there is no nontrivial solution at $m=0$ . Yet, the entropy is dominated by finite number of largest clusters which do not contain hard fields. The two solutions (hard-field and soft-field) join at a connectivity around $4.55$ . Finally at $c_{r}=4.66$ the hard fields arrive to the dominating states (and in consequence to all others).

V.3.3 The overlap structure

We now give some results about the overlap structure in the random coloring to elaborate the intuition about clusters. First, consider marginal probabilities $\psi^{i,\alpha}_{s_{i}}$ within a cluster $\alpha$ . Note that due to the color symmetry there exist another $q!-1$ clusters different only in the permutation of colors. We define the intra-cluster overlap of two solutions (averaged over states) as

[TABLE]

In the paramagnetic phase $\delta=1/q$ , otherwise we have to compute it from the fixed point of equation (20). The overlap between two solutions which lie in two clusters, which differ just by permutation $\pi$ of colors is

[TABLE]

where $j$ is the number of fixed positions in the permutation $\pi$ (in particular $\delta_{q}=\delta$ , and $\delta_{1}=1/q$ ). In fig. 8 we show the overlap structure for 3- and 4-coloring. The probabilities that two random solutions have one of the overlaps can be computed from the Poisson-Dirichlet process described in appendix B, in fact this is not a self-averaging quantity MPVkniha .

V.4 Large q Asymptotics

We give here the exact analytical large $q$ expansion of the previous results. In the asymptotic computations the regular and Erdős-Rényi ensembles are equivalent (the corrections are of smaller order in $q$ that the orders we give). We refer to the appendix E for the explicit derivation of the formulae.

At large $q$ a first set of transitions arises for connectivities scaling as $q\log{q}$ :

[TABLE]

$c_{\rm SP}$ was already computed in ColoringFlo and $c_{r}$ is the rigidity transition. The clustering transition has to appear before the rigidity one $c_{d}<c_{r}$ . For all the finite $q$ cases we looked at, $c_{d}$ was between $c_{\rm SP}$ and $c_{r}$ .

A second set of transitions arises for connectivities scaling as $2q\log{q}$ :

[TABLE]

The condensation thus appears very close the COL/UNCOL transition and both are very far from the clustering and rigidity transitions (those are on a half way in the phase diagram).

We show also in appendix E that for connectivity $c=2q\log{q}-\log{q}+\alpha$ , one has

[TABLE]

Since the RS free energy is correct until $c_{c}$ , which differers just by constant from $c_{s}$ , that means that for all connectivities bellow $c_{c}$ the number of solutions is correctly given by the replica symmetric entropy (14). Indeed, the value $s(m=1)$ can be obtain by a large expansion of eq. (14).

In fig. 9 we plot the complexity of dominating clusters $\Sigma^{*}=\Sigma(m^{*})$ , the total entropy $s^{\rm tot}=\Sigma^{*}+s^{*}$ , and the physical value of $m^{*}$ as a function of connectivity $c=2q\log{q}-\log{q}+\alpha$ . Note that the properly scaled values of the total number of solutions at $c_{c}$ and $c_{s}$ , and the values $c_{c}$ , $c_{s}$ themselves, are already very close to those at $q=3,4,5$ (see figs. 6 and fig. 7 left). The closeness is particularly striking for the values $m^{*}$ for $q=4$ and $q=5$ (see fig. 7 right).

These formulae show that in the large $q$ limit, near to the coloring threshold, it is the number of clusters which change with connectivity (i.e. $\alpha$ ), and not their internal entropy (size). In the leading order, adding constraints near to the COL/UNCOL transition thus destroys clusters of solutions, but do not make them smaller: this is due to the fact that these clusters are dominated by frozen variables so that adding a link kills them most of the time. We also computed the entropy value at the condensation transition, and found $s(m=1)=\log{2}/q$ . The entropy of the last cluster (exactly at the COL/UNCOL transition) is $s=\log{2}/2q$ .

VI Algorithmic consequences

In this section we give some algorithmic consequences of our findings. First, we discuss the whitening procedure. We then introduce a random walk algorithm adapted from the Walk-SAT strategy and study its performance. We show in particular that the clustering/dynamical transition does not correspond to the onset of hardness in the problem and argue that it is instead the rigidity transition. Finally, we discuss the performance of the belief propagation algorithm in counting and finding solutions, and show that is works much better than previously anticipated.

VI.1 The whitening procedure

The whitening procedure as introduced in ParisiWhit can distinguish between solutions which belong to a cluster containing hard fields and those which do not. Generally whitening is equivalent to the warning propagation (version of belief propagation which distinguish only if a field is hard or not). Warning propagation for coloring was derived in Coloring . Let us call $u^{i\to j}=(1,0,0,\dots,0)$ the hard field in the direction of the first color, i.e. in absence of node $j$ the node $i$ takes only the first color in all the colorings belonging to the cluster in consideration, and similarly for other colors. Denote $u^{i\to j}=(0,0,0,\dots,0)$ if $\psi^{i\to j}$ is not frozen in the cluster, we say that the oriented edge $i\to j$ is then “white”. The update for $u$ ’s follows from (4)

[TABLE]

To see if a solution $\{s_{i}\}$ belongs to a cluster with frozen variables or not we initialize warning propagation with $u^{i\to j}_{s}=\delta_{s,s_{i}}$ , and update iteratively according to (49) until a fixed point is reached (the update every time converge, because starting from a solution we are only adding white edges). In the fixed point or all edges are white, then the solution $\{s_{i}\}$ does not belong to a frozen cluster, or some of the edges stay colored (non-white), then the solution $\{s_{i}\}$ belongs to a frozen cluster. Note that in the K-SAT problem (but not in general), whitening is equivalent to a more intuitive procedure, where the directed edged are not considered Elitza ; Riccardo .

We wish to offer here an explanation of a paradox observed in Elitza ; Riccardo . The SP algorithm gives information on the frozen variables in the most numerous clusters ( $m=0$ ). Yet, the solutions which are found by the standard implementation (decimation and SP plus Walk-SAT) do not belong to clusters with frozen variables, since they always give a trivial whitening result (all directed edges are white)Elitza ; Riccardo . We suggest that the decimation strategy drives the system towards a solution belonging to a large cluster, which does not contain frozen variables. In this case, it is logical that the result of the whitening is trivial, as it is observed. We believe this is reason why no nontrivial whitenings are observed so far in the study of the K-SAT problem on large graphs.

Note that beyond the rigidity transition this argument does not work anymore, since there all the clusters (for all $m$ such that $\Sigma(m)>0$ ) contain frozen variables. More precisely, for $q\geq 9$ we could in principle end up in soft-clusters even beyond the rigidity transition (since that one concerns only the dominant states), if this is possible is let for further investigation. Interestingly, in the coloring problem we have not been able to find solutions beyond the rigidity transition even with survey propagation algorithm (compare $c_{r}$ with the performance of SP in Coloring ). Further, more systematic, investigations have to be done about these issues, employing other strategies for the use of the survey propagation equations (for example the reinforcement reinforcement ).

VI.2 A Walk-COL algorithm to color random graph

In this paper, we have computed the correct clustering transition $c_{d}$ for the random coloring problem. Beyond this transition, Monte Carlo algorithms are proven not to reach equilibrium as their time of equilibration diverges MS05 ; MS06 . It was often claimed, or assumed, that this point corresponds to the onset of hardness of the problem. However, the fact that the physical dynamics does not equilibrate just means that the complete set of solutions will not be correctly sampled —indeed Monte-Carlo experiments clearly display slow relaxation MonteCarlo — but not that no solutions can be eventually found. This simple fact explains the results of ColoringSaad where a simple annealing procedure was shown to 3-color a ER graph beyond $c_{d}=4$ .

In this section, we use a local search strategy which does not satisfy the detailed balance condition. Therefore, we do not expect to be able to find typical solutions, however it might be possible to find some solutions to the problem. The Walk-COL algorithm 333A slightly different adaptation, closer to walksat , was performed few years ago by Andrea Pagnani and Martin Weigt (private communication). is a simple adaptation of the celebrated Walk-SAT walksat . More precisely, we adapted the method designed for satisfiability in John . Given a graph, and starting from an initial random configuration, we recursively apply the following procedure:

Choose at random a spin which is not satisfied (i.e. at least one of its neighbors has the same color). 2. 2)

Change randomly its color. Accept this change with probability one if the number of unsatisfied spin has been lowered, otherwise accept it with probability $p$ . 3. 3)

If there still are unsatisfied nodes, go to step 1) unless the maximum running time is reached

The probability $p$ has to be tuned in each different case for a better efficiency of the algorithm. Typically, values between $0.01-0.05$ give good results. We shall now briefly discuss the performance of the algorithm, to illustrate the two following points: (a) When the phase space is RS, we observe that Walk-COL finds a solution in linear time. (b) Even in the “complex” phase for $c>c_{d}$ , the algorithm can find in some cases solutions in linear time.

Concerning the first point, we tested the algorithm in the RS phase of regular random graphs for $q=3,4,5,6,7$ . In all these cases, we were able to color in linear time all the graphs of connectivities that correspond to a replica symmetric solution. In particular, the cases $(q=3,c=5)$ , $(q=5,c=13)$ , $(q=6,c=17)$ , $(q=7,c=21)$ , $(q=7,c=22)$ are found to be colorable with the Walk-COL algorithm even if a nontrivial solution to the SP equations exists.

Concerning the second point, we considered the $3-$ and $4-$ coloring of Erdős-Rényi random graphs. The results are shown in fig. 10 where the percentage of unsatisfied spins versus the number of attempted flips (averaged over $5$ different realizations) divided by $N$ is plotted. We observe that the curves corresponding to different values of $N$ superpose quite well (and that actually the results for $N=2\cdot 10^{5}$ are systematically lower than those for $N=5\cdot 10^{4}$ ) so that an estimation of the time needed to color a graph can be obtained. The connectivities of these graphs are beyond the dynamical transition ( $c_{d}=4$ for 3-coloring and $c_{d}=8.35$ for 4-coloring). It would be interesting to systematically test Walk-COL, as it has been done for Walk-SAT in John , to derive the precise connectivity at which it ceases to be linear.

Already these results show that the dynamical transition is not a problem for the algorithms. This can also be observed in a number of numerical experiments for the satisfiability walksat_whitening ; John and the coloring Jorge ; Stefan problems.

We believe, however, that the rigidity transition plays a fundamental role for the average computational complexity. A first argument for this is that, for large graphs, it seems that all the known algorithms are only able to find solutions with a trivial whitening, i.e. solutions that belong to clusters without hard fields. Beyond the rigidity transition however, the clusters without hard fields become very rare (in the sense that the dominating clusters and all the smaller, more numerous ones, contain hard fields). For $q\geq 9$ maybe the connectivity where hard fields appear in clusters corresponding to $\Sigma(m)=0$ , $m>1$ should be considered. This suggests that the known algorithms will not be able to find a solution beyond this point.

A second argument is the following: local search algorithms are either attracted into a solution or stucked in a metastable state. These metastable states, in order to be able to trap the dynamics, have to contain a finite fraction of hard fields. Given an algorithm, determining which of these two situations happens is not only a question of existence of states, but also a question of basins of attraction and a theoretical analysis of such basins is a very difficult task so that the precise analysis of the behavior of local algorithms remains a hard problem. However, the metastable states are known, from the cavity formalism, to be much more numerous than the zero-energy states. Moreover the basin of attraction of a zero-energy state that contains hard fields does not probably differ much from those of the metastable state (while, on the other hand, the basin of attraction of a zero-energy state which does not contain hard fields might be slightly different and arguably relatively larger). It thus seems to us reasonable that local algorithms will get trapped by the metastable states beyond the rigidity transition.

A similar conclusion was reached recently in Jorge where the recursive implementation of the Walk-COL algorithm was studied and found to be somehow simpler to analyze. Again the strategy was found to be efficient (with linear time with respect to the size of the graph) beyond $c_{d}$ but bellow $c_{r}$ . The precise algorithmic implications of the rigidity transition thus require further investigation, maybe in the lines of walksat_whitening ; Jorge .

VI.3 A belief propagation algorithm to color random graph

Another consequence of our results, that we already discussed shortly in US , is that the standard belief propagation (BP) algorithm gives correct marginals until the condensation transition. It is actually a simple algebraic fact that the 1RSB approach at $m=1$ gives the same results for the marginals (average probabilities over all clusters) as the simple RS approach (see appendix C). Moreover the log-number of solutions in clusters corresponding to $m=1$ is also equal to the RS one. This suggests to use the BP marginals (as was already suggested in BP-Pearl ) and a decimation procedure to find proper colorings. Compared with the SP algorithm which has computational complexity proportional to $q!$ (factorial) the BP is only $q$ . We have seen moreover that for large numbers of colors, the condensation point is very close to the COL/UNCOL transition, so that BP could be used in a large interval of connectivities.

As a simple application, we tested how the straightforward implementation of the BP algorithm plus decimation allows one to find solutions of $3-$ and $4-$ coloring of random Erdős-Rényi graphs. Note that in the $3-$ coloring the clustered but not condensed phase is missing, so the argumentation above does not concern this case. The algorithm works by iterating the following procedure:

(i)

Run BP on the graph for a given number $l$ of iterations.

(ii)

Consider the most biased variable, and color it with its most probable color.

Two problems have to be mentioned. The first one is rather trivial: since at the beginning all colors are symmetric, the first color had to be put at random. The second one is more serious and concerns the convergence of BP. Indeed, we saw that there is local instability in the BP (replica symmetric) equations at connectivity $c=4$ for the $3-$ coloring of random graphs, so that the BP equations do not converge. This seems to be a problem restricted to the $3-$ coloring, but even in the case of $4-$ or more coloring, the BP equations do not converge on the decimated graph when a finite fraction (typically few percent) of variables is fixed. The reason or that is to be understood.

Nevertheless, since we merely want to design an effective tool to solve the coloring problem, we choose to avoid this problem by fixing the number of iteration $l$ at each step and thus ignore the non-convergence. We tried the method on both the $3-$ and $4-$ coloring and obtained unexpectedly good results. We used the following protocol in the code: We first try to find a solution with $l=10$ . If we do not succeed, we restart with $l=20$ and once more with $l=40$ . We tried that on $10$ different samples for different connectivities. The probability to find a proper coloring with these conditions is shown in fig.11.

We thus observe that the BP strategy is able to find solutions, even beyond the condensation transition. This shows clearly that the decimation procedure is a nontrivial one, and that the problem is not really hard in that region of connectivities. Note that the SP algorithm plus decimation has been shown to work in the $3-$ coloring very well until about $4.60$ Coloring : our results are thus very close to those obtained using SP. This rises again the question of the rigidity transition $c_{r}=4.66$ , which might also be problematic for the decimated survey propagation solver.

VII Conclusions

Let us summarize the results. They are perhaps best illustrated looking back to the cartoon in fig. 1, where the importance of the size of clusters is evidenced. We find that the set of solutions of the $q$ -coloring problem undergoes the following transitions as the connectivity is increased:

(i)

At low connectivity, $c<c_{d}$ , many clusters might exist but they are very small and the measure over the set of solutions is dominated by the single giant cluster described by the replica symmetric approach. 2. (ii)

Only at the dynamical transition $c_{d}$ the giant cluster decomposes abruptly into an exponentially large number of clusters (pure states). For connectivities $c_{d}<c<c_{c}$ , the measure is dominated by an exponential number of clusters. Yet, the total number of solutions is given by the replica symmetric entropy (14), and the marginals are given by the fixed point of the replica symmetric equations (belief propagation) (4). Starting from this transition the uniform sampling of solution becomes hard. 3. (iii)

At connectivity $c_{c}$ the set of solutions undergoes a condensation transition, similar to the one appearing in mean field spin glasses. In the condensed phase the measure is dominated by finite number of the largest clusters. The total entropy is strictly smaller than the replica symmetric one and has a discontinuity in the second derivative at $c_{c}$ . 4. (iv)

When connectivity $c_{s}$ is reached, no more clusters exist: this is the COL/UNCOL transition. Note that the entropy of last existing clusters is strictly positive, and not given only by the contribution of the isolated nodes, leaves and other small subgraphs, the COL/UNCOL transition is thus discontinuous in entropy.

.

This picture is very similar to the well-known scenario of the glass transition in temperature, with the dynamical and glass (Kauzmann) transition GLASSTHEORY . In some cases, the main one being the 3-coloring of Erdős-Rényi graphs, the clustering and the condensation transition merge and a continuous transition take place at $c_{d}=c_{c}$ , which then coincide with the local instability of the replica symmetric solution. Interestingly the variational approach of GiulioRemiMartin is very precise near to the continuous clustering transition. Since the 3-SAT problem behaves in the same manner, this solves the apparent contradictions between the results of GiulioRemiMartin and Science .

In addition to the transitions describing the organization of clusters, another important phenomenon concerning the internal structure of clusters takes place. A finite fraction of frozen variables can appear in the clusters (a frozen variable takes the same color in all the solutions that belong to the cluster). We found that the fraction of such variables in each cluster undergoes a first order transition and jumps from zero to a finite fraction at a connectivity that depends on the size of the cluster. In particular:

(v)

There exists a critical connectivity $c_{r}$ (rigidity/freezing) at which the thermodynamically relevant clusters —those that dominate the Gibbs measure— start to contain a finite fraction of frozen variables.

The results above were obtained within the 1RSB scheme, but should not change when considering further steps of RSB (an exception might be the 3-coloring near to the clustering transition).

We discussed some algorithmic consequences of these transitions. First, the belief propagation algorithm is efficient in counting solution and estimating marginals until the condensation transition. More interestingly, it can also be used, just like survey propagation, together with a decimation procedure in order to find solutions as we numerically demonstrated. Secondly, the dynamical transition is not the one at which simple algorithms fail as we illustrated using the Walk-COL strategy. For the 3-coloring of ER graph, there is even a rigorous proof of algorithmic performance beyond $c_{d}=4$ and until $c=4.03$ AchlioptasMoore03 . We argued that, instead, the rigidity phenomenon is responsible for the onset of computational hardness. This is a major point that we hope to see more investigated in the future.

Our study opens a way to many new and promising investigations and developments. For instance, we wrote the equivalent of the survey propagation equations for general value of $m$ , which has a particularly simple form for $m=1$ (39). It would be interesting to use these equations to find solutions. The behavior at finite temperature and the performance of the annealing procedure are also of interest. It would furthermore be interesting to re-discuss other finite connectivity spin glass models like for instance the lattice glass models Rivoire in the light of our findings. The stability towards more steps of replica symmetry breaking, or the super-symmetric approach SUSY , should be further investigated. Finally, it would be interesting to combine the entropic and energetic approach to investigate the frozen variables in the meta-stable states. We hope that our results will stimulate the activity in these lines of thoughts.

Acknowledgements.

We thank Jorge Kurchan, Marc Mézard, Andrea Montanari, Federico Ricci-Tersenghi, Guilhem Semerjian and Riccardo Zecchina for cheerful and very fruitful discussions concerning these issues. The numerical computations were done on the cluster EVERGROW (EU consortium FP6 IST) at LPTMS, Orsay, and on the cluster DALTON at ESPCI, Paris. This work has been partially supported by EVERGROW (EU consortium FP6 IST).

Appendix A Stability of the paramagnetic solution

In this appendix, we show how to compute the stability of the paramagnetic solution towards the continuous appearance of a 1RSB solution. This happens, as usual for continuous transition, when the spin glass correlation length, or equivalently, the spin glass susceptibility, diverges. Obviously, the presence of the diverging correlation length invalidate the premise of the RS cavity method. Recall that the spin glass susceptibility is defined as

[TABLE]

and can be rewritten for the present purpose as

[TABLE]

where we consider the average over graphs, in the thermodynamic limit, where spins $s_{0}$ and $s_{d}$ are at distance $d$ . The factor $\gamma^{d}$ stands for the average number of neighbors at distance $d$ , when $d\ll\log N$ . Assuming that the limit for large $d$ of the summands in (51) exists (with the limit $N\to\infty$ performed first), we relate it to the stability parameter:

[TABLE]

Then the series in (51) is essentially geometric, and converges if and only if $\overline{\lambda}<1$ .

Using the fluctuation-dissipation theorem we relate the correlation $\langle s_{0}s_{d}\rangle_{c}$ to the variation of magnetization in $s_{0}$ , caused by an infinitesimal field in $s_{d}$ . Finally, using the fact that we perform the large- $N$ limit first, the variation above is dominated by the direct influence through the length- $d$ path between the two nodes, and this induces a “chain” relation: if the path involves the nodes $(d,d-1,\ldots,0)$ we have

[TABLE]

The stability parameter of the paramagnetic solution of the cavity equations towards small perturbations can be computed from the following Jacobian

[TABLE]

which gives the infinitesimal probability that a change in the input probability $\psi^{2\to 1}_{\sigma}$ will change the output probability $\psi^{1\to 0}_{\tau}$ . The index RS says that the expression has to be evaluated at the RS paramagnetic solution.

This matrix has only two different entries, all the diagonal elements are equal, and all the non-diagonal elements are also equal. As an immediate consequence all Jacobians commute and are thus simultaneously diagonalizable so that it will be sufficient to study the effect after one cavity iteration (one step in the chain). The matrix $T$ has only two distinct eigenvalues,

[TABLE]

The second eigenvalue corresponds to the homogeneous eigenvector $(1,1,...,1)$ and describes a fluctuation changing all $\psi^{2\to 1}_{\tau},\tau=1,...,q,$ by the same amount, and maintains the color symmetry. It is thus not likely to be the relevant one and we will see that indeed $\lambda_{2}=0$ . The first eigenvalue, however, is $(q-1)$ -fold degenerate and its eigenvectors are spanned by $(1,-1,0,...,0),\ (0,1,-1,0,...,0),\ \dots,\ (0,...,0,1,-1)$ . The corresponding fluctuations explicitly break the color symmetry, and are in fact the critical ones. Using the cavity recursion (4), the two derivatives simply read

[TABLE]

so that the values of the two eigenvalues evaluated at the RS solution, where all $\psi$ are equal to $1/q$ , are

[TABLE]

The stability parameter (52) is thus $\overline{\lambda}=\gamma\lambda_{1}^{2}$ and the critical temperatures bellow which the instability sets in are

[TABLE]

For regular and Erdős-Rényi graphs respectively. Thus at zero temperature the critical connectivities reads

[TABLE]

These results coincide perfectly with the numerical simulations of the cavity recursion of ColoringSaad . The analytical expressions equivalent to (58) were in fact first obtained in KS66 in the context of the reconstruction problem on trees as an upper bound for the Gibbs measure extremality, and its connection with the statistical physics approach was explained in MM05 . The case of bi-regular random graphs can be easily understood by noticing that two recursions should be considered, one with $\gamma=c_{1}-1$ , and one with $\gamma=c_{2}-1$ . As a consequence, the stability point is equivalent in this case to the one of a regular random graph with an effective connectivity equal to $c=1+\sqrt{\left(c_{1}-1\right)\left(c_{2}-1\right)}$ .

Another instability appears when $\gamma|\lambda_{1}|>1$ . This has been refered to as the modulation instability in Rivoire . Actually, this is the continuous instability towards the appearance of the anti-ferromagnetic order. Since at zero temperature $\lambda_{1}=(1-q)^{-1}$ , then for connectivities larger than $c_{mod}=q$ for random regular graphs (and $c_{mod}=q-1$ for Erdős-Rényi) the paramagnetic solution becomes unstable towards the anti-ferromagnetic order. However, this is correct if we study a tree with some given (and well chosen) boundary condition, but as noted in Rivoire , the anti-ferromagnetic solution in impossible on random graphs because of the existence of frustrating loops of arbitrary length. The cavity equations (4) can actually never converge towards an anti-ferromagnetic solution of a random graph. Instead, when iterating, the fields oscillate between different solutions (thus the name modulation).

In other words, although on a random tree with special boundary conditions there exists for $c>c_{mod}$ a nontrivial solution to the cavity recursion (for the Gibbs state is no longer unique (15)), this solution does not exist on a random graph (and the Gibbs state is still extremal (16)). Note that this instability could anyway be a source of numerical problems that can be overcome considering that the distribution of cavity fields ${\cal P}(\psi)$ over the ensemble of random graphs has to be symmetric in the color permutation. Another possibility is to randomly mix the new and old populations in the population dynamics so that the anti-ferromagnetic oscilations are destroyed.

Appendix B The relative sizes of clusters in the condensed phase

In this section, we introduce the Poisson-Dirichlet point process and we shortly review some of its important properties. We also sketch its deep connection with the size of clusters in the condensed phase. Poisson-Dirichlet (PD) point process is a set of points $\{x_{i}\}$ , $i=1,\dots,\infty$ such that $x_{1}>x_{2}>x_{3}>\dots$ and $\sum_{i=1}^{\infty}x_{i}=1$ . To construct these points we consider a Poisson process $\{y_{i}\}$ , $i=1,\dots,\infty$ of intensity measure $y^{-1-m^{*}}$ , $0<m^{*}<1$ (note that this measure is not a probability measure). We order the sequence $\{y_{i}\}$ in such a way that $y_{1}>y_{2}>y_{3}>\dots$ and define the PD point process as

[TABLE]

If we identify the parameter $m^{*}$ with the value for which the complexity is zero $\Sigma(m^{*})=0$ then $y_{i}$ is proportional to the number of solutions in cluster $i$ (or to $e^{-\beta F}$ for non-zero temperature), and $x_{i}$ is the size on that cluster relative to the total number of solutions. This connection was (on a non-rigorous level) understood in MPV , for more mathematical review see Talagrand . Note that that due to the permutation symmetry in graph coloring there are every time $q!$ copies of one clusters (different in the color permutation).

To get feeling about the PD statistics let us answer in fig. 12 to the following question: Given the value $m^{*}$ how many clusters do we need to cover fraction $r$ of solutions, in other words what is the smallest $k$ such that $\sum_{i=1}^{k}x_{i}>r$ ? The mathematical properties of the PD process are very clearly reviewed in PitmanYor . To avoid confusion, note at this point that the PD process we are interested in is the ${\rm PD}(m^{*},0)$ in the notation of PitmanYor . In the mathematical literature, it is often referred to the ${\rm PD}(0,\theta)$ without indexing by the two parameters. Let us remind two useful results. Any moment of any $x_{i}$ can be computed from the generating function

[TABLE]

where $\lambda\geq 0$ and the functions $\phi_{m^{*}}$ and $\psi_{m^{*}}$ are defined as

[TABLE]

Another relation is that the ratio of two consequent points $R_{i}=x_{i+1}/x_{i}$ , $i=1,2,\dots$ is distributed as $im^{*}R_{i}^{im^{*}-1}$ . In particular its expectation is $\mathbb{E}[R_{i}]=im^{*}/(1+im^{*})$ and the random variables $R_{i}$ are mutually independent. We used these relation to obtain data in figure 12.

Appendix C The 1RSB formalism at $m=1$ and the reconstruction equations

In this appendix we discuss the considerable simplification of eq. (28) that is obtained by working directly at $m=1$ . This was first remarked and proved in MM05 when dealing with the tree reconstruction problem (for a discussion of a case where the RS solution is not paramagnetic see THEM ). We first introduce the probability distribution of fields (20) averaged over the graph

[TABLE]

where $Z_{1}$ is computed from (20) as

[TABLE]

where $\overline{\psi}=\int{\rm d}\,\psi\,P(\psi)\,\psi$ . Generally, $\overline{\psi}$ is a solution of the RS equation (12), which is easily seen from (20), (28). Since the RS solution is the paramagnetic one $\overline{\psi}=1/q$ the form of $Z_{1}$ is particularly simple.

In the next step, we want to get rid of the term $Z_{0}$ in eq. (64). We thus introduce $q$ distribution functions $\overline{P}_{s}$

[TABLE]

It is then easy to show that if $\overline{\psi}=1/q$ then $\overline{P}_{s}(\psi)$ satisfies

[TABLE]

where

[TABLE]

We solve eq. (67) by population dynamics. In order to do this, one needs to deal with $q$ populations of $q$ -component fields, and to update them according to (67). Is is only a functional equation and not a double-functional as the general 1RSB equation (28). Moreover the absence of the reweighting term $Z_{0}^{m}$ simplifies the population dynamics algorithm significantly. Finally, it is important to note that the computational complexity here is the same as the one for regular and ER random graphs.

A crucial theorem is also proven in MM05 : the population dynamics of eq. (67) has a nontrivial solution if and only if it converge to a nontrivial solution starting from initial conditions:

[TABLE]

This shows that when a paramagnetic solution is found, then no other solutions exist.

Similar manipulations allow us to obtain the replicated free energy (21) which is equal in this case to the replica symmetric free energy (9), and the free energy (26) inside the corresponding states as

[TABLE]

where the normalization factors $Z_{0}^{i}$ , $Z_{0}^{ij}$ are defined by (6) and (8). The complexity follows from (24). Since the replicated free energy $\Phi(\beta,1)$ is equal, according to (27), to the total free energy, we showed the statement used several time in the paper, i.e. the total free energy (entropy) at $m=1$ is equal to the replica symmetric free energy (9).

Another important point is that one can write the recursion separating the hard and soft fields. In general, at zero temperature, we can write the distribution $\overline{P}_{q}(\psi_{s})$ in eq. (67) as

[TABLE]

Plugging this to eq. (67) and taking into account the initial condition (69) and color symmetry, we see that $\mu_{q,s}=q\eta\delta(q,s)$ , where $\eta$ satisfies

[TABLE]

On ER graphs, the sum can be performed analytically and one finds

[TABLE]

This equation can be solved iteratively starting from $\eta^{\rm init}=1/q$ . It is a very simple equation, as the one obtained for $m=0$ , which gives us a very efficient way to compute the fraction of hard fields at $m=1$ for both regular and ER graphs. Indeed $\eta$ is larger that zero only above a certain average connectivity $c_{r}(m=1)$ .

Appendix D Numerical Methods

In this section, we detail the numerical methods we used to solve the 1RSB equations (20,28), and the procedures used to generate the data. We use a population dynamics method, as introduced in MP99 ; MP03 , and model the distribution $P^{i\to j}(\psi^{i\to j})$ by a population of $N$ vectors $\psi^{i\to j}$ . To compute $P^{i\to j}(\psi^{i\to j})$ knowing the $P^{k\to i}(\psi^{k\to i})$ for all incoming $k$ we perform the 1RSB recursion in eq. (20) in two steps: (i) first we compute the new vectors $\psi^{i\to j}$ using the simple RS recursion in eq. (31) (this is the iterative step) and (ii) we take into account the weight $(Z_{0}^{i\to j})^{m}$ for each of the vectors (this is the reweighting step). For the reweighting we tried different strategies, two of them perform very well.

a)

For every field $\psi$ in the population, we keep its weight $Z_{0}$ . We then compute the cumulative distribution of weights $Z_{0}$ and sample uniformly the incoming fields. Using dichotomy we generate a random fields with its proper weight in $O(\log(N))$ steps. A complete iteration thus takes $O(N\log{N})$ steps.

b)

We compute $N$ new vectors and then we make a new population when we clone some of them while erasing others so that in this new population each field is present according to its weight (in principle, one can even change the size of the population, although we have not implemented this strategy). This second approach can be implemented in linear time (generating an ordered list of random numbers is a linear problem, see GENERATION ), but is a bit less precise as we introduce redundancy in the population.

We finally choose to use the second strategy, as we observed that it performs almost as good at the first one (for a given size of population) while it was much faster, so that, for given computer time, it allows a better representation of the population. We also force the population to be color-symmetric by adding a random shift of colors in the incoming messages. This is needed in order to avoid the anti-ferromagnetic solution. The learned reader will notice that this is equivalent to solving a disordered Potts glass instead of a anti-ferromagnet model. Indeed the fact that an Ising anti-ferromagnet on a random graph is equivalent to an Ising spin glass was already noticed FedeKrzakala .

Another important issue is the presence of hard fields. In fig. 13 are histograms of the first component of the vectors in the population for $3$ - and $4$ -coloring of $5$ - and $9$ -regular random graphs respectively. It is interesting to see how they peak around fractional values due to the presence of hard fields (see the three upper one). Maybe even more interesting are the lower one where no hard fields are present. However, since there are soft fields with values $1-\epsilon$ , where $\epsilon$ can be almost arbitrary small, one cannot see from these picture the absence of frozen variables. For the case $c=9,q=4,m=0.8$ for instance, the presence of the quasi-hard fields makes the distribution clearly concentrate on values around one, zero and half (note however that the amplitude —on a logarithmic scale— is far less important).

The quasi-hard fields are therefore very hard to distinguish numerically from the true hard ones. This is evidenced on fig. 14 where we plot the fraction of hard fields computed using the expression (38) together with a numerical estimate made by population dynamics without the separate hard/soft implementation. We show that the fraction of fields of value $1-\epsilon$ is not zero in regions where we know that there are no hard fields even for $\epsilon=10^{-20}$ . This demonstrates the presence of quasi-hard fields, with $\epsilon$ going to zero as the critical $m$ is approached. This transition is further studied in Guilhem_rear .

An important simplification of the 1RSB update (20) arises when we consider the soft and the hard fields separately. The fraction of hard fields can be computed using the generalized SP equation (38), provided the ratio ${\overline{Z^{m}_{\rm soft}}}/{\overline{Z^{m}_{\rm hard}}}$ is computed. This considerably reduced the size of the population as only soft field has to be kept in memory. Another way to further speed up the code is to generate directly soft fields with a uniform measure instead of waiting for them to come out from the 1RSB recursion. Indeed they might be quite rare in the region of small $m$ and one can spend a considerable amount of time before being able to sample them correctly. Generating the soft fields with a uniform weight turns out to be rather easy using the following method: (i) Choose two random colors $q_{1}$ and $q_{2}$ . (ii) Perform the usual recursion (31) in order to have a new vector but forbid incoming hard fields to $q_{1}$ and $q_{2}$ . (iii) To obtain a uniform soft field generator, the resulting field should be weighted by $1/{s\choose 2}$ where $s$ is the number of non-null components in the vector. This is specially useful in the case of Erdős-Rényi graphs.

The formula for the free entropy $\Phi_{s}(m)$ (32) also simplifies in this case. Consider a given site $i$ ; the site free entropy term can be split into three parts when (i) the field is hard, (ii) the total field is soft and (iii) the field is contradictory. Then

[TABLE]

where $p_{\rm hard}$ / $p_{\rm soft}$ are the probabilities that the total field is frozen/soft, and are given by the SP recursion. Indeed the probability that the total field is not contradictory ( $p_{\rm hard}+p_{\rm soft}$ ) is the denominator in eq. (35) while $p_{\rm hard}$ is the numerator of eq. (35). The link part can also be simplified using the fact that contradictions arise when two incoming frozen messages of the same color are chosen, so that

[TABLE]

where $p_{\rm no~{}contr}$ is simply $(1-q\eta^{i\to j}\eta^{j\to i})$ .

For $m=0$ the formula further simplify as $Z_{\rm hard}^{m}=Z_{\rm soft}^{m}=Z_{\rm no~{}contr}^{m}=1$ so that

[TABLE]

This is precisely what was obtained within the energetic cavity approach in Coloring . The numerical population dynamics implementation with mixed hard/soft strategy is therefore as precise as it could be since we obtain the exact evaluation in the $m=0$ case. This simple computation also demonstrates how one can recover the energetic zero temperature limit from the generic formalism.

Finally, we obtain the function $\Phi_{s}(m)$ . We fit this function using an ansatz $\Phi_{s}(m)=a+b2^{m}+c3^{m}\ldots$ and then perform the Legendre transform to obtain the entropy and complexity. It is also possible to compute directly the complexity from the population data using the expression of the derivative of the potential directly in the code. Both methods lead to very good results. We show an example of the raw data and their fit in fig. 15, where the data have been obtained with relatively small population $(N=5000)$ but where the mixed strategy separating the hard and soft fields have been used. For the purely soft-field branch, we used $N=50000$ . It took few hours up to few days to generate these curves on present Intel PCs.

In the case of bi-regular random graphs, one needs two different populations: one for the fields going from nodes with connectivity $c_{1}$ and one for the fields going from nodes with connectivity $c_{2}$ . Then each iteration for population $1$ (resp. $2$ ) should be performed using as incoming messages the vectors of population $2$ (resp. 1.). Again, one can separately perform the recursion for the hard-field fractions in both population.

The case of Erdős-Rényi random graphs is more involved, as one needs a large number $N_{\rm pop}$ of populations, each of them of size $N$ . In this case, using the separate hard/soft fields implementation and the formulae (74,75) for complexity is crucial, as it allows a good precision even for smaller population sizes. We used typically $2N_{\rm pop}/c\approx(1-3)\cdot 10^{3}$ and $N\approx(1-3)\cdot 10^{2}$ . The error bars in table 2 are computed from several independent runs of the population dynamics. In each case we were able to make the equilibration times and the population sizes large enough such that by doubling the time or the population size we did not observed any significant systematic changes in the average results.

Appendix E High- $q$ asymptotics

The quenched averages in the large $q$ limit are the same for the regular and Erdős-Rényi graphs and we thus consider directly the regular ensemble of connectivity $c=k+1$ . The appearance of a nontrivial 1RSB solution for $m=0$ , which correspond to $c_{\rm SP}$ , was already computed in ColoringFlo and reads

[TABLE]

while the coloring threshold is ColoringFlo

[TABLE]

We now show how the connectivity where a solution with hard fields at $m=1$ first appears, and how the complete free entropy $\Phi_{s}(m)$ (30) can be computed close to the COL/UNCOL transition

E.1 The appearance of hard fields at $m=1$

We first show that the correct scaling for the appearance of hard fields at $m=1$ is

[TABLE]

and compute the value of $\alpha$ . In the order $O(q)$ we can write also $k=(q-1)[\log(q-1)+\log\log(q-1)+\alpha]$ . The starting point is the equation (72), with ${\cal Q}_{1}(x)=\delta(x-k)$ . In the large $q$ limit the fraction of hard fields is $\mu(q,k)=q\eta(q,k)=1-\theta(q,k)$ , where $\theta(q,k)=o(1)$ is the fraction of soft fields. We check self-consistently at the end of the computation that only the two first terms of (72) are important. Then we have

[TABLE]

A self-consistent equation for $\theta(q,k)$ follows

[TABLE]

which is solved by $\theta(q,k)=\gamma(\alpha)/\log(q-1)$ where

[TABLE]

The maximum of the left hand side is $1/e$ for $\gamma(\alpha)=1$ . It means that a solution of (84) exists for $\alpha>1$ . Finally the hard fields appear in the 1RSB solution for $m=1$ at connectivity

[TABLE]

The clustering transition $c_{d}$ should be between $c_{\rm SP}$ and $c_{r}(m=1)$ as this is what we observed for finite $q$ . We see that $c_{\rm SP}$ and $c_{r}(m=1)$ differs only in the third order and both are very far from the coloring threshold and also from the condensation transition as we show in section E.2. It would be interesting to compute a large $q$ expansion of the connectivity at which the hard fields appears in all the clusters (for all $m$ such that $\Sigma(m)>0$ ). Together with our conjecture about rigidity being responsible for the computational hardness that might give a hint about the answer on the long-standing question FrMcDi : “Is there a polynomial algorithm and $\epsilon$ such that the algorithm would color random graphs of average connectivity $(1+\epsilon)q\log q$ for all large $q$ ?”

E.2 The condensation transition

To compute the large- $q$ asymptotic of the condensation transition, we first need to derive the large- $q$ expansion of the free entropy (30) in the connectivity regime $c=2q\log{q}$ . Let us show self-consistently that the following scaling is relevant for the condensation transition in the large $q$ limit

[TABLE]

and compute the constants $\gamma$ , $\alpha$ , $B$ . Using the above scaling, the function $w(\eta)$ (35) is dominated by the first two terms in numerator and denominator, and reads in the first two leading orders $1-qw(\eta)\approx\frac{qe^{-k\eta}}{2}$ so that

[TABLE]

independently of $\gamma$ , $\alpha$ , and $B$ . To take into account the reweighting we expand eq. (38) in the two leading orders

[TABLE]

Note that almost all the incoming fields are hard, i.e. have one component of value $1$ . Since there are on average only $2B\log{q}$ incoming soft fields, the leading order of the hard-field reweighting (the normalization in eq. (4)) is different from $1$ with a probability only $O(\log{q}/q)$ . Similarly, almost all the soft fields have two nonzero and equal components. The normalization in eq. (4) is thus almost surely $2$ , thus the average reweighting factor of the soft fields is

[TABLE]

Finally,

[TABLE]

Therefore the constant $B$ in (87) is $B=2^{m}/2$ , independently of $\gamma$ and $\alpha$ .

The computation of the complexity requires the next order in the hard-field reweighting. Indeed the normalization in (4) might not be $1$ but $1/2$ ; and this happens when there is a soft field arriving of the color corresponding to the hard field in consideration. The probability of this event is $\frac{2c\left(1-q\eta\right)}{q}=O(\frac{q}{\log{q}})$ . The hard-field reweighting is thus

[TABLE]

We now expand the replicated free energy (30) in the large $q$ limit and regime (86). Remind that from (6, 8)

[TABLE]

The averages are over the population in the sense of (21).

The site free energy is the logarithm of the average of the total field normalization. This average can be split into three parts when (i) the total field is a hard field, (ii) the total field is a soft field and (iii) the total field is contradictory (and its normalization zero). The probability that the total field is not contradictory is the denominator in eq. (35)

[TABLE]

where again only the first two terms are relevant in the expansion. The site free energy is then

[TABLE]

where

[TABLE]

To compute the link contribution in (93) we need to consider two fields $\psi_{s}^{i\to j}$ and $\psi_{s}^{j\to i}$ and to compute

[TABLE]

There are three different cases:

Two hard fields are chosen, then $Z^{ij}_{0}=0$ with probability $q\eta^{2}$ (this is of order $1/q$ ) and $Z^{ij}_{0}=1$ with probability $q(q-1)\eta^{2}$ (this is of order $1$ ). 2. 2.

Two soft fields are chosen then $Z^{ij}_{0}=1$ with probability $(1-q\eta)^{2}$ (this is of order $1/q^{2}$ ), all other situations being $O(1/q^{3})$ . Let us remind that the dominant soft fields are two-component of type $(1/2,1/2,0,0,\dots)$ . 3. 3.

One hard and one soft field is chosen, then $Z^{ij}_{0}=1$ with probability $2\eta(1-q\eta)(q-2)/q$ , and $Z^{ij}_{0}=1/2$ with probability $4\eta(1-q\eta)$ (this is of order $1/q^{2}$ ).

On average, one thus obtain for the link contribution

[TABLE]

Putting together the two pieces (95) and (99), expanding $\eta$ according to (91) and considering only the highest order in $c$ , we can finally write the free energy as

[TABLE]

The internal entropy $s(m)$ and the complexity $\Sigma=\Phi_{s}(m)-ms(m)$ are then

[TABLE]

and the complexity is thus zero for $c_{\Sigma=0}=2q\log{q}-\log{q}-2+2^{m}\left[1-m\log{2}\right]+o(1)$ . In particular, one has for the coloring and the condensation thresholds

[TABLE]

For connectivity $c=2q\log{q}-\log{q}+\alpha$ , one gets

[TABLE]

Bibliography84

The reference list from the paper itself. Each links out to its DOI / PubMed record.

1(1) S. Cook, Proceedings of the 3rd Annual ACM Symposium. on Theory of Computing (ACM, New York, 1971), pp. 151-158. M. R. Garey and D. S. Johnson Computers and Intractability: A Guide to the Theory of NP-Completeness , (Freeman, New York, 1979). C.H Papadimitrious, Computational complexity (Addison-Wesley, 1994).
2(2) K. Appel and W. Haken, Illinois J. Math. 21 , 421 (1977). K. Appel and W. Haken, Illinois J. Math. 21 , 491 (1977).
3(3) R. E. Miller and J. W. Thatcher (eds.), Complexity of Computer Computation, Plenum Press. New York, 85 (1972).
4(4) B. Bollobás, Random Graphs (CUP, Cambridge, 2001).
5(5) S.Janson, T. Luczak and A.Rucinski, Random Graphs , John Wiley and Sons, 2000.
6(6) P. Erdős and A. Rényi, Publ. Math. (Debrecen) 6 , 290 (1959).
7(7) D. Achlioptas and E. Friedgut, Random Structures and Algorithms, Volume 14, Issue 1, Pages 63 - 70 (1999).
8(8) D. Achlioptas, A. Naor, Y. Peres, Nature, 435 (2005), 759-764.

TL;DR

Contribution

Findings

Abstract

Peer Reviews

Videos

Phase Transitions in the Coloring of Random Graphs

Abstract

pacs:

I Introduction

II The Model

II.1 Definition of the model

II.2 Ensembles of Random Graphs

III The cavity formalism at the replica symmetric level

III.1 The replica symmetric cavity equations

III.2 Average over the ensemble of graphs and the RS solution

III.3 Validity conditions of the replica symmetric solution

III.3.1 The Gibbs measure uniqueness condition

III.3.2 The Gibbs measure extremality condition

III.3.3 The local stability: a simple self-consistency check

IV One-step replica symmetry breaking framework

IV.1 Analyzing the 1RSB equations

IV.2 Zero temperature limit

IV.3 The role of frozen variables

IV.3.1 Hard fields in the simplest case, m=0m=0m=0

IV.3.2 Generalized survey propagation recursion

IV.3.3 The presence of frozen variables

IV.4 Validity conditions of the 1RSB solution

V The coloring of random graphs: cavity results

V.1 Regular random graphs

V.2 Results for the bi-regular ensemble

V.3 Results for Erdős-Rényi random graphs

V.3.1 The general case for q>3q>3q>3, discontinuous clustering transition

V.3.2 The special case of 3−3-3−coloring, continuous clustering transition

V.3.3 The overlap structure

V.4 Large q Asymptotics

VI Algorithmic consequences

VI.1 The whitening procedure

VI.2 A Walk-COL algorithm to color random graph

VI.3 A belief propagation algorithm to color random graph

VII Conclusions

Acknowledgements.

Appendix A Stability of the paramagnetic solution

Appendix B The relative sizes of clusters in the condensed phase

Appendix C The 1RSB formalism at m=1m=1m=1 and the reconstruction equations

Appendix D Numerical Methods

Appendix E High-qqq asymptotics

E.1 The appearance of hard fields at m=1m=1m=1

E.2 The condensation transition

IV.3.1 Hard fields in the simplest case, $m=0$

V.3.1 The general case for $q>3$ , discontinuous clustering transition

V.3.2 The special case of $3-$ coloring, continuous clustering transition

Appendix C The 1RSB formalism at $m=1$ and the reconstruction equations

Appendix E High- $q$ asymptotics

E.1 The appearance of hard fields at $m=1$