The Power of Self-Reducibility: Selectivity, Information, and Approximation

TL;DR

This paper offers a tutorial on self-reducibility, guiding readers through proofs of key theorems related to selectivity, information, and approximation, aiming to enhance understanding and teaching of this proof technique.

Contribution

It provides a hands-on tutorial and lesson plan to help readers learn and apply self-reducibility in complexity theory proofs, especially for educational purposes.

Findings

Guides readers to discover proofs using self-reducibility

Includes challenge problems related to selectivity and approximation

Provides educational resources for teaching complexity theory

Abstract

This chapter provides a hands-on tutorial on the important technique known as self-reducibility. Through a series of "Challenge Problems" that are theorems that the reader will---after being given definitions and tools---try to prove, the tutorial will ask the reader not to read proofs that use self-reducibility, but rather to discover proofs that use self-reducibility. In particular, the chapter will seek to guide the reader to the discovery of proofs of four interesting theorems---whose focus areas range from selectivity to information to approximation---from the literature, whose proofs draw on self-reducibility. The chapter's goal is to allow interested readers to add self-reducibility to their collection of proof tools. The chapter simultaneously has a related but different goal, namely, to provide a "lesson plan" (and a coordinated set of slides is available online to support this use [Hem19]) for a lecture to a two-lecture series that can be given to undergraduate students---even those with no background other than basic discrete mathematics and an understanding of what polynomial-time computation is---to immerse them in hands-on proving, and by doing that, to serve as an invitation to them to take courses on Models of Computation or Complexity Theory.

Equations

$$F(x_{1},x_{2},\dots,x_{k})\in{\rm SAT}\Longleftrightarrow\bigl{(}F(\mathrm{True},x_{2},\dots,x_{k})\in{\rm SAT}\,\lor\,F(\mathrm{False},x_{2},\dots,x_{k})\in{\rm SAT}\bigr{)}.$$

$$(\forall a,b\in\Sigma^{\ast})[f(a,b)\in\{a,b\}\land\bigl{(}\{a,b\}\cap L\neq\emptyset\implies f(a,b)\in L\bigr{)}].$$

$$(\forall x\in {\Sigma }^{\ast })[x\in A⟺g(x)\in B].$$

$$\Vert \{x\mid x\in S\wedge |x|\le n\}\Vert \le q(n).$$

$$F(\mathrm{True},x_2,\dots ,x_k)\in \mathrm{SAT}⟺F(\mathrm{False},x_2,\dots ,x_k)\in \mathrm{SAT}.$$

$$g(F(\mathrm{True},x_2,\dots ,x_k))\in T⟺F(\mathrm{True},x_2,\dots ,x_k)\in \mathrm{SAT}$$

$$g(F(\mathrm{False},x_2,\dots ,x_k))\in T⟺F(\mathrm{False},x_2,\dots ,x_k)\in \mathrm{SAT}.$$

$$H=(F\wedge z)\vee (\bar{z}\wedge x_1\wedge \cdots \wedge x_n\wedge G\wedge z^{\prime })$$

$$H=\mathrm{combiner}(F,\mathrm{combiner}(F_{left},F_{right})),$$

Full text

The chapter’s goal is to allow interested readers to add self-reducibility to their collection of proof tools. The chapter simultaneously has a related but different goal, namely, to provide a “lesson plan” (and a coordinated set of slides is available online to support this use [Hem19]) for a lecture to a two-lecture series that can be given to undergraduate students—even those with no background other than basic discrete mathematics and an understanding of what polynomial-time computation is—to immerse them in hands-on proving, and by doing that, to serve as an invitation to them to take courses on Models of Computation or Complexity Theory.

In memory of Ker-I Ko, whose indelible contributions to computational complexity included important work (e.g., [KM81, Ko82, Ko83, Ko87]) on each of this chapter’s topics: self-reducibility, selectivity, information, and approximation.

1 Introduction

Section 1.1 explains the two quite different audiences that this chapter is intended for, and for each describes how that group might use the chapter. If you’re not a computer science professor it would make sense to skip Section 1.1.2, and if you are a computer science professor you might at least on a first reading choose to skip Section 1.1.1

Section 1.2 introduces the type of self-reducibility that this chapter will focus on, and the chapter’s central set, ${\rm SAT}$ (the satisfiability problem for propositional Boolean formulas).

1.1 A Note on the Two Audiences, and How to Read This Chapter

This chapter is unusual in that it has two intended audiences, and those audiences differ dramatically in their amounts of background in theoretical computer science.

1.1.1 For Those Unfamiliar with Complexity Theory

The main intended audience is those—most especially young students—who are not yet familiar with complexity theory, or perhaps have not even yet taken a models of computation course. If that describes you, then this chapter is intended to be a few-hour tutorial immersion in—and invitation to—the world of theoretical computer science research. As you go through this tutorial, you’ll try to solve—hands-on—research issues that are sufficiently important that their original solutions appeared in some of theoretical computer science’s best conferences and journals.

You’ll be given the definitions and some tools before being asked to try your hand at finding a solution to a problem. And with luck, for at least a few of our four challenge problems, you will find a solution to the problem. Even if you don’t find a solution for the given problem—and the problems increase in difficulty and especially the later ones require bold, flexible exploration to find possible paths to the solution—the fact that you have spent time trying to solve the problem will give you more insight into the solution when the solution is then presented in this chapter.

A big-picture goal here is to make it clear that doing theoretical computer science research is often about playful, creative, flexible puzzle-solving. The underlying hope here is that many people who thought that theoretical computer science was intimidating and something that they could never do or even understand will realize that they can do theoretical computer science and perhaps even that they (gasp!) enjoy doing theoretical computer science.

The four problems also are tacitly bringing out a different issue, one more specifically about complexity. Most people, and even most computer science professors, think that complexity theory is extraordinarily abstract and hard to grasp. Yet in each of our four challenge problems, we’ll see that doing complexity is often extremely concrete, and in fact is about building a program that solves a given problem. Building programs is something that many people already have done, e.g., anyone who has taken an introduction to programming course or a data structures course. The only difference in the programs one builds when doing proofs in complexity theory is that the programs one builds typically draw on some hypothesis that provides a piece of the program’s action. For example, our fourth challenge problem will be to show that if a certain problem is easy to approximate, then it can be solved exactly. So your task, when solving it, will be to write a program that exactly solves the problem. But in writing your program you will assume that you have as a black box that you can draw on as a program (a subroutine) that given an instance of the problem gives an approximate solution.

This view that complexity is largely about something that is quite concrete, namely building programs, in fact is the basis of an entire graduate-level complexity-theory textbook [HO02], in which the situation is described as follows:

Most people view complexity theory as an arcane realm populated by pointy-hatted (if not indeed pointy-headed) sorcerers stirring cauldrons of recursion theory with wands of combinatorics, while chanting incantations involving complexity classes whose very names contain hundreds of characters and sear the tongues of mere mortals. This stereotype has sprung up in part due to the small amount of esoteric research that fits this bill, but the stereotype is more strongly attributable to the failure of complexity theorists to communicate in expository forums the central role that algorithms play in complexity theory.

Expected Background

To keep this chapter as accessible as possible, the amount of expected background has been kept quite small. But there are some types of background that are being assumed here. The reader is assumed to know following material, which would typically be learned within about the first two courses of most computer science departments’ introductory course sequences.

What a polynomial is.

As an example, $p(n)=n^{1492}+42n^{42}+13$ is a polynomial; $e(n)=2^{n}$ is not. 2. 2.

What it means for a set or function to be computable in polynomial time, i.e., to be computed in time polynomial in the number of bits in the input to the problem. The class of all sets that can be computed in polynomial time is denoted P, and is one of the most important classes in computer science.

As an example, the set of all positive integers that are multiples of 10 is a set that belongs to ${\rm P}$. 3. 3.

Some basics of logic such as the meaning of quantifiers ($\exists$ and $\forall$) and what a (propositional) Boolean formula is.

As an example of the latter, the formula $x_{1}\land(x_{2}\lor\overline{x_{3}})$ is a such a formula, and evaluates as $\mathrm{True}$—with each of $x_{1}$, $x_{2}$, and $x_{3}$ being variables whose potential values are $\mathrm{True}$ or $\mathrm{False}$—exactly if $x_{1}$ is $\mathrm{True}$ and either $x_{2}$ is $\mathrm{True}$ or the negation of $x_{3}$ is $\mathrm{True}$.

If you have that background in hand, wonderful! You have the background to tackle this chapter’s puzzles and challenges, and please (unless you’re a professor thinking of modeling a lecture series on this chapter) skip from here right on to Section 1.2.

1.1.2 For Computer Science Professors

Precisely because this chapter is designed to be accessible and fun for students who don’t have a background in theoretical computer science, the chapter avoids—until Section 7—trying to abstract away from the focus on ${\rm SAT}$. In particular, this chapter either avoids mentioning complexity class names such as NP, coNP, and PSPACE, or at least, when it does mention them, uses phrasings such as “classes known as” to make clear that the reader is not expected to possess that knowledge.

Despite that, computer science professors are very much an intended audience for this chapter, though in a way that is reflecting the fact that the real target audience for these challenges is young students. In particular, in addition to providing a tutorial introduction for students, of the flavor described in Section 1.1.1, this chapter also has as its goal to provide to you, as a teacher, a “lesson plan” to help you offer in your course a one- or two-day lecture (but really hands-on workshop) sequence111To cover all four problems would take two class sessions. Covering just the first two or perhaps three of the problems could be done in a single 75-minute class session. in which you present the definitions and tools of the first of these problems, and then ask the students to break into groups and in groups spend about 10–25 minutes working on solving the problem,222In this chapter, since student readers of the chapter will be working as individuals, I suggest to the reader, for most of the problems, longer amounts of time. But in a classroom setting where students are working in groups, 10–25 minutes may be an appropriate amount of time; perhaps 10 minutes for the first challenge problem, 15 for the second, 15 for the third, and 25 for the fourth. You’ll need to yourself judge the time amounts that are best, based on your knowledge of your students. For many classes, the just-mentioned times will not be enough. Myself, I try to keep track of whether the groups seem to have found an answer, and I will sometimes stretch out the time window if many groups seem to be still working intensely and with interest. Also, if TAs happen to be available who don’t already know the answers, I may assign them to groups so that the class’s groups will have more experienced members, though the TAs do know to guide rather than dominate a group’s discussions. and then you ask whether some group has found a solution and would like to present it to the class, and if so you and the class listen to and if needed correct the solution (and if no group found a solution, you and the class will work together to reach a solution). And then you go on to similarly treat the other three problems, again with the class working in teams. This provides students with a hands-on immersion in team-based, on-the-spot theorem-proving—something that most students never get in class. I’ve done this in classes of size up to 79 students, and they love it. The approach does not treat them as receptors of information lectured at them, but rather shows them that they too can make discoveries—even ones that when first obtained appeared in such top forums as CCC (the yearly Computational Complexity Conference), ICALP (the yearly International Colloquium on Automata, Languages, and Programming), the journal Information and Computation, and SIAM Journal on Computing.

To support this use of the chapter as a teaching tool in class, I have made publicly available a set of LaTeX/Beamer slides that can be used for a one- or two-class hands-on workshop series on this chapter. The slides are available online [Hem19], both as pdf slides and, for teachers who might wish to modify the slides, as a zip archive of the source files.

Since the slides might be used in courses where students already do know of such classes as NP and coNP, the slides don’t defer the use of those classes as aggressively as this chapter itself does. But the slides are designed so that the mention of the connections to those classes is parenthetical (literally so—typically a parenthetical, at the end of a theorem statement, noting the more general application of the claim to all of NP or all of ${\rm coNP}$), and those parentheticals can be simply skipped over. Also, the slides define on the fly both NP and coNP, so that if you do wish to cover the more general versions of the theorems, the slides will support that too.

The slides don’t themselves present the solutions to Challenge Problems 1, 2, or 3. Rather, they assume that one of the class’s groups will present a solution on the board (or document camera) or will speak the solution with the professor acting as a scribe at the board or the document camera. Challenge Problems 1, 2, and 3 are doable enough that usually at least one group will either have solved the question, or at least will made enough progress that, with some help from classmates or some hints/help from the professor, a solution can quickly be reached building on the group’s work. (The professor ideally should have read the solutions in this chapter to those problems, so that even if a solution isn’t reached or almost reached by the students on one or two of those problems, the professor can provide a solution at the board or document camera. However, in the ideal case, the solutions of those problems will be heavily student-driven and won’t need much, if any, professorial steering.)

Challenge Problem 4 is sufficiently hard that the slides do include both a slide explaining why a certain very natural approach—which is the one students often (quite reasonably) try to make work—cannot possibly work, and thus why the approach that the slides gave to students as a gentle, oblique hint may be the way to go, and then the slides present a solution along those lines.

The difficulty of Challenge Problem 4 has a point. Although this chapter is trying to show students that they can do theory research, there is also an obligation not to give an artificial sense that all problems are easily solved. Challenge Problem 4 shows students that some problems can have multiple twists and turns on the way to finding a solution. Ideally, the students won’t be put off by this, but rather will appreciate both that solving problems is something they can do, and that in doing so one may well run into obstacles that will take some out-of-the-box thinking to try to get around—obstacles that might take not minutes of thought but rather hours or days or more, as well as working closely with others to share ideas as to what might work.

1.2 Self-Reducibility and SAT

Now that you have read whatever part of Section 1.1 applied to you, to get the lay of the land as to what this chapter is trying to provide you, let us discuss the approach that will be our lodestar throughout this chapter.

One of the most central paradigms of computer science is “divide and conquer.” Some of the most powerful realizations of that approach occur though the use of what is known as self-reducibility, which is our chapter’s central focus.

Loosely put, a set is self-reducible if any membership question regarding the set can be easily resolved by asking (perhaps more than one) membership questions about smaller strings.

That certainly divides, but does it conquer?

The answer varies greatly depending on the setting. Self-reducibility itself, depending on which polynomial-time variant one is looking at, gives upper bounds on a set’s complexity. However, those bounds—which in some cases are the complexity classes known as NP and PSPACE—are nowhere near to putting the set into deterministic polynomial time (aka, ${\rm P}$).

The magic of self-reducibility, however, comes when one adds another ingredient to one’s stew. Often, one can prove that if a set is self-reducible and has some other property regarding its structure, then the set is feasible, i.e., is in deterministic polynomial time.

This tutorial will ask the reader to—and help the reader to—discover for him- or herself the famous proofs of three such magic cases (due to Selman, Berman, and Fortune), and then of a fourth case that is about the “counting” analogue of what was described in the previous paragraph.

Beyond that, I hope you’ll keep the tool/technique of self-reducibility in mind for the rest of your year, decade, and lifetime—and on each new challenge will spend at least a few moments asking, “Can self-reducibility play a helpful role in my study of this problem?” And with luck, sooner or later, the answer may be, “Yes! Goodness… what a surprise!”

Throughout this chapter, our model language (set) will be “${\rm SAT}$,” i.e., the question of whether a given Boolean formula, for some way of assigning each of its variables to $\mathrm{True}$ or to $\mathrm{False}$, evaluates to $\mathrm{True}$. ${\rm SAT}$ is a central problem in computer science, and possesses a strong form of self-reducibility. As a quiet bonus, though we won’t focus on this in our main traversal of the problems and their solutions, ${\rm SAT}$ has certain “completeness” properties that make results proven about ${\rm SAT}$ often yield results for an entire important class of problems known as the “NP-complete” sets; for those interested in that, Section 7, “Going Big: Complexity-Class Implications,” briefly covers that broader view.

2 Definitions Used Throughout: SAT and Self-Reducibility

The game plan of this chapter, as mentioned above, is this: For each of the four challenge problems (theorems), you will be given definitions and some other background or tools. Then the challenge problem (theorem) will be stated, and you’ll be asked to try to solve it, that is, you’ll be asked to prove the theorem. After you do, or after you hit a wall so completely that you feel you can’t solve the theorem even with additional time, you’ll read a proof of the theorem. Each of the four challenge problems has an appendix presenting a proof of the result.

But before we start on the problems, we need to define ${\rm SAT}$ and discuss its self-reducibility.

Definition 1**.**

${\rm SAT}$* is the set of all satisfiable (propositional) Boolean formulas.*

Example 1**.**

$x\land\overline{x}\not\in{\rm SAT}$, since neither of the two possible assignments to $x$ causes the formula to evaluate to $\mathrm{True}$. 2. 2.

$(x_{1}\land x_{2}\land\overline{x_{3}})\lor(x_{4}\land\overline{x_{4}})\in{\rm SAT}$, since that formula evaluates to $\mathrm{True}$ under at least one of the eight possible ways that the four variables can each be assigned to be $\mathrm{True}$ or $\mathrm{False}$. For example, when we take $x_{1}=x_{2}=x_{4}=\mathrm{True}$ and $x_{3}=\mathrm{False}$, the formula evaluates to $\mathrm{True}$.

${\rm SAT}$ has the following “divide and conquer” property.

Fact 1** (2-disjunctive length-decreasing self-reducibility).**

Let $k\geq 1$. Let $F(x_{1},x_{2},\dots,x_{k})$ be a Boolean formula (without loss of generality, assume that each of the variables actually occurs in the formula). Then

[TABLE]

The above fact says that SAT is self-reducible (in particular, in the lingo, it says that ${\rm SAT}$ is 2-disjunctive length-decreasing self-reducible).

Note: We won’t at all focus in this chapter on details of the encoding of formulas and other objects. That indeed is an issue if one wants to do an utterly detailed discussion/proof. But the issue is not a particularly interesting one, and certainly can be put to the side during a first traversal, such as that which this chapter is inviting you to make.

A Bit of History

In this chapter, we typically won’t focus much on references. It is best to immerse oneself in the challenges, without getting overwhelmed with a lot of detailed historical context. However, so that those who are interested in history can have some sense of the history, and so that those who invented the concepts and proved the theorems are property credited, we will for most sections have an “A Bit of History” paragraph that extremely briefly gives literature citations and sometimes a bit of history and context. As to the present section, self-reducibility dates back to the 1970s, and in particular is due to the work of Schnorr [Sch76] and Meyer and Paterson [MP79].

3 Challenge Problem 1: Is SAT even Semi-Feasible?

Pretty much no one believes that ${\rm SAT}$ has a polynomial-time decision algorithm, i.e., that ${\rm SAT}\in{\rm P}$ [Gas19]. This section asks you to show that it even is unlikely that ${\rm SAT}$ has a polynomial-time semi-decision algorithm—a polynomial-time algorithm that, given any two formulas, always outputs one of them and does so in such a way that if at least one of the input formulas is satisfiable then the formula that is output is satisfiable.

3.1 Needed Definitions

A set $L$ is said to be feasible (in the sense of belonging to P) if there is a polynomial-time algorithm that decides membership in $L$.

A set is said to be semi-feasible (aka P-selective) if there is a polynomial-time algorithm that semi-decides membership, i.e., that given any two strings, outputs one that is “more likely” to be in the set (to be formally cleaner, since the probabilities are all 0 and 1 and can tie, what is really meant is “no less likely” to be in the set). The following definition makes this formal. (Here and elsewhere, $\Sigma$ will denote our (finite) alphabet and $\Sigma^{\ast}$ will denote the set of finite strings over that alphabet.)

Definition 2**.**

A set $L$ is ${\rm P}$-selective if there exists a polynomial-time function, $f:\Sigma^{\ast}\times\Sigma^{\ast}\rightarrow\Sigma^{\ast}$ such that,

[TABLE]

It is known that some P-selective sets can be very hard. Some even have the property known as being “undecidable.” Despite that, our first challenge problem is to prove that ${\rm SAT}$ cannot be P-selective unless ${\rm SAT}$ is outright easy computationally, i.e., ${\rm SAT}\in{\rm P}$. Since it is close to an article of faith in computer science that ${\rm SAT}\not\in{\rm P}$, showing that some hypothesis implies that ${\rm SAT}\in{\rm P}$ is considered, with the full weight of modern computer science’s current understanding and intuition, to be extremely strong evidence that the hypothesis is unlikely to be true. (In the lingo, the hypothesis is implying that ${\rm P}={\rm NP}$. Although it is possible that ${\rm P}={\rm NP}$ is true, basically no one believes that it is [Gas19]. However, the issue is the most important open issue in applied mathematics, and there is currently a $1,000,000 prize for whoever resolves the issue [Cla19].)

A Bit of History

Inspired by an analogue in computability theory, P-selectivity was defined by Alan L. Selman in a seminal series of papers [Sel79, Sel81, Sel82b, Sel82a], which included a proof of our first challenge theorem. The fact, alluded to above, that P-selective sets can be very hard is due to Alan L. Selman’s above work and to the work of the researcher in whose memory this chapter is written, Ker-I Ko [Ko82]. In that same paper, Ko also did very important early work showing that P-selective sets are unlikely to have what is known as “small circuits.” For those particularly interested in the P-selective sets, they and their nondeterministic cousins are the subject of a book, Theory of Semi-Feasible Algorithms [HT03].

3.2 Can SAT Be P-Selective?

Challenge Problem 1**.**

(Prove that) if ${\rm SAT}$ is ${\rm P}$-selective, then ${\rm SAT}\in{\rm P}$.

Keep in mind that what you should be trying to do is this. You may assume that ${\rm SAT}$ is P-selective. So you may act as if you have in hand a polynomial-time computable function, $f$, that in the sense of Definition 2 shows that ${\rm SAT}$ is P-selective. And your task is to give a polynomial-time algorithm for ${\rm SAT}$, i.e., a program that in time polynomial in the number of bits in its input determines whether the input string belongs to ${\rm SAT}$. (Your algorithm surely will be making calls to $f$—possibly quite a few calls.)

So that you have them easily at hand while working on this, here are some of the key definitions and tools that you might want to draw on while trying to prove this theorem.

SAT

${\rm SAT}$ is the set of all satisfiable (propositional) Boolean formulas.

Self-reducibility

Let $k\geq 1$. Let $F(x_{1},x_{2},\dots,x_{k})$ be a Boolean formula (without loss of generality assume that each of the variables occurs in the formula). Then F(x_{1},x_{2},\dots,x_{k})\in{\rm SAT}\Longleftrightarrow\bigl{(}F(\mathrm{True},x_{2},\dots,x_{k})\in{\rm SAT}\lor F(\mathrm{False},x_{2},\dots,x_{k})\in{\rm SAT}\bigr{)}.

P-selectivity

A set $L$ is ${\rm P}$-selective if there exists a polynomial-time function, $f:\Sigma^{\ast}\times\Sigma^{\ast}\rightarrow\Sigma^{\ast}$ such that, (\forall a,b\in\Sigma^{\ast})[f(a,b)\in\{a,b\}\land\bigl{(}\{a,b\}\cap L\neq\emptyset\implies f(a,b)\in L\bigr{)}].

My suggestion to you would be to work on proving this theorem until either you find a proof, or you’ve put in at least 20 minutes of thought, are stuck, and don’t think that more time will be helpful.

When you’ve reached one or the other of those states, please go on to Appendix A to read a proof of this theorem. Having that proof will help you check whether your proof (if you found one) is correct, and if you did not find a proof, will show you a proof. Knowing the answer to this challenge problem before going on to the other three challenge problems is important, since an aspect of this problem’s solution will show up in the solutions to the other challenge problems.

I’ve put the solutions in separate appendix sections so that you can avoid accidentally seeing them before you wish to. But please do (unless you are completely certain that your solution to the first problem is correct) read the solution for the first problem before moving on to the second problem. If you did not find a proof for this first challenge problem, don’t feel bad; everyone has days when we see things and days when we don’t. On the other hand, if you did find a proof of this first challenge theorem, wonderful, and if you felt that it was easy, well, the four problems get steadily harder, until by the fourth problem almost anyone would have to work very, very hard and be a bit lucky to find a solution.

4 Challenge Problem 2: Low Information Content and SAT, Part 1:

Can SAT Reduce to a Tally Set?

Can ${\rm SAT}$ have low information content? To answer that, one needs to formalize what notion of low information content one wishes to study. There are many such notions, but a particularly important one is whether a given set can “many-one polynomial-time reduce” to a tally set (a set over a 1-letter alphabet).

A Bit of History

Our second challenge theorem was stated and proved by Piotr Berman [Ber78]. Berman’s paper started a remarkably long and productive line of work, which we will discuss in more detail in the “A Bit of History” note accompanying the third challenge problem. That same note will provide pointers to surveys of that line of work, for those interested in additional reading.

4.1 Needed Definitions

$\epsilon$ will denote the empty string.

Definition 3**.**

A set $T$ is a tally set if $T\subseteq\{\epsilon,0,00,000,\dots\}$.

Definition 4**.**

We say that $A\,\leq_{m}^{{{p}}}\,B$ ($A$ many-one polynomial-time reduces to $B$) if there is a polynomial-time computable function $g$ such that

[TABLE]

Informally, this says that $B$ is so powerful that each membership query to $A$ can be efficiently transformed into a membership query to $B$ that gets the same answer as would the question regarding membership in $A$.

4.2 Can SAT Reduce to a Tally Set?

Challenge Problem 2**.**

(Prove that) if there exists a tally set $T$ such that ${\rm SAT}\,\leq_{m}^{{{p}}}\,T$, then ${\rm SAT}\in{\rm P}$.

Keep in mind that what you should be trying to do is this. You may assume that there exists a tally set $T$ such that ${\rm SAT}\,\leq_{m}^{{{p}}}\,T$. You may not assume that you have a polynomial-time algorithm for $T$; you are assuming that $T$ exists, but for all we know, $T$ might well be very hard. On the other hand, you may assume that you have in hand a polynomial-time computable function $g$ that reduces from ${\rm SAT}$ to $T$ in the sense of Definition 4. (After all, that reduction is (if it exists) a finite-sized program.) Your task here is to give a polynomial-time algorithm for ${\rm SAT}$, i.e., a program that in time polynomial in the number of bits in its input determines whether the input string belongs to ${\rm SAT}$. (Your algorithm surely will be making calls to $g$—possibly quite a few calls.)

So that you have them easily at hand while working on this, here are some of the key definitions and tools that you might want to draw on while trying to prove this theorem.

SAT

${\rm SAT}$ is the set of all satisfiable (propositional) Boolean formulas.

Self-reducibility

Let $k\geq 1$. Let $F(x_{1},x_{2},\dots,x_{k})$ be a Boolean formula (without loss of generality assume that each of the $x_{i}$ actually occurs in the formula). Then F(x_{1},x_{2},\dots,x_{k})\in{\rm SAT}\Longleftrightarrow\bigl{(}F(\mathrm{True},x_{2},\dots,x_{k})\in{\rm SAT}\lor F(\mathrm{False},x_{2},\dots,x_{k})\in{\rm SAT}\bigr{)}.

Tally sets

A set $T$ is a tally set if $T\subseteq\{\epsilon,0,00,000,\dots\}$.

Many-one reductions

We say that $A\,\leq_{m}^{{{p}}}\,B$ if there is a polynomial-time computable function $g$ such that $(\forall x\in\Sigma^{\ast})[x\in A\iff g(x)\in B]$.

My suggestion to you would be to work on proving this theorem until either you find a proof, or you’ve put in at least 30 minutes of thought, are stuck, and don’t think that more time will be helpful.

When you’ve reached one or the other of those states, please go on to Appendix B to read a proof of this theorem. The solution to the third challenge problem is an extension of this problem’s solution, so knowing the answer to this challenge problem before going on to the third challenge problem is important.

5 Challenge Problem 3: Low Information Content and SAT, Part 2: Can $\overline{{\bf SAT}}$ Reduce to a Sparse Set?

This problem challenges you to show that even a class of sets that is far broader than the tally sets, namely, the so-called sparse sets, cannot be reduced to from $\overline{{\rm SAT}}$ unless ${\rm SAT}\in{\rm P}$.

A Bit of History

This third challenge problem was stated and proved by Steve Fortune [For79]. It was another step in what was a long line of advances—employing more and more creative and sometimes difficult proofs—that eventually led to the understanding that, unless ${\rm SAT}\in{\rm P}$, no sparse set can be hard for ${\rm SAT}$ even with respect to extremely flexible types of reductions. The most famous result within this line is known as Mahaney’s Theorem: If there is a sparse set $S$ such that ${\rm SAT}\,\leq_{m}^{{{p}}}\,S$, then ${\rm SAT}\in{\rm P}$ [Mah82]. There are many surveys of the just-mentioned line of work, e.g., [Mah86, Mah89, You92, HOW92]. The currently strongest result in that line is due to Glaßer [Gla00] (see the survey/treatment of that in [GH00], and see also the results of Arvind et al. [AHH*+*93]).

5.1 Needed Definitions

Let $\|S\|$ denote the cardinality of set $S$, e.g., $\|\{\epsilon,0,0,0,00\}\|=3$.

For any set $L$, let $\overline{L}$ denote the complement of $L$.

Let $|x|$ denote the length string $x$, e.g., $|\text{moon}|=4$.

Definition 5**.**

A set $S$ is sparse if there exists a polynomial $q$ such that, for each natural number $n\in\{0,1,2,\dots\},$ it holds that

[TABLE]

Informally put, the sparse sets are the sets whose number of strings up to a given length is at most polynomial. $\{0,1\}^{*}$ is, for example, not a sparse set, since up to length $n$ it has $2^{n+1}-1$ strings. But all tally sets are sparse, indeed all via the bounding polynomial $q(n)=n+1$.

5.2 Can $\overline{{\bf SAT}}$ Reduce to a Sparse Set?

Challenge Problem 3**.**

(Prove that) if there exists a sparse set $S$ such that $\overline{{\rm SAT}}\,\leq_{m}^{{{p}}}\,S$, then ${\rm SAT}\in{\rm P}$.

Keep in mind that what you should be trying to do is this. You may assume that there exists a sparse set $S$ such that $\overline{{\rm SAT}}\,\leq_{m}^{{{p}}}\,S$. You may not assume that you have a polynomial-time algorithm for $S$; you are assuming that $S$ exists, but for all we know, $S$ might well be very hard. On the other hand, you may assume that you have in hand a polynomial-time computable function $g$ that reduces from $\overline{{\rm SAT}}$ to $S$ in the sense of Definition 4. (After all, that reduction is—if it exists—a finite-sized program.) And you may assume that you have in hand a polynomial that upper-bounds the sparseness of $S$ in the sense of Definition 5. (After all, one of the countably infinite list of simple polynomials $n^{k}+k$—for $k=1,2,3,\dots$—will provide such an upper bound, if any such polynomial upper bound exists.) Your task here is to give a polynomial-time algorithm for ${\rm SAT}$, i.e., a program that in time polynomial in the number of bits in its input determines whether the input string belongs to ${\rm SAT}$. (Your algorithm surely will be making calls to $g$—possibly quite a few calls.)

One might wonder why I just said that you should build a polynomial-time algorithm for ${\rm SAT}$, given that the theorem speaks of $\overline{{\rm SAT}}$. However, since it is clear that ${\rm SAT}\in{\rm P}\iff\overline{{\rm SAT}}\in{\rm P}$ (namely, given a polynomial-time algorithm for ${\rm SAT}$, if we simply reverse the answer on each input, then we now have a polynomial-time algorithm for $\overline{{\rm SAT}}$), it is legal to focus on ${\rm SAT}$—and most people find doing so more natural and intuitive.

Do be careful here. Solving this challenge problem may take an “aha!… insight” moment. Knowing the solution to Challenge Problem 2 will be a help here, but even with that knowledge in hand one hits an obstacle. And then the challenge is to find a way around that obstacle.

So that you have them easily at hand while working on this, here are some of the key definitions and tools that you might want to draw on while trying to prove this theorem.

SAT and $\rm\overline{SAT}$

${\rm SAT}$ is the set of all satisfiable (propositional) Boolean formulas. $\overline{{\rm SAT}}$ denotes the complement of ${\rm SAT}$.

Self-reducibility

Let $k\geq 1$. Let $F(x_{1},x_{2},\dots,x_{k})$ be a Boolean formula (without loss of generality assume that each of the $x_{i}$ actually occurs in the formula). Then F(x_{1},x_{2},\dots,x_{k})\in{\rm SAT}\Longleftrightarrow\bigl{(}F(\mathrm{True},x_{2},\dots,x_{k})\in{\rm SAT}\lor F(\mathrm{False},x_{2},\dots,x_{k})\in{\rm SAT}\bigr{)}.

Sparse sets

A set $S$ is sparse if there exists a polynomial $q$ such that, for each natural number $n\in\{0,1,2,\dots\}$, it holds that $\|\{x\mid x\in S\land|x|\leq n\}\|\leq q(n)$.

Many-one reductions

We say that $A\,\leq_{m}^{{{p}}}\,B$ if there is a polynomial-time computable function $g$ such that $(\forall x\in\Sigma^{\ast})[x\in A\iff g(x)\in B]$.

My suggestion to you would be to work on proving this theorem until either you find a proof, or you’ve put in at least 40 minutes of thought, are stuck, and don’t think that more time will be helpful.

When you’ve reached one or the other of those states, please go on to Appendix C to read a proof of this theorem.

6 Challenge Problem 4: Is #SAT as Hard to (Enumeratively) Approximate as

It Is to Solve Exactly?

This final challenge is harder than the three previous ones. To solve it, you’ll have to have multiple insights—as to what approach to use, what building blocks to use, and how to use them.

The problem is sufficiently hard that the solution is structured to give you, if you did not solve the problem already, a second bite at the apple! That is, the solution—after discussing why the problem can be hard to solve—gives a very big hint, and then invites you to re-try to problem with that hint in hand.

A Bit of History

The function ${\rm\#SAT}$, the counting version of ${\rm SAT}$, will play a central role in this challenge problem. ${\rm\#SAT}$ was introduced and studied by Valiant [Val79a, Val79b]. This final challenge problem, its proof (including the lemma given in the solution I give here and the proof of that lemma), and the notion of enumerators and enumerative approximation are due to Cai and Hemachandra [CH89]. The challenge problem is a weaker version of the main result of that paper, which proves the result for not just 2-enumerators but even for sublinear-enumerators; later work showed that the result even holds for all polynomial-time computable enumerators [CH91].

6.1 Needed Definitions

$|F|$ will denote the length of (the encoding of) formula $F$.

${\rm\#SAT}$ is the function that given as input a Boolean formula $F(x_{1},x_{2},\dots,x_{k})$—without loss of generality assume that each of the variables occurs in $F$—outputs the number of satisfying assignments the formula has (i.e., of the $2^{k}$ possible assignments of the variables to $\mathrm{True}$/$\mathrm{False}$, the number of those under which $F$ evaluates to $\mathrm{True}$; so the output will be a natural number in the interval $[0,2^{k}]$). For example, ${\rm\#SAT}(x_{1}\lor x_{2})=3$ and ${\rm\#SAT}(x_{1}\land\overline{x_{1}})=0$.

Definition 6**.**

We say that ${\rm\#SAT}$ has a polynomial-time $2$-enumerator (aka, is polynomial-time 2-enumerably approximable) if there is a polynomial-time computable function $h$ such that on each input $x$,

$h(x)$* outputs a list of two (perhaps identical) natural numbers, and* 2. 2.

${\rm\#SAT}(x)$* appears in the list output by $h(x)$.*

So a 2-enumerator $h$ outputs a list of (at most) two candidate values for the value of ${\rm\#SAT}$ on the given input, and the actual output is always somewhere in that list. This notion generalizes in the natural way to other list cardinalities, e.g., $1492$-enumerators and, for each $k\in\{1,2,3,\dots\}$, $\max(1,|F|^{k})$-enumerators.

6.2 Food for Thought

You’ll certainly want to use some analogue of the key self-reducibility observation, except now respun by you to be about the number of solutions of a formula and how it relates to or is determined by the number of solutions of its two “child” formulas.

But doing that is just the first step your quest. So… please play around with ideas and approaches. Don’t be afraid to be bold and ambitious. For example, you might say “Hmmmm, if we could do/build XYZ (where perhaps XYZ might be some particular insight about combining formulas), that would be a powerful tool in solving this, and I suspect we can do/build XYZ.” And then you might want to work both on building XYZ and on showing in detail how, if you did have tool XYZ in hand, you could use it to show the theorem.

6.3 Is #SAT as Hard to (Enumeratively) Approximate as It Is to Solve Exactly?

Challenge Problem 4** (Cai and Hemachandra).**

(Prove that) if ${\rm\#SAT}$ has a polynomial-time $2$-enumerator, then there is a polynomial-time algorithm for ${\rm\#SAT}$.

Keep in mind that what you should be trying to do is this. You may assume that you have in hand a polynomial-time 2-enumerator for ${\rm\#SAT}$. Your task here is to give a polynomial-time algorithm for ${\rm\#SAT}$, i.e., a program that in time polynomial in the number of bits in its input determines the number of satisfying assignments of the (formula encoded by the) input string. (Your algorithm surely will be making calls to the 2-enumerator—possibly quite a few calls.)

Do be careful here. Proving this may take about three “aha!… insight” moments; Section 6.2 gave slight hints regarding two of those.

So that you have them easily at hand while working on this, here are some of the key definitions and tools that you might want to draw on while trying to prove this theorem.

#SAT

${\rm\#SAT}$ is the function that given as input a Boolean formula $F(x_{1},x_{2},\dots,x_{k})$—without loss of generality assume that each of the variables occurs in $F$—outputs the number of satisfying assignments the formula has (i.e., of the $2^{k}$ possible assignments of the variables to $\mathrm{True}$/$\mathrm{False}$, the number of those under which $F$ evaluates to $\mathrm{True}$; so the output will be a natural number in the interval $[0,2^{k}]$). For example, ${\rm\#SAT}(x_{1}\lor x_{2})=3$ and ${\rm\#SAT}(x_{1}\land\overline{x_{1}})=0$.

Enumerative approximation

We say that ${\rm\#SAT}$ has a polynomial-time $2$-enumerator (aka, is polynomial-time 2-enumerably approximable) if there is a polynomial-time computable function $h$ such that on each input $x$, (a) $h(x)$ outputs a list of two (perhaps identical) natural numbers, and (b) ${\rm\#SAT}(x)$ appears in the list output by $h(x)$.

My suggestion to you would be to work on proving this theorem until either you find a proof, or you’ve put in at least 30–60 minutes of thought, are stuck, and don’t think that more time will be helpful.

When you’ve reached one or the other of those states, please go on to Appendix D, where you will find first a discussion of what the most tempting dead end here is, why it is a dead end, and a tool that will help you avoid the dead end. And then you’ll be urged to attack the problem again with that extra tool in hand.

7 Going Big: Complexity-Class Implications

During all four of our challenge problems, we focused just on the concrete problem, ${\rm SAT}$, in its language version or in its counting analogue, ${\rm\#SAT}$.

However, the challenge results in fact apply to broader classes of problems. Although we (mostly) won’t prove those broader results in this chapter, this section will briefly note some of those (and the reader may well be able to in most cases easily fill in the proofs). The original papers, cited in the “A Bit of History” notes, are an excellent source to go to for more coverage. None of the claims below, of course, are due to the present tutorial paper, but rather they are generally right from the original papers. Also often of use for a gentler treatment than the original papers is the textbook, The Complexity Theory Companion [HO02], in which coverage related to our four problems can be found in, respectively, chapters 1, 1 [sic], 3, and (using a different technique and focusing on a concrete but different target problem) 6.

Let us define the complexity class NP by ${\rm NP}=\{L\mid L\,\leq_{m}^{{{p}}}\,{\rm SAT}\}$. NP more commonly is defined as the class of sets accepted by nondeterministic polynomial-time Turing machines; but that definition in fact yields the same class of sets as the alternate definition just given, and would require a detailed discussion of what Turing machines are.

Recall that $\overline{L}$ denotes the complement of $L$. Let us define the complexity class ${\rm coNP}$ by ${\rm coNP}=\{L\mid\overline{L}\in{\rm NP}\}$.

A set $H$ is said to be hard for a class $\cal C$ if for each set $L\in\cal C$ it holds that $L\,\leq_{m}^{{{p}}}\,H$. If in addition $H\in{\cal C}$, then we say that $H$ is ${\cal C}$-complete. It is well known—although it takes quite a bit of work to show and showing this was one of the most important steps in the history of computer science—that ${\rm SAT}$ is NP-complete [Coo71, Lev75, Kar72].

The following theorem follows easily from our first challenge theorem, basically because if some NP-hard set is P-selective, that causes ${\rm SAT}$ to be P-selective. (Why? Our P-selector function for ${\rm SAT}$ will simply polynomial-time reduce each of its two inputs to the NP-hard set, will run that set’s P-selector function on those two strings, and then will select as the more likely to belong to ${\rm SAT}$ whichever input string corresponded to the selected string, and for definiteness will choose its first argument in the degenerate case where both its arguments map to the same string.)

Theorem 1**.**

If there exists an ${\rm NP}$-hard, ${\rm P}$-selective set, then ${\rm P}={\rm NP}$.

The converse of the above theorem also holds, since if ${\rm P}={\rm NP}$ then ${\rm SAT}$ and indeed all of NP is P-selective, since P sets unconditionally are P-selective.

The following theorem follows easily from our second challenge theorem.

Theorem 2**.**

If there exists an ${\rm NP}$-hard tally set, then ${\rm P}={\rm NP}$.

The converse of the above theorem also holds.

The following theorem follows easily from our third challenge theorem.

Theorem 3**.**

If there exists a ${\rm coNP}$-hard sparse set, then ${\rm P}={\rm NP}$.

The converse of the above theorem also holds.

To state the complexity-class analogue of the fourth challenge problem takes a bit more background, since the result is about function classes rather than language classes.

There is a complexity class, which we will not define here, defined by Valiant and known as ${\rm\#P}$ [Val79a], that is the set of functions that count the numbers of accepting paths of what are known as nondeterministic polynomial-time Turing machines.

Metric reductions give a reduction notion that applies to the case of functions rather than languages, and are defined as follows. A function $f:\Sigma^{*}\rightarrow\{0,1,2,\dots\}$ is said to polynomial-time metric reduce to a function $g:\Sigma^{*}\rightarrow\{0,1,2,\dots\}$ if there exist two polynomial-time computable functions, $\varphi$ and $\psi$, such that $(\forall x\in\Sigma^{*})[f(x)=\psi(x,g(\varphi(x)))]$ [Kre88]. (We are assuming that our output natural numbers are naturally coded in binary.) We say a function $f$ is hard for ${\rm\#P}$ with respect to polynomial-time metric reductions if for every $f^{\prime}\in{\rm\#P}$ it holds that $f^{\prime}$ polynomial-time metric reduces to $f$; if in addition $f\in{\rm\#P}$, we say that $f$ is ${\rm\#P}$-complete with respect to polynomial-time metric reductions.

With that groundwork in hand, we can now state the analogue, for counting classes, of our fourth challenge theorem. Since we have not defined ${\rm\#P}$ here, we’ll state the theorem both in terms of ${\rm\#SAT}$ and in terms of ${\rm\#P}$ (the two statements below in fact turn out to be equivalent).

Theorem 4**.**

If there exists a function $f$ such that ${\rm\#SAT}$ polynomial-time metric reduces to $f$ and $f$ has a $2$-enumerator, then there is a polynomial-time algorithm for ${\rm\#SAT}$. 2. 2.

If there exists a function that is ${\rm\#P}$-hard with respect to polynomial-time metric reductions and has a $2$-enumerator, then there is a polynomial-time algorithm for ${\rm\#SAT}$.

The converse of each of the theorem parts also holds. The above theorem parts (and their converses) even hold if one asks not about 2-enumerators but rather about polynomial-time enumerators that have no limit on the number of elements in their output lists (aside from the polynomial limit that is implicit from the fact that the enumerators have only polynomial time to write their lists).

8 Conclusions

In conclusion, self-reducibility provides a powerful tool with applications across a broad range of settings.

Myself, I have found self-reducibility and its generalizations to be useful in understanding topics ranging from election manipulation [HHM13] to backbones of and backdoors to Boolean formulas [HN17, HN19] to the complexity of sparse sets [HS95], space-efficient language recognition [HOT94], logspace computation [HJ97], and approximation [HZ96, HH03].

My guess and hope is that perhaps you too may find self-reducibility useful in your future work. That is, please, if it is not already there, consider adding this tool to your personal research toolkit: When you face a problem, think (if only for a moment) whether the problem happens to be one where the concept of self-reducibility will help you gain insight. Who knows? One of these years, you might be happily surprised in finding that your answer to such a question is “Yes!”

Acknowledgments

I am grateful to the students and faculty at the computer science departments of RWTH Aachen University, Heinrich Heine University Düsseldorf, and the University of Rochester. I “test drove” this chapter at each of those schools in the form of a lecture or lecture series. Particular thanks go to Peter Rossmanith, Jörg Rothe, and Muthu Venkitasubramaniam, who invited me to speak, and to Gerhard Woeginger regarding the counterexample in Appendix D. My warm thanks to Ding-Zhu Du, Bin Liu, and Jie Wang for inviting me to contribute to this project that they have organized in memory of the wonderful Ker-I Ko, whose work contributed so richly to the beautiful, ever-growing tapestry that is complexity theory.

Appendices

Appendix A Solution to Challenge Problem

1

Before we start on the proof, let us put up a figure that shows the flavor of a structure that we will use to help us understand and exploit ${\rm SAT}$’s self-reducibility. The structure is known as the self-reducibility tree of a formula. At the root of this tree sits the formula. At the next level as the root’s children, we have the formula with its first variable assigned to $\mathrm{True}$ and to $\mathrm{False}$. At the level below that, we have the two formulas from the second level, except with each of their first variables (i.e., the second variable of the original formula) assigned to both $\mathrm{True}$ and $\mathrm{False}$. Figure 1 shows the self-reducibility tree of a two-variable formula.

Self-reducibility tells us that, for each node $N$ in such a self-reducibility tree (except the leaves, since they have no children), $N$ is satisfiable if and only if at least one of its two children is satisfiable. Inductively, the formula at the root of the tree is satisfiable if and only if each level of the self-reducibility tree has at least one satisfiable node. And, also, the formula at the root of the tree is satisfiable if and only if every level of the self-reducibility tree has at least one satisfiable node.

How helpful is this tree? Well, we certainly don’t want to solve ${\rm SAT}$ by checking every leaf of the self-reducibility tree. On formulas with $k$ variables, that would take time at least $2^{k}$—basically a brute-force exponential-time algorithm. Yuck! That isn’t surprising though. After all, the tree is really just listing all assignments to the formula.

But the magic here, which we will exploit, is that the “self-reducibility” relationship between nodes and their children as to satisfiability will, at least with certain extra assumptions such as about P-selectivity, allow us to not explore the whole tree. Rather, we’ll be able to prune away, quickly, all but a polynomially large subtree. In fact, though on its surface this chapter is about four questions from complexity theory, it really is about tree-pruning—a topic more commonly associated with algorithms than with complexity. To us, though, that is not a problem but an advantage. As we mentioned earlier, complexity is largely about building algorithms, and that helps make complexity far more inviting and intuitive than most people realize.

That being said, let us move on to giving a proof of the first challenge problem. Namely, in this section we sketch a proof of the result:

If ${\rm SAT}$ is ${\rm P}$-selective, then ${\rm SAT}\in{\rm P}$.

So assume that ${\rm SAT}$ is P-selective, via (in the sense of Definition 2) polynomial-time computable function $f$. Let us give a polynomial-time algorithm for ${\rm SAT}$. Suppose the input to our algorithm is the formula $F(x_{1},x_{2},\dots,x_{k})$. (If the input is not a syntactically legal formula we immediately reject, and if the input is a formula that has zero variables, e.g., $\mathrm{True}\land\mathrm{True}\land\mathrm{False}$, we simply evaluate it and accept if and only if it evaluates to $\mathrm{True}$.) Let us focus on $F$ and $F$’s two children in the self-reducibility tree, as shown in Figure 2.

Now, run $f$ on $F$’s two children. That is, compute, in polynomial time, $f(F(\mathrm{True},x_{2},\dots,x_{k}),\,F(\mathrm{False},x_{2},\dots,x_{k}))$. Due to the properties of P-selectivity and self-reducibility, note that the output of that application of $f$ is a formula/node that has the property that the original formula is satisfiable if and only if that child-node is satisfiable.

In particular, if $f(F(\mathrm{True},x_{2},\dots,x_{k}),\,F(\mathrm{False},x_{2},\dots,x_{k}))=F(\mathrm{True},x_{2},\dots,x_{k})$ then we know that $F(x_{1},x_{2},\dots,x_{k})$ is satisfiable if and only if $F(\mathrm{True},x_{2},\dots,x_{k})$ is satisfiable. And if $f(F(\mathrm{True},x_{2},\dots,x_{k}),\,F(\mathrm{False},x_{2},\dots,x_{k}))\neq F(\mathrm{True},x_{2},\dots,x_{k})$ then we know that $F(x_{1},x_{2},\dots,x_{k})$ is satisfiable if and only if $F(\mathrm{False},x_{2},\dots,x_{k})$ is satisfiable.

Either way, we have in time polynomial in the input’s size eliminated the need to pay attention to one of the two child nodes, and now may focus just on the other one.

Repeat the above process on the child that, as per the above, was selected by the selector function. Now, “split” that formula by assigning $x_{2}$ both possible ways. That will create two children, and then analogously to what was done above, use the selector function to decide which of those two children is the more promising branch to follow.

Repeat this until we have assigned all variables. We now have a fully assigned formula, but due to how we got to it, we know that it evaluates to $\mathrm{True}$ if and only if the original formula is satisfiable. So if that fully assigned formula evaluates to $\mathrm{True}$,then we state that the original formula is satisfiable (and indeed, our path down the self-reducibility tree has outright put into our hands a satisfying assignment). And, more interestingly, if the fully assigned formula evaluates to $\mathrm{False}$, then we state that the original formula is not satisfiable. We are correct in stating that, because at each iterative stage we know that if the formula we start that stage focused on is satisfiable, then the child the selector function chooses for us will also be satisfiable.

The process above is an at most polynomial number of at most polynomial-time “descend one level having made a linkage” stages, and so overall itself runs in polynomial time. Thus we have given a polynomial-time algorithm for ${\rm SAT}$, under the hypothesis that ${\rm SAT}$ is P-selective. This completes the proof sketch.

Our algorithm was mostly focused on tree pruning. Though $F$ induces a giant binary tree as to doing variable assignments one variable at a time in all possible ways, thanks to the guidance of the selector function, we walked just a single path through that tree.

Keeping this flavor of approach in mind might be helpful on Challenge Problem 2, although that is a different problem and so perhaps you’ll have to bring some new twist, or greater flexibility, to what you do to tackle that.

And now, please pop right on back to the main body of the chapter, to read and tackle Challenge Problem 2!

Appendix B Solution to Challenge Problem 2

In this section we sketch a proof of the result:

If there exists a tally set $T$ such that ${\rm SAT}\,\leq_{m}^{{{p}}}\,T$, then ${\rm SAT}\in{\rm P}$.

So assume that there exists a tally set $T$ such that ${\rm SAT}\,\leq_{m}^{{{p}}}\,T$. Let $g$ be the polynomial-time computable function performing that reduction, in the sense of Definition 4. (Keep in mind that we may not assume that $T\in{\rm P}$. We have no argument line in hand that would tell us that that happens to be true.) Let us give a polynomial-time algorithm for ${\rm SAT}$.

Suppose the input to our algorithm is the formula $F(x_{1},x_{2},\dots,x_{k})$. (If the input is not a syntactically legal formula we immediately reject, and if the input is a formula that has zero variables we simply evaluate it and accept if and only if it evaluates to $\mathrm{True}$.)

Let us focus first on $F$. Compute, in polynomial time, $g(F(x_{1},x_{2},\dots,x_{k}))$. If $g(F(x_{1},x_{2},\dots,x_{k}))\not\in\{\epsilon,0,00,\dots\}$, then clearly $F(x_{1},x_{2},\dots,x_{k})\not\in{\rm SAT}$, since we know that (a) $T\subseteq\{\epsilon,0,00,\dots\}$ and (b) $F(x_{1},x_{2},\dots,x_{k})\in{\rm SAT}\iff g(F(x_{1},x_{2},\dots,x_{k}))\in T$. So in that case, we output that $F(x_{1},x_{2},\dots,x_{k})\not\in{\rm SAT}$. Otherwise, we descend to the next level of the “self-reducibility tree” as follows.

We consider the nodes (i.e., in this case, formulas) $F(\mathrm{True},x_{2},\dots,x_{k})$ and $F(\mathrm{False},x_{2},\dots,x_{k})$. Compute $g(F(\mathrm{True},x_{2},\dots,x_{k}))$ and $g(F(\mathrm{False},x_{2},\dots,x_{k}))$. If either of our two nodes in question does not, under the action just computed of $g$, map to a string in $\{\epsilon,0,00,\dots\}$, then that node certainly is not a satisfiable formula, and we can henceforward mentally ignore it and the entire tree (created by assigning more of its variables) rooted at it. This is one key type of pruning that we will use: eliminating from consideration nodes that map to “nontally” strings.

But there is a second type of pruning that we will use: If it happens to be the case that $g(F(\mathrm{True},x_{2},\dots,x_{k}))\in\{\epsilon,0,00,\dots\}$ and $g(F(\mathrm{True},x_{2},\dots,x_{k}))=g(F(\mathrm{False},x_{2},\dots,x_{k}))$, then at this point it may not be clear to us whether $F(\mathrm{True},x_{2},\dots,x_{k})$ is or is not satisfiable. However, what is clear is that

[TABLE]

How do we know this? Since $g$ reduces ${\rm SAT}$ to $T$, we know that

[TABLE]

and

[TABLE]

By those observations, the fact that $g(F(\mathrm{True},x_{2},\dots,x_{k}))=g(F(\mathrm{False},x_{2},\dots,x_{k}))$, and the transitivity of “${\iff}$”, we indeed have that $F(\mathrm{True},x_{2},\dots,x_{k})\in{\rm SAT}\iff F(\mathrm{False},x_{2},\dots,x_{k})\in{\rm SAT}$. But since that says that either both or neither of these nodes is a formula belonging to ${\rm SAT}$, there is no need at all for us to further explore more than one of them, since they stand or fall together as to membership in ${\rm SAT}$. So if we have $g(F(\mathrm{True},x_{2},\dots,x_{k}))=g(F(\mathrm{False},x_{2},\dots,x_{k}))$, we can mentally dismiss $F(\mathrm{False},x_{2},\dots,x_{k})$—and of course also the entire subtree rooted at it—from all further consideration.

After doing the two types of pruning just mentioned, we will have either one or two nodes left at the level of the tree—the level one down from the root—that we are considering. (If we have zero nodes left, we have pruned away all possible paths and can safely reject). Also, if $k=1$, then we can simply check whether at least one node that has not been pruned away evaluates to $\mathrm{True}$, and if so we accept and if not we reject.

But what we have outlined can iteratively be carried out in a way that drives us right down through the tree, one level at a time. At each level, we take all nodes (i.e., formulas; we will speak interchangeably of the node and the formula that it is representing) that have not yet been eliminated from consideration, and for each, take the next unassigned variable and make two child formulas, one with that variable assigned to $\mathrm{True}$ and one with that variable assigned to $\mathrm{False}$. So if at a given level after pruning we end up with $j$ formulas, we in this process start the next level with $2j$ formulas, each with one fewer variable. Then for those $2j$ formulas we do the following: For each of them, if $g$ applied to that formula outputs a string that is not a member of $\{\epsilon,0,00,\dots\}$, then eliminate that node from all further consideration. After all, the node clearly is not a satisfiable formula. Also, for all nodes among the $2j$ such that the string $z$ that $g$ maps them to belongs to $\{\epsilon,0,00,\dots\}$ and $z$ is mapped to by $g$ by at least one other of the $2j$ nodes, for each such cluster of nodes that map to the same string $z$ (of the form $\{\epsilon,0,00,\dots\}$) eliminate all but one of the nodes from consideration. After all, by the argument given above, either all of that cluster are satisfiable or none of them are, so we can eliminate all but one from consideration, since eliminating all the others still leaves one that is satisfiable, if in fact the nodes in the cluster are satisfiable.

Continue this process until (it internally terminates with a decision, or) we reach a level where all variables are assigned. If there were $j$ nodes at the level above that after pruning, then at this no-variables-left-to-assign level we have at most $2j$ formulas. The construction is such that $F(x_{1},x_{2},\dots,x_{k})\in{\rm SAT}$ if and only if at least one of these at most $2j$ variable-free formulas belongs to ${\rm SAT}$, i.e., evaluates to $\mathrm{True}$. But we can easily check that in time polynomial in $2j\times|F(x_{1},x_{2},\dots,x_{k})|$.

Is the proof done? Not yet. If $j$ can be huge, we’re dead, as we might have just sketched an exponential-time algorithm. But fortunately, and this was the key insight in Piotr Berman’s paper that proved this result, as we go down the tree, level by level, the tree never grows too wide. In particular, it is at most polynomially wide!

How can we know this? The insight that Berman (and with luck, also you!) had is that there are not many “tally” strings that can be reached by the reduction $g$ on the inputs that it will be run on in our construction on a given input. And that fact will ensure us that after we do our two kinds of pruning, we have at most polynomially many strings left at the now-pruned level.

Let us be more concrete about this, since it is not just the heart of this problem’s solution, but also might well (hint!, hint!) be useful when tacking the third challenge problem.

In particular, we know that $g$ is polynomial-time computable. So there certainly is some natural number $k$ such that, for each natural number $n$, the function $g$ runs in time at most $n^{k}+k$ on all inputs of length $n$. Let $m=|F(x_{1},x_{2},\dots,x_{k})|$. Note that, at least if the encoding scheme is reasonable and we if needed do reasonable, obvious simplifications (e.g., $\mathrm{True}\land y\equiv y$, $\mathrm{True}\lor y\equiv\mathrm{True}$, $\neg\mathrm{True}\equiv\mathrm{False}$, and $\neg\mathrm{False}\equiv\mathrm{True}$), then each formula in the tree is of length less than or equal to $m$. Crucially, $g$ applied to strings of length less than or equal to $m$ can never output any string of length greater than $m^{k}+k$. And so there are at most $m^{k}+k+1$ strings (the “+ 1” is because the empty string is one of the strings that can be reached) in $\{\epsilon,0,00,\dots\}$ that can be mapped to by any of the nodes that are part of our proof’s self-reducibility tree when the input is $F(x_{1},x_{2},\dots,x_{k})$. So at each level of our tree-pruning, we eliminate all nodes that map to strings that do not belong to $\{\epsilon,0,00,\dots\}$, and since we leave at most one node mapping to each string that is mapped to in $\{\epsilon,0,00,\dots\}$, and as we just argued that there are at most $m^{k}+k+1$ of those, at the end of pruning a given level, at most $m^{k}+k+1$ nodes are still under consideration. But $m$ is the length of our problem’s input, so each level, after pruning, finishes with at most at most $m^{k}+k+1$ nodes, and so the level after it, after we split each of the current level’s nodes, will begin with at most $2(m^{k}+k+1)$ nodes. And after pruning that level, it too ends up with at most $m^{k}+k+1$ nodes still in play. The tree indeed remains at most polynomially wide.

Thus when we reach the “no variables left unassigned” level, we come into it with a polynomial-sized set of possible satisfying assignments (namely, a set of at most $m^{k}+k+1$ assignments), and we know that the original formula is satisfiable if and only if at least one of these assignments satisfies $F$.

Thus the entire algorithm is a polynomial number of rounds (one per variable eliminated), each taking polynomial time. So overall it is a polynomial-time algorithm that it is correctly deciding ${\rm SAT}$. This completes the proof sketch.

And now, please pop right on back to the main body of the chapter, to read and tackle Challenge Problem 3! While doing so, please keep this proof in mind, since doing so will be useful on Challenge Problem 3… though you also will need to discover a quite cool additional insight—the same one Steve Fortune discovered when he originally proved the theorem that is our Challenge Problem 3.

Appendix C Solution to Challenge Problem 3

In this section we sketch a proof of the result:

If there exists a sparse set $S$ such that $\overline{{\rm SAT}}\,\leq_{m}^{{{p}}}\,S$, then ${\rm SAT}\in{\rm P}$.

So assume that there exists a sparse set $S$ such that $\overline{{\rm SAT}}\,\leq_{m}^{{{p}}}\,S$. Let $g$ be the polynomial-time computable function performing that reduction, in the sense of Definition 4. (Keep in mind that we may not assume that $S\in{\rm P}$. We have no argument line in hand that would tell us that that happens to be true.) Let us give a polynomial-time algorithm for ${\rm SAT}$.

Suppose the input to our algorithm is the formula $F(x_{1},x_{2},\dots,x_{k})$. (If the input is not a syntactically legal formula we immediately reject, and if the input is a formula that has zero variables we simply evaluate it and accept if and only if it evaluates to $\mathrm{True}$.)

What we are going to do here is that we are going to mimic the proof that solved Challenge Problem 2. We are going to go level by level down the self-reducibility tree, pruning at each level, and arguing that the tree never gets too wide—at least if we are careful and employ a rather jolting insight that Steve Fortune (and with luck, also you!) had.

Note that of the two types of pruning we used in the Challenge Problem 2 proof, one applies perfectly well here. If two or more nodes on a given level of the tree map under $g$ to the same string, we can eliminate from consideration all but one of them, since either all of them or none of them are satisfiable.

However, the other type of pruning—eliminating all nodes not mapping to a string in $\{\epsilon,0,00,\dots\}$—completely disappears here. Sparse sets don’t have too many strings per level, but the strings are not trapped to always being of a specific, well-known form.

Is the one type of pruning that is left to us enough to keep the tree from growing exponentially bushy as we go down it? At first glance, it seems that exponential width growth is very much possible, e.g., imagine the case that every node of the tree maps to a different string than all the others at the node’s same level. Then with each level our tree would be doubling in size, and by its base, if we started with $k$ variables, we’d have $2^{k}$ nodes at the base level—clearly an exponentially bushy tree.

But Fortune stepped back and realized something lovely. He realized that if the tree ever became too bushy, then that itself would be an implicit proof that $F$ is satisfiable! Wow; mind-blowing!

In particular, Fortune used the following beautiful reasoning.

We know $g$ runs in polynomial time. So let the polynomial $r(n)$ bound $g$’s running time on inputs of length $n$, and without loss of generality, assume that $r$ is nondecreasing. We know that $S$ is sparse, so let the polynomial $q(n)$ bound the number of strings in $S$ up to and including length $n$, and without loss of generality, assume that $q$ is nondecreasing.

Let $m=|F(x_{1},x_{2},\dots,x_{k})|$, and as before, note that all the nodes in our proof are of length less than or equal to $m$.

How many distinct strings in $S$ can be reached by applying $g$ to strings of length at most $m$? On inputs of length at most $m$, clearly $g$ maps to strings of length at most $r(m)$. But note that the number of strings in $S$ of length at most $r(m)$ is at most $q(r(m))$.

Now, there are two cases. Suppose that at each level of our tree we have, after pruning, at most $q(r(m))$ nodes left active. Since $q(r(m))$ itself is a polynomial in the input size, $m$, that means our tree remains at most polynomially bushy (since levels of our tree are never, even right after splitting a level’s nodes to create the next level, wider than $2q(r(m))$). Analogously to the argument of Challenge Problem 2’s proof, when we reach the “all variables assigned” level, we enter it with a set of at most $2q(r(m))$ no-variables-left formulas such that $F$ is satisfiable if and only if at least one of those formulas evaluates to $\mathrm{True}$. So in that case, we easily do compute in polynomial time whether the given input is satisfiable, analogously to the previous proof.

On the other hand, suppose that on some level, after pruning, we have at least $1+q(r(m))$ nodes. This means that at that level, we had at least $1+q(r(m))$ distinct labels. But there are only $q(r(m))$ distinct strings that $g$ can possibly reach, on our inputs, that belong to $S$. So at least one of the $1+q(r(m))$ formulas in our surviving nodes maps to a string that does not belong to $S$. But $g$ was a reduction from $\overline{{\rm SAT}}$ to $S$, so that node that mapped to a string that does not belong to $S$ must itself be a satisfiable formula. Ka-zam! That node is satisfiable, and yet that node is simply $F$ with some of its variables fixed. And so $F$ itself certainly is satisfiable. We are done, and so the moment our algorithm finds a level that has $1+q(r(m))$ distinct labels, our algorithm halts and declares that $F(x_{1},x_{2},\dots,x_{k})$ is satisfiable.

Note how subtle the action here is. The algorithm is correct in reasoning that, when we have at least $1+q(r(m))$ distinct labels at a level, at least one of the still-live nodes at that level must be satisfiable, and thus $F(x_{1},x_{2},\dots,x_{k})$ is satisfiable. However, the algorithm doesn’t know a particular one of those at-least-$1+q(r(m))$-nodes that it can point to as being satisfiable. It merely knows that at least one of them is. And that is enough to allow the algorithm to act correctly. (One can, if one wants, extend the above approach to actually drive onward to the base of the tree; what one does is that at each level, the moment one gets to $1+q(r(m))$ distinct labels, one stops handling that level, and goes immediately on to the next level, splitting each of those $1+q(r(m))$ nodes into two at the next level. This works since we know that at least one of the nodes is satisfiable, and so we have ensured that at least node at the next level will be satisfiable.) This completes the proof sketch.

And now, please pop right on back to the main body of the chapter, to read and tackle Challenge Problem 4! There, you’ll be working within a related but changed and rather challenging setting: you’ll be working in the realms of functions and counting. Buckle up!

Appendix D Solution to Challenge Problem 4

D.1 Why One Natural Approach Is Hopeless

One natural approach would be to run the hypothetical 2-enumerator $h$ on the input formula $F$ and both of $F$’s $x_{1}$-assigned subformulas, and to argue that purely based on the two options that $h$ gives for each of those three, i.e., viewing the formulas for a moment as black boxes (note: without loss of generality, we may assume that each of the three applications of the 2-enumerator has two distinct outputs; the other cases are even easier), we can either output $\|F\|$ or can identify at least one of the subformulas such that we can show a particular 1-to-1 linkage between which of the two predicted numbers of solutions it has and which of the two predicted numbers of solutions $F$ has. And then we would iteratively walk down the tree, doing that.

But the following example, based on one suggested by Gerhard Woeginger, shows that that is impossible. Suppose $h$ predicts outputs $\{0,1\}$ for $F$, and $h$ predicts outputs $\{0,1\}$ for the left subformula, and $h$ predicts outputs $\{0,1\}$ for the right subformula. That is, for each, it says “this formula either has zero satisfying assignments or has exactly one satisfying assignment.” In this case, note that the values of the root can’t be, based solely on the numbers the enumerator output, linked 1-to-1 to those of the left subformula, since [math] solutions for the left subformula can correspond to a root value of 0 ($0+0=0$) or to a root value of 1 ($0+1=1$). The same clearly also holds for the right subformula.

The three separate number-pairs just don’t have enough information to make the desired link! But don’t despair: we can make $h$ help us far more powerfully than was done above!

D.2 XYZ Idea/Statement

To get around the obstacle just mentioned, we can try to trick the enumerator into giving us linked/coordinated guesses! Let us see how to do that.

What I was thinking of, when I mentioned XYZ in the food-for-thought hint (Section 6.2), is the fact that we can efficiently combine two Boolean formulas into a new one such that from the number of satisfying assignments of the new formula we can easily “read off” the number of satisfying assignments of both the original formulas. In fact, it turns out that we can do the combining in such a way that if we concatenate the (appropriately padded as needed) bitstrings capturing the numbers of solutions of the two formulas, we get the (appropriately padded as needed) bitstring capturing the number of solutions of the new “combined” formula. We will, when $F$ is a Boolean formula, use $\|F\|$ to denote the number of satisfying assignments of $F$.

Lemma 1**.**

There are polynomial-time computable functions $\mathrm{combiner}$ and $\mathrm{decoder}$ such that for any Boolean formulas $F$ and $G$, $\mathrm{combiner}(F,\,G)$ is a Boolean formula and $\mathrm{decoder}(F,G,\|\mathrm{combiner}(F,G)\|)$ prints $\|F\|,\|G\|$.

Proof Sketch.

Let $F=F(x_{1},\ldots,\,x_{n})$ and $G=G(y_{1},\ldots,\,y_{m})$, where $x_{1},\ldots,\,x_{n},y_{1},\ldots,\,y_{m}$ are all distinct. Let $z$ and $z^{\prime}$ be two new Boolean variables. Then

[TABLE]

gives the desired combination, since $\|h\|=\|f\|2^{m+1}+\|g\|$ and $\|g\|\leq 2^{m}$. ∎

We can easily extend this technique to combine three, four, or even polynomially many formulas.

D.3 Invitation to a Second Bite at the Apple

Now that you have in hand the extra tool that is Lemma 1, this would be a great time, unless you already found a solution to the fourth challenge problem, to try again to solve the problem. My guess is that if you did not already solve the fourth challenge problem, then the ideas you had while trying to solve it will stand you in good stead when you with the combining lemma in hand revisit the problem.

My suggestion to you would be to work again on proving Challenge Problem 4 until either you find a proof, or you’ve put in at least 15 more minutes of thought, are stuck, and don’t think that more time will be helpful.

When you’ve reached one or the other of those states, please go on to Section D.4 to read a proof of the theorem.

D.4 Proof Sketch of the Theorem

Recall that we are trying to prove:

If ${\rm\#SAT}$ is has a polynomial-time $2$-enumerator, then there is a polynomial-time algorithm for ${\rm\#SAT}$.

Here is a quick proof sketch. Start with our input formula, $F$, whose number of solutions we wish to compute in polynomial time.

If $F$ has no variables, we can simply directly output the right number of solutions, namely, 1 (if $F$ evaluates to $\mathrm{True}$), or 0 (otherwise). Otherwise, self-reduce formula $F$ on its first variable. Using the XYZ trick, twice, combine the original formula and the two subformulas into a single formula, $H$, whose number of solutions gives the number of solutions of all three. For example, if our three formulas are $F=F(x_{1}x_{2},x_{3},\dots)$, $F_{\mathit{left}}=F(\mathrm{True},x_{2},x_{3},\dots)$, and $F_{\mathit{right}}=F(\mathrm{False},x_{2},x_{3},\dots)$, our combined formula can be

[TABLE]

and the decoding process is clear from this and Lemma 1 (and its proof). Run the 2-enumerator on $H$. If either of $H$’s output’s two decoded guesses are inconsistent ($a\neq b+c$), then ignore that line and the other one is the truth. If both are consistent and agree on $\|F\|$, then we’re also done. Otherwise, the two guesses must each be internally consistent and the two guesses must disagree on $\|F\|$, and so it follows that the two guesses differ in their claims about at least one of $\|F_{\mathit{left}}\|$ and $\|F_{\mathit{right}}\|$. Thus if we know the number of solutions of that one, shorter formula, we know the number of solutions of $\|F\|$.

Repeat the above on that formula, and so on, right on down the three, and then (unless the process resolves internally or ripples back up earlier) at the end we have reached a zero-variable formula and for it we by inspection will know how many solutions it has (either 1 or 0), and so using that we can ripple our way all the way back up through the tree, using our linkages between each level and the next, and thus we now have computed $\|F\|$. The entire process is a polynomial number of polynomial-time actions, and so runs in polynomial time overall.

That ends the proof sketch, but let us give an example regarding the key step from the proof sketch, as that will help make clear what is going on.

[TABLE]

In this example, note that we can conclude that $\|F\|=100$ if $\|F(\mathrm{False},x_{2},x_{3},\dots)\|=17$, and $\|F\|=101$ if $\|F(\mathrm{False},x_{2},x_{3},\dots)\|=16$; and we know that $\|F(\mathrm{False},x_{2},x_{3},\dots)\|\in\{16,17\}$.

So we have in polynomial time completely linked $\|F(x_{1},x_{2},x_{3},\dots)\|$ to the issue of the number of satisfying assignments of the (after simplifying) shorter formula $F(\mathrm{False},x_{2},x_{3},\dots)$. This completes our example of the key linking step.