Statistical physics is the first topic in the seminar course I am co-teaching with Boaz this fall, and one of our primary goals is to explore this theory. This blog post is a re-working of a lecture I gave in class this past Friday. It is meant to serve as an introduction to statistical physics, and is composed of two parts: in the first part, I introduce the basic concepts from statistical physics in a hands-on manner, by demonstrating a phase transition for the Ising model on the complete graph. In the second part, I introduce random k-SAT and the satisfiability conjecture, and give some moment-method based proofs of bounds on the satisfiability threshold.

*Update on September 16, 3:48pm: the first version of this post contained an incorrect plot of the energy density of the Ising model on the complete graph, which I have amended below.*

In statistical physics, the goal is to understand how materials behave on a macroscopic scale based on a simple model of particle-particle interactions.

For example, consider a block of iron. In a block of iron, we have many iron particles, and each has a net -polarization or “spin” which is induced by the quantum spins of its unpaired electrons. On the microscopic scale, nearby iron atoms “want” to have the same spin. From what I was able to gather on Wikipedia, this is because the unpaired electrons in the distinct iron atoms repel each other, and if two nearby iron atoms have the same spins, then this allows them to be in a physical configuration where the atoms are further apart in space, which results in a **lower energy state** (because of the repulsion between electrons).

When most of the particles in a block of iron have correlated spins, then on a macroscopic scale we observe this correlation as the phenomenon of magnetism (or ferromagnetism if we want to be technically correct).

In the 1890’s, Pierre Curie showed that if you heat up a block of iron (introducing energy into the system), it eventually loses its magnetization. In fact, magnetization exhibits a **phase transition**: there is a critical temperature, , below which a block of iron will act as a magnet, and above which it will suddenly lose its magnetism. This is called the “Curie temperature”. This phase transition is in contrast to the alternative, in which the iron would gradually lose its magnetization as it is heated.

We’ll now set up a simple model of the microscopic particle-particle interactions, and see how the global phenomenon of the magnetization phase transition emerges. This is called the *Ising model*, and it is one of the more canonical models in statistical physics.

Suppse that we have iron atoms, and that their interactions are described by the (for simplicity unweighted) graph with adjacency matrix . For example, we may think of the atoms as being arranged in a 3D cubic lattice, and then would be the 3D cubic lattice graph. We give each atom a label in , and we associate with each atom a spin .

For each choice of spins or *state* we associate the *total energy*

.

If two interacting particles have the same spin, then they are in a “lower energy” configuration, and then they contribute to the total energy. If two neighboring particles have opposite spins, then they are in a “higher energy” configuration, and they contribute to the total energy.

We also introduce a *temperature* parameter . At each , we want to describe what a “typical” configuration for our block of iron looks like. When , there is no kinetic energy in the system, so we expect the system to be in the lowest-energy state, i.e. all atoms have the same spin. As the temperature increases, the kinetic energy also increases, and we will begin to see more anomalies.

In statistical physics, the “description” takes the form of a probability distribution over states . To this end we define the *Boltzmann distribution*, with density function :

As , , becomes supported entirely on the that minimize ; we call these the *ground states* (for connected these are exactly ). On the other hand as , all are weighted equally according to .

Above we have defined the Boltzmann distribution to be proportional to . To spell it out,

The normalizing quantity is referred to as the *partition function*, and is interesting in its own right. For example, from we can compute the *free energy* of the system, as well as the *internal energy* and the *entropy* :

Using some straightforward calculus, we can then see that is the Shannon entropy,

that is the average energy in the system,

and that the free energy is the difference of the internal energy and the product of the temperature and the entropy,

just like the classical thermodynamic definitons!

The free energy, internal energy, and entropy encode information about the typical behavior of the system at temperature . We can get some intuition by considering the extremes, and .

In cold systems with , if we let be the energy of the ground state, be the number of ground state configurations, and be the energy gap, then

where the notation hides factors that do not depend on . From this it isn’t hard to work out that

We can see that the behavior of the system is dominated by the few ground states. As , all of the free energy can be attributed to the internal energy term.

On the other hand, as ,

and the behavior of the system is chaotic, with the free energy dominated by the entropy term.

We say that the system undergoes a *phase transition* at if the *energy density* is not analytic at . Often, this comes from a shift in the relative contributions of the internal energy and entropy terms to . Phase transitions are often associated as well with a **qualitative** change in system behavior.

For example, we’ll now show that for the Ising model with the complete graph with self loops, the system has a phase transition at (the self-loops don’t make much physical sense, but are convenient to work with). Furthermore, we’ll show that this phase transition corresponds to a qualitative change in the system, i.e. the loss of magnetism.

Define the *magnetization* of the system with spins to be . If , then we say the system is *magnetized*.

In the complete graph, normalized so that the total interaction of each particle is , there is a direct relationship between the energy and the magnetization:

The magnetization takes values for . So, letting be the number of states with magnetization , we have that

Now, is just the number of strings with Hamming weight , so . By Stirling’s approximation , where is the entropy function, so up to lower-order terms

Now we apply the following simplification: for , , and then . Treating our summands as the entries of the vector , from this we have,

By definition of the energy density,

and since independently of , we also have

because the error from rounding to the nearest factor of is .

We can see that the first term in the expression for corresponds to the square of the magnetization (and therefore the energy); the more magnetized the system is, the larger the contribution from the first term. The second term corresponds to the entropy, or the number of configurations in the support; the larger the support, the larger the contribution of the second term. As , the contribution of the entropy term overwhelms the contribution of the energy term; this is consistent with our physical intuition.

We’ll now demonstrate that there is indeed a phase transition in . To do so, we solve for this maximum. Taking the derivative with respect to , we have that

so the derivative is whenever . From this, we can check the maxima. When , there are two maxima equidistant from the origin, corresponding to negatively or positively-magnetized states. When , the maximizer is , corresponding to an unmagnetized state.

Given the maximizers, we now have the energy density. When we plot the energy density, the phase transition at is subtle (an earlier version of this post contained a mistaken plot):

But when we plot the derivative, we can see that it is not smooth at :

And with some calculus it is possible to show that the second derivative is indeed not continuous at .

Qualitatively, it is convincing that this phase transition in the energy density is related to a transition in the magnetization (because the maximizing corresponds to the typical magnetization). One can make this formal by performing a similar calculation to show that the internal energy undergoes a phase transition, which in this case is proportional to the expected squared magnetization, .

The Ising model on the complete graph (also called the Curie-Weiss model) is perhaps not a very convincing model for a physical block of iron; we expect that locality should govern the strength of the interactions. But because the energy and the magnetization are related so simply, it is easy to solve.

Solutions are also known for the 1D and 2D grids; solving it on higher-dimensional lattices, as well as in many other interesting settings, remains open. Interestingly, the conformal bootstrap method that Boaz mentioned has been used towards solving the Ising model on higher-dimensional grids.

For those familiar with constraint satisfaction problems (CSPs), it may have already been clear that the Ising model is a CSP. The spins are Boolean variables, and the energy function is an objective function corresponding to the EQUALITY CSP on (a pretty boring CSP, when taken without negations). The Boltzmann distribution gives a probability distribution over assignments to the variables , and the temperature determines the objective value of a typical .

We can similarly define the energy, Boltzmann distribution, and free energy/entropy for any CSP (and even to continuous domains such as ). Especially popular with statistical physicists are:

- CSPs (such as the Ising model) over grids and other lattices.
- Gaussian spin glasses: CSPs over or where the energy function is proportional to , where is an order- symmetric tensor with entries sampled i.i.d. from . The Ising model on a graph with random Gaussian weights is an example for .
- Random instances of -SAT: Of the possible clauses (with negations) on variables, clauses are sampled uniformly at random, and the energy function is the number of satisfied clauses, .
- Random instances of other Boolean and larger-alphabet CSPs.

In some cases, these CSPs are reasonable models for physical systems; in

other cases, they are primarily of theoretical interest.

As theoretical computer scientists, we are used to seeing CSPs in the contex of optimization. In statistical physics, the goal is to understand the qualitative behavior of the system as described by the Boltzmann distribution. They ask algorithmic questions such as:

- Can we estimate the free energy ? The partition function ? And, relatedly,
- Can we sample from ?

But these tasks are not so different from optimization. For example, if our system is an instance of 3SAT, when , the Boltzmann distribution is the uniform distribution over maximally satisfying assignments, and so estimating is equivalent to deciding the SAT formula, and sampling from is equivalent to solving the SAT formula. As increases, sampling from corresponds to sampling an approximate solution.

Clearly in the worst case, these tasks are NP-Hard (and even #P-hard). But even for random instances these algorithmic questions are interesting.

In random -SAT, the system is controlled not only by the temperature but also by the *clause density* . For the remainder of the post, we will focus on the zero-temperature regime , and we will see that -SAT exhibits phase transitions in as well.

The most natural “physical” trait to track in a -SAT formula is whether or not it is *satisfiable*. When , -SAT instances are clearly satisfiable, because they have no constraints. Similarly when , random -SAT instances cannot be satisfiable, because for any set of variables they will contain all possible -SAT constraints (clearly unsatisfiable). It is natural to ask: is there a satisfiability phase transition in ?

For , one can show that the answer is yes. For , numerical evidence strongly points to this being the case; further, the following theorem of Friedgut gives a partial answer:

**Theorem:**

For there exists a function such that for any , if is a random -SAT formula on variables with clauses, then

However, this theorem allows for the possibility that the threshold depends on . From a statistical physics standpoint, this would be ridiculous, as it suggests that the behavior of the system depends on the number of particles that participate in it. We state the commonly held stronger conjecture:

**Conjecture:**

for all , there exists a constant depending only on such that if is a random -SAT instance in variables and clauses, then

In 2015, Jian Ding, Allan Sly, and Nike Sun established the -SAT conjecture all larger than some fixed constant , and we will be hearing about the proof from Nike later on in the course.

Let us move to a simpler model of random -SAT formulas, which is a bit easier to work with and is a close approximation to our original model. Instead of sampling -SAT clauses without replacement, we will sample them with replacement and also allow variables to appear multiple times in the same clause (so each literal is chosen uniformly at random). The independence of the clauses makes computations in this model simpler.

We’ll prove the following bounds. The upper bound is a fairly straightforward computation, and the lower bound is given by an elegant argument due to Achlioptas and Peres.

**Theorem:**

For every , .

Let be a random -SAT formula with clauses, . For an assignment , let be the indicator that satisfies . Finally, let be the number of satisfying assignments of .

We have by Markov’s inequality that

Fix an assignment . Then by the independence of the clauses,

since each clause has satisfying assignments. Summing over all

,

We can see that if , this quantity will go to with . So we have:

To lower bound , we can use the *second moment method*. We’ll

calculate the second moment of . An easy use of Cauchy-Schwarz (for non-negative , ) implies that if there is a constant such that

,

then with at least constant probability , and then Friedgut’s theorem implies that . From above we have an expression for the numerator, so we now set out to bound the second moment of . We have that

and by the independence of the clauses,

for a single random -SAT clause . But, for , the events and are not independent. This is easier to see if we apply inclusion-exclusion,

The event that both when is a -SAT clause occurs only when and agree exactly on the assignments to the variables of , since otherwise at least one of or must be satisfied (because each literal in is negated for ). Thus, this probability depends on the Hamming distance between and .

For with , the probability that ‘s variables are all in the subset on which agree is (up to lower order terms). So, we have

and then for with ,

Because there are pairs at distance ,

and using the Stirling bound ,

Using Laplace’s method, we can show that this sum will be dominated by the terms around the maximizing summand; so defining ,

If we want that , then we require that

However, we can see (using calculus) that this inequality does not hold whenever :

So the naive second moment method fails to establish any lower bound on . The problem here is that the second moment is dominated by the large correlations of close strings; whenever , the sum is dominated by pairs of strings which are closer than in Hamming distance, which is atypical. For strings at Hamming distance , what is relevant is , which is always

which is equal to . At distance , the value of on pairs is essentially uncorrelated (one can think of drawing a uniformly random given ), so such pairs are representative of .

To get a good bound, we have to perform a fancier second moment method calculation, due to Achlioptas and Peres. We will re-weight the terms so that more typical pairs, at distance , are dominant. Rather than computing , we compute , where

where if , and if is satisfied by variables. Since only if has satisfying assignments, the goal is to still bound . Again using the independence of the clauses, we have that

And calculating again the second moment,

For a fixed clause , we can partition the variables in its scope into 4 sets, according to whether agree on the variable, and whether the variable does or does not satisfy the literals of the clause. Suppose that has variables on which agree, and that of these are satisfying for and are not.

Then, has variables on which disagree, and any variable which does not satisfy for must satisfy it for . For a to be nonzero, either there must be at least one literal in the variables on which agree which agrees with , or otherwise there must be at least one literal in the variables on which disagree which agrees with each string. Therefore, if agree on a -fraction of variables,

where the first sum ignores the cases when or , the second sum subtracts for the cases when the contribution of the terms where all the literals agree with either or , and the final term accounts for the fact that the term is subtracted

twice. Simplifying,

Define

.

So we have (using Laplace’s method again)

For

When ,

which is equal to the log of the square of the expectation, so again for the second moment method to succeed, must be the global maximum.

Guided by this consideration, we can set so that the derivative , so that achieves a local maximum at . The choice for this (after doing some calculus) turns out to be the positive, real solution to the equation . With this choice, one can show that the global maximum is indeed achieved at as long as . Below, we plot with this optimal choice of for several values of at :

So we have bounded .

What is the correct answer for ? We now have it in a window of size . Experimental data and heuristic predictions indicate that it is closer to (and in fact for large , Ding, Sly and Sun showed that is a specific constant in the interval ). So why can’t we push the second moment method further?

It turns out that there is a good reason for this, having to do with another phase transition. In fact, we know that -SAT has not only satisfiable and unsatisfiable phases, but also *clustering* and *condensation* phases:

In the clustering phase, there are exponentially many clusters of solutions, each containing exponentially many solutions, and each at hamming distance from each other. In the condensation phase, there are even fewer clusters, but solutions still exist. We can see evidence of this already in the way that the second moment method failed us. When , we see that the global maximum of the exponent is actually attained close to . This is because solutions with large overlap come to dominate the set of satisfying assignments.

One can also establish the existence of clusters using the method of moments. The trick is to compute the probability that two solutions, and with overlap , are both satisfying for . In fact, we have already done this. From above,

Now, by a union bound, an upper bound on the probability that there exists a pair of solutions with overlap for any is at most

and if the function is such that for all , then we conclude that the probability that there is a pair of satisfying assignments at distance between and is .

Achlioptas and Ricci-Tersenghi showed that for , , and a fixed constant, the above function is less than . Rather than doing the tedious calculus, we can verify by plotting for (with ):

They also use the second moment method to show the clusters are non-empty, that there are exponentially many of them, each containing exponentially many solutions. This gives a proof of the existence of a clustering regime.

This study of the space of solutions is referred to as *solution geometry*, and understanding solution geometry turns out to be essential in proving better bounds on the critical density.

Solution geometry is also intimately related to the success of local search algorithms such as belief propagation, and to heuristics for predicting phase transitions such as replica symmetry breaking.

These topics and more to follow, in the coming weeks.

In preparing this blog post/lecture I leaned heavily on Chapters 2 and 10 of Marc Mézard and Andrea Montanari’s “Information, Physics, and Computation”, on Chapters 12,13,&14 of Cris Moore and Stephan Mertens’ “The nature of Computation,” and on Dimitris Achlioptas and Federico Ricci-Tersenghi’s manuscript “On the Solution-Space Geometry of Random CSPs”. I also consulted Wikipedia for some physics basics.

]]>This is a kind reminder that the deadline for the early rate registration fees for FOCS 2018 is this **Sunday, September 9, 2018.**

]]>===================================

FOCS 2018 – Second Call for Participation

===================================

https://www.irif.fr/~focs2018/

The 59th Annual IEEE Symposium on Foundations of Computer Science (FOCS 2018) will take place in Paris, France, on 7-9 October 2018, with workshops and tutorials on October 6.

The program is now available, together with the list of the workshops/tutorials (as well as a list of a number of co-located events), on the conference webpage.

**Early registration rate ends September 9, 2018.**

All scientific and local information, and a link to the registration page, can be found at https://www.irif.fr/~focs2018/

Looking forward to seeing you in Paris !

One of the interesting features of physics is the prevalence of “thought experiments”, including Maxwell’s demon, Einstein’s Train, Schrödinger’s cat, and many more. One could think that these experiments are merely “verbal fluff” which obscures the “real math” but there is a reason that physicists return time and again to these types of mental exercises. In a nutshell, this is because while physicists use math to *model* reality, the mathematical model is *not equal* to reality.

For example, in the early days of quantum mechanics, several calculations of energy shifts seemed to give out infinite numbers. While initially this was viewed as a sign that something is deeply wrong with quantum mechanics, ultimately it turned out that these infinities canceled each other, as long as you only tried to compute *observable quantities*. One lesson that physicists drew from this is that while such mathematical inconsistencies may (and in this case quite possibly do) indicate some issue with a theory, they are not a reason to discard it. It is OK if a theory involves mathematical steps that do not make sense, as long as this does not lead to an *observable paradox*: i.e., an actual “thought experiment” with a nonsensical outcome.

A priori, this seems rather weird. An outsider impression of the enterprise of physics is that it is all about explaining the behavior of larger systems in terms of smaller parts. We explain materials by molecules, molecules by atoms, and atoms by elementary particles. Every term in our mathematical model is supposed to correspond to something “real” in the world.

However, with modern physics, and particular quantum mechanics, this connection breaks down. In quantum mechanics we model the state of the world using a vector (or “wave function”) but the destructiveness of quantum measurements tells us that we can never know all the coordinates of this vector. (This is also related to the so called “uncertainty principle”.) While physicists and philosophers can debate whether these wave functions “really exist”, their existence is not the reason why quantum mechanics is so successful. It is successful because these wave functions yield a mathematically *simple* model to predict observations. Hence we have moved from trying to explain bigger physical systems in terms of *smaller physical systems* to trying to explain complicated observations in terms of *simpler mathematical models*. (Indeed the focus has moved from “things” such as particles to concepts such as forces and symmetries as the most fundamental notions.) These simpler models do not *necessarily* correspond to any real physical entities that we’d ever be able to observe. Hence such models can in principle contain weird things such as infinite quantities, as long as these don’t mess up our predictions for actual observations.

Nevertheless, there are still real issues in physics that people have not been able to settle. In particular the so called “standard model” uses quantum mechanics to explain the strong force, the weak force, and the electromagnetic force, which dominate over short (i.e., subatomic) distances, but it does not incorporate the force of gravity. Gravity is explained by the theory of general relativity which is inconsistent with quantum mechanics but is predictive for phenomena over larger distances.

By and large physicists believe that quantum mechanics will form the basis for a unified theory, that will involve incorporating gravity into it by putting general relativity on quantum mechanical foundations. One of the most promising approaches in this direction is known as the AdS/CFT correspodence of Maldacena, which we describe briefly below.

Alas, in 2012, Almheiri, Marolf, Polchinski, and Sully (AMPS) gave a description of a mental experiment, known as a the “firewall paradox” that showed a significant issue with any quantum-mechanical description of gravity, including the Ads/CFT correspondence. Harlow and Hayden (see also chapter 6 of Aaronson’s notes and this overview of Susskind) proposed a way to resolve this paradox using computational complexity.

In this post I will briefly discuss these issues. Hopefully someone in Tselil’s and my upcoming seminar will present this in more detail and also write a blog post about it.

Edwin Abbott’s 1884 novel “Flatland”, describes a world in which people live in only two dimensions. At some point a sphere visits this world, and opens the eye of one of its inhabitants (the narrator, which is a square) to the fact its two-dimensional world was merely an illusion and the “real world” actually has more dimensions.

However, modern physics suggest that things might be the other way around: we might actually be living in flatland ourselves. That is, it might be that the true description of our world has one less spatial dimension than what we perceive. For example, though we think we live in three dimensions, perhaps we are merely shadows (or a “hologram”) of a two dimensional description of the world. One can ask how could this be? After all, if our world is “really” two dimensional, what happens when I climb the stairs in my house? The idea is that the geometry of the two-dimensional world is radically different, but it contains all the *information* that would allow to decode the state of our three dimensional world. You can imagine that when I climb the stairs in my house, my flatland analog goes from the first floor to the second floor in (some encoding of) the two-dimensional *blueprint* of my house. (Perhaps this lower-dimensional representation is the reason the Wachowskis called their movie *“The Matrix”* as opposed to *“The Tensor”*?)

The main idea is that in this “flat” description, gravity does not really exist and physics has a pure quantum mechanical description which is *scale free* in the sense that the theory is the same independently of distance. Gravity and our spacetime gemoetry *emerge* in our world via the projection from this lower dimensional space. (This projection is supposed to give rise to some kind of string theory.) As far as I can tell, at the moment physicists can only perform this projection (and even this at a rather heuristic level) under the assumption that our universe is *contracting* or in physics terminology an “anti de-Sitter (AdS) space”. This is the assumption that the geometry of the universe is hyperbolic and hence one can envision spacetime as being bounded in some finite area of space: some kind of a dimensional cylinder that has a -dimensional *boundary*. The idea is that all the information on what’s going on in the inside or *bulk* of the cylinder is encoded in this boundary. One caveat is that our physical universe is actually *expanding* rather than contracting, but as the theory is hard enough to work out for a contracting space, at the moment they sensibly focus on this more tractable setting. Since the quantum mechanical theory on the boundary is scale free (and also rotation invariant) it is known as a *Conformal Field Theory (CFT)*. Thus this one to one mapping of the boundary and the bulk is also known as the “AdS/CFT correspondence”.

If it is possible to carry over this description, in terms of *information* it would be possible to describe the universe in purely quantum mechanical terms. One can imagine that the universe starts at some quantum state , and at each step in time progresses to the state where is some unitary transformation.

In particular this means that information is never *lost*. However, *black holes* pose a conundrum to this view since they seem to swallow all information that enters them. Recall that the “escape velocity” of earth – the speed needed to escape the gravitational field and go to space – is about 25,000 mph or Mach 33. In a black hole the “escape velocity” is the speed of light which means that *nothing*, not even light, can escape it. More specifically, there is a certain region in spacetime which corresponds to the *event horizon* of a black hole. Once you are in this event horizon then you have passed the point of no return since even if you travel in the speed of light, you will not be able to escape. Though it might take a *very* long time, eventually you will perish in the black hole’s so called “singularity”.

Entering the event horizon should not feel particularly special (a condition physicists colorfully refer to as *“no drama”*). Indeed, as far as I know, it is theoretically possible that 10 years from now a black hole would be created in our solar system with a radius larger than 100 light years. If this future event will happen, this means that we are *already* in a black hole event horizon even though we don’t know it.

The above seems to mean that information that enters the black hole is irrevocably lost, contradicting unitarity. However, physicists now believe that through a phenomenon known as Hawking radiation black holes might actually emit the information that was contained in them. That is, if the qubits that enter the event horizon are in the state then (up to a unitary tranformation) the qubits that are emitted in the radiation would be in the state as well, and hence no information is lost. Indeed, Hawking himself conceded the bet he made with Preskill on information loss.

Nevertheless, there is one fly in this ointment. If we drop an qubit state in this black hole, then they are eventually radiated (in the same state, up to an invertible transformation), but the original qubits never come out. (It is a black hole after all.) Since we now have two copies of these qubits (one inside the black hole and one outside it), this seems to violate the famous “no cloning principle” of quantum mechanics which says that you can’t copy a qubit. Luckily however, this seemed to be one more of those cases where it is an issue with the math that could never effect an actual observer. The reason is that an observer inside the black hole event horizon can never come out, while an observer outside can never peer inside. Thus, even if the no cloning principle is violated in our mathematical model of the whole universe, no such violation would have been seen by either an outside or an inside observer. In fact, even if Alice – a brave observer outside the event horizon – obtained the state of the Hawking radiation and then jumped with it into the event horizon so that she can see a violation of the no-cloning principle then it wouldn’t work. The reason is that by the time all the qubits are radiated, the black hole fully evaporates and inside the black hole the original qubits have already entered the singularity. Hence Alice would not be able to “catch the black hole in the act” of cloning qubits.

What AMPS noticed is that a more sophisticated (yet equally brave) observer could actually obtain a violation of quantum mechanics. The idea is the following. Alice will wait until almost all (say 99 percent) of the black hole evaporated, which means that at this point she can observe of the qubits of the Hawking radiation , while there are still about qubits inside the event horizon that have not yet reached the singularity. So far, this does not seem to be any violation of the no cloning principle, but it turns out that *entanglement* (which you can think of as the quantum analog of mutual information) plays a subtle role. Specifically, for information to be preserved the radiation will be in a *highly entangled* state, which means that in particular if we look at the qubit that has just radiated from the event horizon then it will be highly entangled with the qubits we observed before.

On the other hand, from the continuity of spacetime, if we look at a qubit that is just adjacent to but *inside* the event horizon then it will be highly entangled with as well. For our classical intuition, this seems to be fine: a -valued random variable could have large (say at least ) mutual information with two distinct random variables and . But quantum entanglement behaves differently: it satisfies a notion known as *monogamy of entanglement*, which implies that the sum of entanglement of a qubit with two disjoint registers can be at most one. (Monogamy of entangelement is actually equivalent to the no cloning principle, see for example slide 14 here.)

Specifically, Alice could use a unitary transformation to “distill” from a qubit that is highly entangled with and then jump with and into the event horizon to observe there a triple of qubits which violates the monogamy of entanglement.

One potential solution to the AMPS paradox is to drop the assumption that spacetime is continuous at the event horizon. This would mean that there is a huge energy barrier (i.e., a “firewall”) at the event horizon. Alas, a huge wall of fire is as close as one can get to the definition of “drama” without involving Omarose Manigault Newman.

The “firewall paradox” is a matter of great debate among physicists. (For example after the AMPS paper came out, a “rapid response workshop” was organized for people to suggest possible solutions.) As mentioned above, Daniel Harlow and Patrick Hayden suggested a fascinating way to resolve this paradox. They observed that to actually run this experiment, Alice would have to apply a certain “entanglement distillation” unitary to the qubits of the Hawking radiation. However, under reasonable complexity assumptions, computing would require an *exponential number of quantum gates!*. This means that by the time Alice is done with the computation, the black hole is likely to completely evaporate, and hence there would be nothing left to jump into!

The above is by no means the last word of this story. Other approaches for resolving this paradox have been put forward, as well as ways to poke holes in the Harlow-Hayden resolution. Nor is it the only appearance of complexity in the AdS/CFT correspondence or quantum gravity at large. Indeed, the whole approach places much more emphasis on the *information* content of the world as opposed to the more traditional view of *spacetime* as the fundamental “canvas” for our universe. Hence information and computation play a key role in understanding how our spacetime can emerge from the conformal picture.

In the fall seminar, we will learn more about these issues, and will report here as we do so.

]]>Johan will be presented with the award at the upcoming FOCS.

]]>(However there is no mention of Scott’s role as the criminal mastermind behind the great Philadelphia Airport Heist.)

While it’s by no means “great literature”, Factor Man is a fun page-turner. I think it can be a particularly enjoyable read for computer scientists, as it might prompt you to come up with your own scenarios as to how things would play out if someone discovers such an algorithm. Unsurprisingly, the book is not technically perfect. The technical error that annoyed me the most was that the protagonist demonstrates his algorithm by factoring integers of sizes that can in fact be fairly easily factored today (the book refers to factoring 128 or 256 bit numbers as impressive, while 768 bit integers of general form have been factored, see also this page and this paper). If you just imagine that when the book says “ bit” numbers it actually means *byte* numbers then this is fine. Network security researchers might also take issue with other points in the book (such as the ability of the protagonist to use gmail and blogspot without being identified by neither the NSA nor Google, as well as using a SAT algorithm to provide a “final security patch” for a product).

Regardless of these technical issues, I recommend reading this book if you’re the type of person that enjoys both computer science and spy thrillers, and I do plan to mention it to students taking my introduction to theoretical CS course.

]]>But a deeper reason to envy physicists is that (with certain important exceptions) they often have fairly good intuitions into how their systems of interest behave, even if they can’t always prove them. In contrast, we theoretical computer scientists are more often than not completely “in the dark”. A -step algorithm to compute the mapping can be modeled as applications of some simple local update rule to the initial state . Studying the evolution of systems is the bread-and-butter of physics, but many physical intuitions fail for the progression of an algorithm’s computation. For example, for general algorithms, we do not have a natural sense in which the state of the system after steps is “closer” to than it is to . The intermediate state of a general algorithm is rather non informative. Similarly, we do not have a nice, even conjectural, way to characterize the set of functions that can be computed by steps: the lack of such clean “complexity measures” is strongly related to the natural proofs barrier for proving circuit lower bounds.

But there are *some* algorithms for which better “physical intuition” exists. Many optimization algorithms have a “potential function” that improves at every step, and other algorithms such as Monte-Carlo Markov-Chain sampling, are inspired by and can be analyzed using physics intuition. These connections have been recently explored, resulting in both new algorithms as well as better understanding of algorithmic techniques for optimization and learning, as well as the regimes in which they apply.

There are also cases where the computer-science intuition can help in analyzing physical systems. Quantum computers are of course one example, but apparently there are other interesting physical systems (maybe even black holes? see also this summer school) which are “disordered” enough that the best way of thinking of them might be to treat them as random circuits of certain complexity. More generally, in recent years physicists have began to view information and computation as an increasingly useful lens through which to understand physics. The “it from qubit” perspective, whereby spacetime emerges from information rather than the other way around, is growing in popularity.

Finally, on a more “meta” level, the task of doing theoretical physics itself can be thought of as a computational problem. In a fascinating talk, physicist Nima Arkani-Hamed discusses the problem of finding a theory of physics as essentially solving an optimization problem in the *“space of ideas”*. Specifically, it is a *non convex* problem, and so a *local* optimum is not necessarily a *global* one. Arkani-Hamed calls classical physics *“the top of a local mountain in the space of ideas”*, i.e., a *local optimum*, while quantum mechanics is the top of a *“taller mountain”*. The reason it was hard to make the leap to quantum mechanics is exactly because they *“they’re not smoothly connected”*.

By classical physics being a local optimum, we mean that if you try to “tweak” classical physics by turning (in his words) *“knobs, and little wheels, and twiddles”*, you will only get a theory that is less beautiful and with less explanatory power. To get to the better theory of quantum mechanics, one needs to make a conceptual jump, rather than a series of small tweaks. Just like classical physics, quantum mechanics itself is a local optimum, for which every small “tweak” will only make it less beautiful and predictive. This is one explanation as to why it has been so difficult to find the grander theory that unifies general relativity and quantum mechanics. As Harkani-Hamed says, to find such a theory *“there’s going to have to be a jump of a comparable magnitude, in the jump that people have to make in going from classical to quantum”*. In a related point to “it from qubit”, he also says that *“many, many of us suspect that the notion of spacetime can’t be fundamental and it has to be replaced by something else.”*

Interestingly, in certain settings *convex* optimization can be applied to explore the “space of ideas”. In particular some works on the “bootstrap method” use *semidefinite programming* to explore the space of quantum field theories that satisfy certain symmetries. It turns out that sometimes the constraints that one can derive from these symmetries are so powerful, that they completely determine the theory.

The bottom line is that we’re seeing a more and more interesting exchange of ideas between physics and theoretical computer science. As I’m sure I already demonstrated in some cringe-worthy statements above, I know very little about this interface, but am interested in finding out more.

So, Tselil Schramm and I will be running a graduate seminar this fall. We will be learning together with the seminar participants about some of these connections, and hopefully by the end of the term we will all be a little less ignorant.

Some of the topics we will discuss include:

- Connections between statistical physics and algorithms, understanding the physics predictions for hard and easy regimes via
*phase transitions*. *Quantum information theory*: quantum-inspired classical results, as well as classical algorithms for quantum problems.- The
*conformal bootstrap*: exploring the space of possible physical theories using semidefinite programming. *Black holes*,*bulk/boundary correspondence*, and*computational complexity*.*Quantum superiority*– understanding the current proposals for demonstrating exponential speedups for quantum computers, and the evidence for their classical difficulty.*Quantum Hamiltonian Complexity*– the quantum analog of constraint satisfaction problems, with questions such as the existence of a “Quantum PCP Theorem”.

Each one of these is probably worth a semester course on its own, and is typically presented to people with significant physics background. But we are hoping we can create a “tasting menu” and manage to take away some insights and ideas from each of those areas, even if we can’t cover the whole ground.

Participants in the seminar will not only present papers or surveys, but also write a blog post about them, which I will post here, so stay tuned for more information.

]]>As part of the CRYPTO 2018 conference (August 19-23, Santa Barbara, CA), there is a set of of affiliated events. The conference organizers (Tal Rabin, Elette Boyle, and Fabrice Benhamouda) asked me to advertise the workshop Beyond Crypto: A TCS Perspective (itself organized by Yuval Ishai and Guy Rothblum) on August 19th that can be of particular interest to theoretical computer scientists.

Among the speakers will be:

- Aleksander Madry (MIT) will talk about
**“Machine Learning and Security: The Good, the Bad, and the Hopeful”** - Cynthia Dwork (Harvard) will talk about
**“Theory for Society: Crypto on Steroids”**. - Virginia Vassilevska Williams (MIT) will talk about
**“A Fine-Grained Approach to Complexity”** - Mary Wootters (Stanford) will talk about
**“Cryptography, Local Decoding, and Distributed Storage”** - Scott Aaronson (UT Austin) will talk about
**“Certified Randomness from Quantum Supremacy”.**

(I am also speaking in this workshop, and my talk is titled “On Optimal Algorithms and Assumption Factories”).

]]>

Still, while we are all taking down our TheoryFest trees, we can remind ourselves that there are other holidays on the theory calendar. In particular FOCS 2018 will be in October in Paris, and it also contains a “workshop and tutorial day”. If you want to organize a workshop or tutorial, see the call for proposals. Key points are:

- Workshop and Tutorial Day:
**Saturday, October 6, 2018**(Paris) - Workshop and Tutorial Co-Chairs:
**Robert Kleinberg**and**James R. Lee** - Submission deadline:
**August 1st, 2018** - Notification:
**August 6th, 2018** - Send proposals and questions to
**focs2018workshops@gmail.com**

There are worse things in life than organizing a workshop in Paris..

]]>

I just got back from dinner with some of the great speakers who will be at our TheoryFest workshop tomorrow afternoon on computational thresholds for average-case problems, and I am very excited for what’s coming! Since I didn’t get much chance to introduce the speakers in my last post, and not all of them are the “usual suspects” for a STOC workshop, I’d like to take a few paragraphs to do so here, and discuss a little further what they might speak about.

The talks will run the gamut from statistical physics to machine learning to good old theory of computing, and in particular will aim to address questions at their 3-wise intersection. This intersection is full of open problems which are both interesting and approachable. I hope to see lots of people there!

On to the speakers:

**Florent Krzakala: **Florent is a statistical physicist at the Sorbonne in Paris. For more than 10 years he has been one of the leaders in studying high-dimensional statistical inference and average-case computational problems through the lens of statistical physics.

One of my favorite lines Florent’s work was the construction (with several others) of random measurement matrices and corresponding sparse-signal reconstruction algorithms which can recover a random sparse vector with nonzero entries from the measurement , *where the dimension of is only *. (That is, algorithms which recover a -sparse vector from only measurements — not , or even !) This involved bringing together insights from compressed sensing and spin-glass theory in a rather remarkable way.

Lately, Florent tells me he has been interested in the physics of matrix factorization and completion problems, and of neural networks. Tomorrow he is going to discuss computational-versus-statistical gaps for a variety of high-dimensional inference problems, addressing the question of why polynomial-time inference algorithms don’t necessarily achieve information-theoretically-optimal results from a statistical physics perspective.

**Nike Sun: **Nike is a probabilist & computer scientist at UC Berkeley, before which she was the Schramm postdoctoral fellow at MIT and MSR. Many of you may know her from the tour de force work (joint with Jian Ding and Allan Sly) establishing the -SAT threshold for large — this was the biggest progress in years on **the **problem in random CSPs.

Her research spans many topics in probability, but tomorrow she is discussing random CSPs, on which she is among the world experts. In this area her work has made great steps in rigorous-izing the predictions of statistical physics regarding the *geometry of the space of solutions to a random CSP instance. *This geometry is rich: at some clause densities random CSPs seem to have *well-connected *spaces of solutions, and at others the solution spaces are *shattered. *In between are a wealth of phase transitions whose algorithmic implications remain poorly understood (read: a great source of open problems!).

**Jacob Steinardt: **Jacob is a graduating PhD student at Stanford, and a rising star in provable approaches to machine learning. One of his focuses of late has been design of algorithms for* *learning problems in the presence of untrustworthy data, model misspecification, and other challenges the real world imposes on the idealized learning settings we often see here at STOC.

His paper with Moses Charikar and Greg Valiant on *list-learning *has attracted a lot of attention and sparked several follow-up works at this year’s STOC. In that paper, Jacob and his coauthors realized that the problem of learning parameters of a distribution when only a tiny fraction (say 0.001 percent) of your samples come from that distribution provides a generalization of and useful perspective on many classic learning problems, like learning mixture models. (In the list learning setting, one aims to output a *list *of candidate parameters such that one of the sets of parameters is close to the parameters of the distribution you wanted to learn.)

Jacob’s work has explored the notion that polynomial-time tractability of learning problems is intimately related to robustness. A caricature: if any learning problem solvable in polynomial time can also be solved in polynomial time in the presence of some untrustworthy data, then polynomial-time algorithms cannot solve inference problems which are *statistically impossible in the face of untrustworthy data. *This offers an intriguing explanation for the existence of average-case/statistical problems which are unsolvable by polynomial-time algorithms, in spite of their information-theoretic (i.e. exponential-time) solvability.

**I **will also give a talk.

See you there!

**Edit: **correct some attributions.

TheoryFest is in full swing in Los Angeles! There is lots to be written about the wealth of STOC talks, invited papers, keynotes, and (maybe most importantly) the excellent food hall across the street which is swarming with theorists and mathematicians. But for now I want to bring to your attention the workshops planned for Friday.

As always, the schedule is packed with interesting talks! There are workshops on major themes in theory, like

- deep learning
- fairness in algorithms
- recent constructions of extractors
- Vijay Vazirani’s birthday, and
- average-case problems and computational thresholds

Since I helped organize the last of these, let me say a bit about it.

Recently we have seen a surge of interest in algorithms and lower bounds for average-case problems. One driver of this surge has been theory’s increasing connection with machine learning, which has suggested a number of high-dimensional and noisy statistical inference problems to study. Another driver has been the substantial progress on long-standing problems regarding random instances—for example, a series of recent works proving many old conjectures on random *k*-SAT instances, and another series of works analyzing algorithms for community detection and recovery in very sparse random graphs.

A common theme in average-case computational problems is that their *statistical *difficulty (that is, whether they are solvable by any algorithm at all, regardless of running time) is not predictive of their *algorithmic *difficulty (that is, whether they are solvable by polynomial-time algorithms).

A now-standard example is the -community stochastic blockmodel. In this problem, nodes are randomly partitioned into communities, and then a sparse random graph of average degree is sampled on those vertices is sampled by including an edge with probability if are in differing communities and otherwise with probability (this produces a graph of average degree ).

It turns out that so long as for a big-enough constant , in exponential time one may recover the underlying communities from the graph (*approximately, *in the sense that it is possible to partition the graph into classes such that this partitioning has nonvanishing correlation with the ground-truth partition).

However, it is conjectured that this community recovery is possible in polynomial time *if and only if (!!) *. This threshold is conjectured to be *sharp*, in the sense that if no polynomial-time algorithm should exist, while one is known to when .

This kind of threshold phenomenon is ubiquitous for high-dimensional inference problems, and remains largely unexplained. Of course, we are used to algorithmic intractability! But the appearance of intractability for average-case problems seems not to be explained by worst-case theories of hardness (like NP-completeness). And very little is known about what features of an average-case problem are predictive of computational tractability, and which might predict intractability (while in the worst case we know that, say, NP-completeness is an excellent predictor of intractability). The *sharpness *of a threshold like is striking and mysterious in a field where ignoring logarithmic factors is practically a (inter)national pastime.

We will have four talks, spanning algorithms and complexity, with (at least) four perspectives on computational thresholds for average-case problems and related matters. **Florent Krzakala** will lead off with a statistical-physics perspective on algorithms and hardness for matrix and tensor factorization problems. **Nike Sun** will give us a taste of the wild world of random constraint satisfaction problems.** Jacob Steinhardt** will talk about the relationship of outlier robustness and inaccurate/partial problem specification to efficient algorithms for statistical inference. Finally, I will survey some recent developments at the intersection of convex programming, the Sum of Squares method, and high-dimensional inference problems.

I hope to see you there!

]]>