Physics & Computation Blog Post Roundup
In the Fall, Boaz and I cotaught a grad seminar on physics and computation (see here for some of the original press coverage). We were lucky to attract an intrepid group of students from multiple fields, with representatives from computer science, physics, math and biology. As part of the course, we asked our students to give presentations and write expository blog posts on a number of topics at the intersection of computation and physics, including algorithms from statistical physics, quantum area laws, and the firewall paradox. The students worked hard to produce posts that make the physics concepts accessible to a computer science audience, and the result is a nice collection of posts that create a “bridge” from CS to physics.
Here, the aim is to give a (long overdue) table of contents for their posts. Followers of the blog may have noticed a landslide of physics & computation posts around December and January; if you weren’t able to keep up with them all at the time, then here they are, rounded up and organized by topic.
Statistical Physics
Background and intro to phase transitions:
 Statistical Physics: an introduction in two parts (by Tselil Schramm)
Algorithms from statistical physics: belief propagation and approximate message passing:
 Belief Propagation and the Stochastic Block Model (by Thomas Orton)
 Introduction to AMP and the Replica Trick (by Preetum Nakkiran and Yueqi Sheng)
Informational and computational phase transitions:
 Algorithmic and Information Theoretic Decoding Thresholds for Low density ParityCheck Code (by Jeremy Dohmann, Vanessa Wong, and Venkat Arun)
Proving the existence of phase transitions:
 Ising Perceptron under Gaussian Disorder, and kNAESAT (based on a guest lecture by Nike Sun, as scribed by Patrick Guo, VinhKha Le, Shyam Narayanan, and David Stoner)
Using convex relaxations to approximate partition functions:
 Approximating Partition Functions (based on a guest lecture by Andrej Risteski as scribed by Alex Kelser, Alex Lin, Amil Merchant, and Suproteem Sarkar)
Quantum Computation
Background on Quantum Hamiltonians and tensor network representations:
 What is Quantum Hamiltonian Complexity? (by Ben Edelman)
 Tensor Networks, Matrix Product States and Density Matrix Renormalization Group (by Fred Zhang)
Area laws:
 A 1D Area Law for Gapped Local Hamiltonians (by Boriana Gjura and Prayaag Venkat)
Quantum algorithms:
 Introduction to Quantum Walks (by Beatrice Nash)
 Efficient preparation of thermal states of quantum systems: natural or artificial (based on a guest lecture by Aram Harrow as scribed by Sinho Chewi, William S. Moses, Tasha Schoenstein, Ary Swaminathan)
 Quantum Approximate Optimization Algorithm and Applications (by
Amir Karamlou)
Quantum games and quantum PCPs:
 Towards Quantum PCP: A Proof of the NLETS Theorem (by Abhijit Mudigonda, Richard Wang, and Lisa Yang)
 Quantum Games (by Nilin Abrahamsen, Daniel Alabi, Mitali Bafna, Emil Khabiboulline, and Juspreet Sandhu)
Quantum supremacy:
 Quantum circuits and their role in demonstrating quantum supremacy (by Hikari Sorensen)
Quantum error correction:
 Peter Shor on Quantum Error Correction (based on a guest lecture by Peter Shor as scribed by Annie Wei)
Black Holes and the Firewall Paradox
(See also Boaz’s introductory post)
Background on black holes and the Firewall Paradox:
 Why physicists care about the Firewall Paradox (by Noah Miller)
 Black hole paradoxes: A conservative yet radical journey (by Abhishek Anand and Noah Miller)
Firewall Paradox meets Computational Complexity:
 Black Holes, a complexity theory perspective (by ChiNing Chou and Parth Mehta)
Nominate TCS papers for research highlights
[Guest post by Aleksander Mądry]
To me, one of the best things about working in theoretical computer
science has always the exciting rate of progress we make as a community.
On (what appears to be) a regular basis, we produce breakthroughs on
problems that are absolutely fundamental to our field. Problems that
often look impossible to tackle, right up until someone actually tackles
them.
However, as inspiring as all these developments were to me, I also
always felt that we, as a community, could do more to properly recognize
and highlight them, both internally and to the outside world. This kind
of outreach would make it easier for us to capitalize on the
breakthroughs as well as to accelerate the impact of the underlying
ideas on the other areas of computer science, and beyond.
Fortunately, this is about to change!
One of the first decisions of our newly (re)elected SIGACT committee was to create a committee (as committees are wont to do 🙂 whose mission will be to help promote top computer science theory research. This SIGACT Research Highlights Committee – consisting of Boaz Barak, Omer Reingold, Mary Wootters and myself – will, in particular, work to identify results to be recommended for consideration for the CACM Research Highlights section as well as other generalaudience research outlets in computer science and other fields.
Of course, to do a proper job here we require your help! To this end,
the committee solicits two types of nominations:
1) Conference nominations. Each year, the committee will ask the PC
chairs of a broad set of theoretical computer science conferences to
send a selection of up to three top papers from these conferences
(selected based on both their technical merit and the potential
significant interest to nontheory audiences) and forwarding them to the
committee for consideration.
2) Community nominations. The committee will accept nominations from the members of the community. Each such nomination should summarize the contribution of the nominated (and recently published) paper and also
argue why this paper particularly merits a broader outreach. The
nomination should be no more than a page in length and can be submitted
at any time by emailing it to sigact.cacm.nominations@gmail.com.
Selfnominations are discouraged.
To be considered in the upcoming round of our deliberations, we need to
receive your nomination by April 30.
Looking forward to learning about all the new exciting research that you
all are doing!
News addicts: Sign up for the CATCS newsletter
If, like others following the pace of modern life, you’re the kind of person that needs to get just on time updates on the state of theoretical computer science, consider signing up for the newsletter of CATCS. You can get information about funding opportunities, advocacy efforts, and more. Sure, at the hectic rate of two messages per year, it might flood your inbox, but it is worth it.
Dear Theoretical Computer Scientist,
The Committee for the Advancement of Theoretical Computer Science (CATCS) was established by SIGACT to deal with funding, outreach, and advocacy issues for our community. If you would like to receive our newsletter (no more than twice annually) we encourage you to sign up for the Google group below:
https://groups.google.com/forum/#!forum/catcsnews
(Navigate to the page above and click “Join group.” You can unsubscribe at any time from the same page.)
Best wishes, CATCS
Submit your failures to CFAIL 2019
[Posted at the request of Craig Gentry. I think this is actually a great idea that should be imitated by other subareas of TCS as well. –Boaz]
Update: The Conference for Failed Approaches and Insightful Losses in cryptology is celebrating our first failure early: the failure to keep a deadline! We are extending the submission deadline to 11:59 pm EST on April 7 to give everyone more time to work on submissions. In these last weeks leading up to the deadline, you can follow the @cfail2019 twitter account for inspiring jokes about failure to help you reach the finish line!
The CFAIL 2019 program chairs would like to remind you that some failures are good. Failure to attack a cryptosystem? Good! It might be a strong cryptosystem. Failure to publish in Eurocrypt? No problem! You’ll get ’em next time. Failure to cut back on caffeine like you promised in your New Year’s resolution? Good! You must be full of energy!
But failure to submit to CFAIL… lame!
Work on those submissions, cryptologists! We want to see your failures in all their glory!
Just a friendly reminder that the CFAIL submission deadline is April 1 April 7th.
Black Holes, a Complexity Theory perspective
Guest post by ChiNing Chou and Parth Mehta from the physics and computation seminar.
Abstract
The firewall paradox (introduced here) is a bewitching thought experiment that mandates a deeper understanding of our reality. As luck would have it, QFT predictions seem sound, GR calculations appear valid, and semiclassical approximations look reasonable: no one is willing to budge! To save Alice from burning in the miserable firewall, therefore, we must come up with a radically new proposal. This blog post aims to map what seems to be a hard, physics dilemma into a Computer Science problem that we can, using the grace of a lazy programmer, show to be hard to solve. In particular, we present an overview of the HarlowHayden decoding task and show how it maps the Firewall Paradox to a hard computation on a quantum computer. We end by rigorously defining quantum circuit complexity, Aaronson’s improved proof, AdS/CFT correspondence, and some fascinating homework (open) problems.
Why all the fuss?
Have you ever confessed to yourself that you don’t quite understand Black Hole complementarity well? In the past decade or so, physicists realized they did not grasp the concept thoroughly either. The firewall paradox is a natural result of bewildered physicists trying to make sense of reality. Thus far, no satisfying physical explanation reaches people’s consensus. Nevertheless, Daniel Harlow and Patrick Hayden [HH13] proposed a tempting solution to the firewall paradox using Computational Complexity (CC). Concretely, they showed the following.
We elaborate on this deep connection throughout this post.
Problem Solving: Physics v.s. Computer Science
The notion of a `conjecture’ has different implications for either field. In Physics, a wrong conjecture often delights physicists since there is more work left to do and better theory required to explain the physical phenomenon under study. For complexity theorists, however, if, say, the famous is proved to be false, a few consequences follow. First, the authors of the proof win a million dollars (See the Millennium problems.). Second, such a result would break almost all the foundations of computational complexity and cryptography. That is, refuting an (important) conjecture in computational complexity is tantamount to resulting in realworld catastrophes! Below in Table 1 is a short summary.
Theoretical Physics  Theoretical Computer Science  
Object  Are the mathematical models for our physical world correct?  Is our intuition about the mathematical models we defined correct? 
Consequences of disproving  After few days/months/years, physicists will come up with a new model and try to falsify it.  The belief system of complexity theorists collapses. Some super algorithms might show up and shake the world. 
How to prove/disprove  Checking mathematical consistency, doing both thought and empirical experiments.  Using fancy mathematics or designing super algorithms. 
Table 1: “Conjecture”, as used in Physics and Computer Science.
We labour above to convince the reader about these differences because the HarlowHayden decoding task has vital implications for both, Physics and Computer Science. The connections between Black Holes and Computational Complexity can be thought of as a new testbench for physical models.
Reckless review: Quantum Information
Gates
In Quantum Computation, gates are unitary operators. Some common gates used in the Quantum Information literature are as follows:

Singlequbit: Pauli matrices (i.e., ), phase operator , Hadamard matrix .

Twoqubit: CNOT, Toffoli, CZ.
For more details, please refer to [NC02]. Interestingly enough, singequbit and twoqubit gates are sufficient to construct any qubit gates! Such a set of operators is said to be universal. For example, and are universal for almost every singlequbit operator. Furthermore, Kitaev and Solovay gave a qualitative version of the universality theorem by showing that getting an approximation to an qubit operator in trace norm, only gates are needed. A final remark on unitary operators: an qubit operator is actually a matrix of size by . Namely, it requires complex numbers to describe an qubit operator. (Note the difference between and .)
Quantum circuits
A quantum circuit has inputs consisting of qubits, potentially with ancilla bits. The computation is done by interior gates from some universal gate set, e.g., . The outputs are qubits with potentially bits of garbage. See the following example of quantum circuit for the qubit Hadamard operator in Figure 1.
Similarly, the size of a quantum circuit is defined as the number of interior gates. In Figure 1 for example, the size of the circuit is .
Quantum circuit complexity: BQP/poly
Let be a boolean function. Define its quantum circuit complexity as the size of the smallest quantum circuit such that for any
Let denote the class of boolean functions of quantum circuit complexity at most . The complexity class is defined as . It immediately follows from definition that . As proving lower bound for (i.e., finding a problem that is not in ) is a longstanding extremely difficult problem, it is believed to be hard to prove lower bound against .
Uniform quantum circuit complexity: BQP
As is too powerful to work with, one might want to define a weaker version of the quantum complexity measure. A natural choice is considering a uniform computational model.
In the classical setting, a uniform computational model is defined using a Turing machine. However, it is not clear how to define the corresponding version, a quantum Turing machine. One way to do so is via uniform circuits, defined as follows. We say a circuit family is uniform if there exists a polynomial time Turing machine such that on input , it outputs .
Let be a boolean function. Define its uniform quantum circuit complexity as the size of the smallest uniform quantum circuit such that for any
Let denote the class of boolean functions of quantum circuit complexity at most . The complexity class is defined as . It immediately follows from definition that .
Unitary complexity: C(U)
Let be an unitary matrix. Define be the smallest quantum circuit such that
This unitary complexity can be thought of as a relaxation of the quantum circuit complexity. The reason is that here a unitary matrix might not compute a boolean function. Thus, proving a lower bound for implies a lower bound for unitary complexity while the converse is not clear. Namely, proving a superpolynomial lower bound for the unitary complexity might be an easier task.
However, no nontrivial^{1} lower bound for the unitary complexity is known and there is, unfortunately, no formal barrier result explaining why this is difficult to prove.
Warmup: GottesmanKnill
We defined quantum circuits above, and we hope you find them exotic – at least startup investors do. But given how fundamental quantum circuits are to the HarlowHayden decoding task, we ask: is it possible to efficiently (classically) simulate a quantum circuit made up of a restricted but nontrivial set of quantum gates? We show below a restricted variant of the popular GottesmanKnill Theorem:
Theorem (GottesmanKnill).
1. Given: Clifford circuit made up of gates , where is measured on its first output line.
2.Task: Show that it is possible to (classically) efficiently sample the output distribution of .
Proof:
where . Since the projector can be written as , we get
where since we only measure the first output line of . At first glance, might look like a monstrous computation to perform since, in general, the operator in the middle is a matrix, so the calculating the inner product would require exponential time classically. However, recognizing that Clifford gates are normalizers of the Pauli Group on qubits, note that where is some Pauli matrix. It is straightforward to show that these update rules can be computed efficiently. We thus have
which is a product of terms. We have thus reduced the (exponentially large) burden of computing a giant matrix
to computing matrices size , so we can sample the output distribution efficiently.
The Firewall paradox and the HarlowHayden decoding task
Physics to CS
All of the black hole physics covered in the previous blog post leads to the moment (we hope) you have been waiting for: a charming resolution of the firewall paradox. Consider the interior of an old, rusty black hole that has radiated away more than half of its matter. Let be the old Hawking radiation, and let represent the fresh Hawking radiation coming right out of the boundary of the Black Hole. Alice is our canonical Ph.D. student who is brave enough to risk her life for physics. Since is a giant information scrambler, we expect to find entanglement between and with overwhelming probability. We know from QFT that there are bell pairs straddling the event horizon of the black hole, so and should be maximally entangled. But this is a problem because cannot be entangled with both and ! The AMPS argument shows that if Alice is able to distill a bell pair between and , then we should see a firewall of photons at the event horizon, thus violating the nodrama postulate. See Figure 2 for more intuition about the set up. (Note that the ‘s represent Bell Pairs, as consistent with the 3DQuon Language) If we take Black Hole complimentary seriously, then we have an answer! If Alice does not distill a Bell pair between and , then nothing really happens. However, if Alice does manage to distill the entanglement between and , then we witness a firewall. Is not this answer so very unsatisfactory? Why should the existence of a firewall depend on Alice’s ability to distill entanglement? What is so special about this decoding task?
The HH decoding task answers precisely this question. Intuitively, it says that if Alice manages to distill a Bell pair between and , she could also invert a oneway function, a task we believe is very hard to perform! We conjecture that Alice would take exponential time to decode the entanglement, so the Black Hole would disappear long before Alice even makes a dent in the problem! Before we provide an in depth resolution of the paradox through the HH decoding, let us (as good philosophers do) briefly review assumptions:
 The Black Hole can be modelled by a finite collection of qubits, say qubits.
 Alice is told that the initial state of is the product basis .
 Black Hole dynamics are assumed to be unitary, so Alice need not worry about some spooky Mtheory that may claim to evolve in a nonunitary fashion.
 is a giant information scrambler, represented by some random circuit .
 Fresh radiation is a single qubit, w.l.o.g., since any additional qubits could be made a part of .
 is not HaarRandom. Miniexercise: prove that if is HaarRandom, our job becomes easy because the circuit complexity of the HH decoding task grows exponentially with , the size of .
 Alice has access only to circuit and but not . Trivial Miniexercise: prove that if Alice has access to , then it is easy to distill the entanglement between and .
 Alice may be an intellectual Goddess who just knows which unitary to apply, or, more realistically, someone who has exponential time to prepare before the Black Hole forms. Of crucial importance therefore is the circuit complexity of the unitary Alice applies to distill the Bell pair, not so much the process of finding the unitary.
Distilling the BR Bell pair
Let us jump into the definition of the HarlowHayden decoding task.
Definition (HarlowHayden decoding task).
Given a (polynomialsize) quantum circuit as input such that where are three disjoint part of the qubits. Furthermore, it is guaranteed that there exists a unitary operator acting only on the qubits in such that after applying , the rightmost bit of and the leftmost bit of forms a bell pair . The goal of the HarlowHayden decoding task is then to find a quantum circuit for such U on the qubits in . See Figure 3.
A necessary condition for the firewall paradox to make sense is that the HarlowHayden decoding task should be easy. If Alice cannot distill the entanglement efficiently, the black hole will evaporate before Alice is ready to witness the firewall!
To refute the firewall paradox, Harlow and Hayden proved the following theorem.
Theorem 1.
If the HarlowHayden decoding task can be done in , then .
We won’t formally define the complexity class . However, it is important to know that the foundation of the latticebased cryptography, a promising quantumsecure crypto framework, is based on the hardness of some problem in . If , then all latticebased cryptosystems can be broken by polynomial time quantum algorithm!
Instead of a proof for Theorem 1, which is more involved, we give a proof for an improvement of the HarlowHayden theorem due to Scott Aaronson. (Aaronson also showed that there might not even exist quantumsecure cryptography if the HarlowHayden decoding task can be efficiently solved!)
Aaronson’s improvement
In Aaronson’s lecture notes [Aar16], he showed the following improvement on Theorem 1.
Theorem 2.
If the HarlowHayden decoding task can be done in , then quantumsecure injective oneway function does not exist.
Before formally defining a one way function, it is paramount to understand its impact: modern cryptosystems are built from some variant of a oneway function. Intuitively, primitives that have the oneway property are (i) easy to implement (e.g., encrypt) but (ii) hard to invert (e.g., be attacked). As a result, if there is no quantumsecure injective oneway function, then that is strong evidence that quantumsecure cryptography might not exist.
Now, let us formally define what quantumsecure injective oneway function is and give a formal proof for Theorem 2.
Definition 1 (Quantumsecure injective oneway function).
A boolean function is a quantumsecure injective oneway function if
 is injective,
 , and
 for any polynomial time quantum algorithm
Note that since is injective, the last condition can actually be phrased as . Also, the condition should be read as “on input , the quantum algorithm outputs ”, namely, inverts .
Suppose the HarlowHayden decoding task is in , we are going to show that for any injective computable by some polynomial size quantum circuit, there is a polynomial time quantum algorithm that inverts . Namely, is not a quantumsecure injective oneway function.To get an efficient inverting algorithm for , let us first prepare a special circuit from and treat it as an input to the HarlowHayden decoding task. The circuit will simply map the to the following state
Note that as has a polynomial size quantum circuit, the circuit can also be implemented in polynomial size.Next, the easiness of the HarlowHayden decoding task guarantees us the existence of a unitary operation on the qubits in such that for any
for some state . By restricting on the first qubits, one can get unitary operators and such that for all ,
Thus, inverts because for any ,
Furthermore, as we are guaranteed that the HarlowHayden decoding task is in , as well as and all have polynomial size quantum circuits! Namely, can be efficiently inverted by a quantum algorithm and thus is not a quantumsecure injective oneway function.
What’s next?
The HarlowHayden decoding task as well as the Aaronson’s improvement can be interpreted as (strong) evidence that distilling the BR Bell pair is hard (in the worstcase^{2}). One might hope for an averagecase hardness for the HarlowHayden decoding task and thus infer that most black holes are difficult to distill. However, even if such averagecase hardness results existed, physicists would still remain dissatisfied! The foremost grievance a physicist may have is the lack of a coherent causal framework to model reality. That is, what happens if, in the
very small but nonzero chance, a black hole is easy to distill? Does that mean that a firewall exists in such black hole? How can a unifying theory explain such situation coherently? An ideal theory for theoretical physicists should work for every black hole instead of for most black holes! Second, physicists seem to dislike the abstract, processtheoretic approach undertaken by computer scientists. Here, we have completely ignored talking about the internal dynamics of a black hole or even a full description of its evolving Hilbert space. They would, for instance, like to see a differential equation that captures the difficulty of distilling a black hole throughout its evolution. Resolutions to the firewall paradox or effort towards building a theory of quantum gravity should be somewhat explicit in the sense that one can really instantiate some (toy) examples from the theory and see how the system evolves and examine whether this fits the real experience from the world. In other words, a theory with a black box (i.e., a complexity conjecture) might not be regarded as a resolution.
Homework
 What powers would Alice need to ensure that she can efficiently distill the BR bell pair. What if we assume ?
 Can we show that the decoding is hard on average, rather than for the worst case?
 What are some similar deep connections between black holes and complexity theory?
 For people interested in the quantum complexity theory, there are many open problems regarding the quantum circuit complexity: consider the unitary synthesis problem^{3} proposed by Scott Aaronson [Aar16].
 Another interesting problem is connecting the difficulty of proving quantum circuit lower bounds to other complexity problem such as classical circuit lower bounds or cryptographic assumptions.
Footnotes
^{1}Nontrivial here means the unitary matrix is explicit in the sense that given , one can efficiently compute .
^{2}Hard in worstcase means that there does not exist efficient algorithm that works on every input. Another hardness notion is hard on average, by which we mean there does not exist efficient algorithm the works for most of the input. Showing averagecase hardness is in general a more difficult task than proving worstcase hardness.
^{3}Does the following hold: for any unitary matrix , there exists a classical oracle such that where is the minimum size of quantum circuit that approximates with oracle access to .
References
Black hole paradoxes: A conservative yet radical journey
Guest post by Abhishek Anand and Noah Miller from the physics and computation seminar.
In 2013, Harlow and Hayden drew an unexpected connection between theoretical computer science and theoretical physics as they proposed a potential resolution to the famous black hole Firewall paradox using computational complexity arguments. This blog post attempts to lay out the Firewall paradox and other peculiar (at first) properties associated with black holes that make them such intriguing objects to study. This post is inspired by Scott Aaronson’s [1] and Daniel Harlow’s [2] excellent notes on the same topic. The notes accompanying this post provides a thorough and selfcontained introduction to theoretical physics from a CS perspective. Furthermore, for a quick and intuitive summary of the Firewall paradox and it’s link to computational complexity, refer to this blog post by Professor Barak last summer.
Black holes and conservative radicalism
Black holes are fascinating objects. Very briefly, they are regions of spacetime where the matterenergy density is so high and hence, where the gravitational effects are so strong that no particle (not even light!) can escape from it. More specifically, we define a particular distance called the “Schwarzschild radius” and anything that enters within the Schwarzschild radius, (also known as the “event horizon,”) cannot ever escape from the black hole. General relativity predicts that this particle is bound to hit the “singularity,” where spacetime curvature becomes infinite. In the truest sense of the word, they represent the “edge cases” of our Universe. Hence, perhaps, it is fitting that physicists believe that through thought experiments at these edges cases, they can investigate the true behavior of the laws that govern our Universe.
Once you know that such an object exists, many questions arise: what would it look it from the outside? Could we already be within the event horizon of a future black hole? How much information does it store? Would something special be happening at the Schwarzschild radius? How would the singularity manifest physically?
The journey of trying to answer these questions can aptly be described by the term “radical conservatism.” This is a phrase that has become quite popular in the physics community. A “radical conservative” would be someone that tries to modify as few laws of physics as possible (that’s the conservative part) and through their dogmatic refusal to modify these laws and go wherever their reasoning leads (that’s the radical part) is able to derive amazing things. We radically use the given system of beliefs to lead to certain conclusions (sometimes paradoxes!) and then conservatively update the system of beliefs to resolve the created paradox and iterate. We shall go through a few such cycles and end at the Firewall paradox. Let’s begin with the first problem: how much information does a black hole store?
Entropy of a black hole
A black hole is a physical object. Hence, it could be able to store some information. But how much? In other words, what should the entropy of a black hole be? There are two simple ways of looking at this problem:
 0: The nohair theorem postulates that an outside observer can measure a small number of quantities which completely characterize the black hole. There’s the mass of the black hole, which is its most important quantity. Interestingly, if the star was spinning before it collapsed, the black hole will also have some angular momentum, and its equator will bulge out a bit. Hence, the black hole is also characterized by an angular momentum vector. Also, if the object had some net charge, the black hole would also have that net charge. This means that if two black holes were created due to a book and a pizza, respectively, with the same mass, charge and angular momentum, there would settle down to the “same” black hole with no observable difference. If an outside observer knows these quantities, they will now know everything about the black hole. So, in this view, we should expect for the entropy of a black hole to be 0.
 Unbounded: But maybe that’s not entirely fair. After all, the contents of the star should somehow be contained in the singularity, hidden behind the horizon. As we saw above, all of the specific details of the star from before the collapse do not have any effect on the properties of the resulting black hole. The only stuff that matters it the total mass, total angular momentum, and the total charge. That leaves an infinite number of possible objects that could all have produced the same black hole: a pizza or a book or a PlayStation and so on. So actually, perhaps, we should expect the entropy of a black hole to be unbounded.
The first answer troubled Jacob Bekenstein. He was a firm believer in the Second Law of Thermodynamics: the total entropy of an isolated system can never decrease over time. However, if the entropy of a black hole is 0, it provides with a way to reduce the entropy of any system: just dump objects with nonzero entropy into the black hole.
Bekenstein drew connections between the area of the black hole and its entropy. For example, the way in which a black hole’s area could only increase (according to classical general relativity) seemed reminiscent of entropy. Moreover, when two black holes merge, the area of the final black hole will always exceed the sum of the areas of the two original black holes This is surprising as for two spheres, the area/radius of the merged sphere, is always less than the sum of the areas/radii of two individual spheres:
Most things we’re used to, like a box of gas, have an entropy that scales linearly with its volume. However, black holes are not like most things. He predicted that entropy of a black hole should be proportional to its area, A and not its volume. We now believe that Bekenstein was right and it turns out that the entropy of the black hole can be written as:
where is Boltzmann constant and is the Plancklength, a length scale where physicists believe quantum mechanics breaks down and a quantum theory of gravity will be required. Interestingly, it seems as though the entropy of the black hole is (onefourth times) the number of Plancklengthsized squares it would take to tile the horizon area. (Perhaps, the microstates of the black hole are “stored” on the horizon?) Using “natural units” where we set all constants to 1, we can write this as
which is very pretty. Even though this number of not infinite, it is very large. Here are some numerical estimates from [2]. The entropy of the universe (minus all the black holes) mostly comes from cosmic microwave background radiation and is about in some units. Meanwhile, in the same units, the entropy of a solar mass black hole is . The entropy of our sun, as it is now, is a much smaller . The entropy of the supermassive black hole in the center of our galaxy is , larger than the rest of the universe combined (minus black holes). The entropy of any of the largest known supermassive black holes would be . Hence, there is a simple “argument” which suggests that black holes are the most efficient information storage devices in the universe: if you wanted to store a lot of information in a region smaller than a black hole horizon, it would probably have to be so dense that it would just be a black hole anyway.
However, this resolution to “maintain” the second law of thermodynamics leads to a radical conclusion: if a black hole has nonzero entropy, it must have a nonzero temperature and hence, must emit thermal radiation. This troubled Hawking.
Hawking radiation and black hole evaporation
Hawking did a semiclassical computation looking at energy fluctuations near the horizon and actually found that black holes do radiate! They emit energy in the form of very lowenergy particles. This is a unique feature of what happens to black holes when you take quantum field theory into account and is very surprising. However, the Hawking radiation from any actually existing black hole is far too weak to have been detected experimentally.
One simplified way to understand the Hawking radiation is by thinking about highly coupled modes (think “particles”) being formed continuously near the horizon. As this formation must conserve the conservation of energy, one of these particles has negative energy and one of the particles has the same energy but with a positive sign and hence, they are maximally entangled (if you know the energy of one of the particles, you know the energy of the other one): we will be referring to this as shortrange entanglement. The one with negative energy falls into the black hole while the one with positive energy comes out as Hawking radiation. The maximallyentangled state of the modes looks like:
Here is a cartoon that represents the process:
Because energetic particles are leaving the black hole and negative energy particles are adding to it, the black hole itself will actually shrink, which would never happen classically! And, eventually a blackhole will disappear. In fact, the time of evaporation of the black hole scales polynomially in the radius of the black hole, as . The black holes that we know about are simply too big and would be shrinking too slowly. A stellarmass black hole would take years to disappear from Hawking radiation.
However, the fact that black holes disappear does not play nicely with another core belief in physics: reversibility.
Unitary evolution and thermal radiation
A core tenet of quantum mechanics is unitary evolution: every operation that happens to a quantum state must be reversible (invertible). That is: if we know the final state and the set and order of operations performed, we should be able to invert the operations and get back the initial state. No information is lost. However, something weird happens with an evaporating black hole. First, let us quickly review pure and mixed quantum states. A pure state is a quantum state that can be described by a single ket vector while a mixed state represents a classical (probabilistic) mixture of pure states and can be expressed using density matrices. For example, in both, the pure state and mixed state would one measure half the time and 50% half the time. However, in the later one would not observe any quantum effects (think interference patterns of the doubleslit experiment).
People outside of the black hole will not be able to measure the objects (quantum degrees of freedom) that are inside the black hole. They will only be able to perform measurements on a subset of the information: the one available outside of the event horizon. So, the state they would measure would be a mixed state. A simple example to explain what this means is that if the state of the particles near the horizon is:
tracing over the qubit A leaves us with the state and density matrix:
,
which is a classical mixed state (50% of times results in 1 and 50% of times results in 0). The nondiagonal entries of the density matrix encode the “quantum inference” of the quantum state. Here, are they are, in some sense we have lost the “quantum” aspect of the information.
In fact, Hawking went and traced over the field degrees of freedom that were hidden behind the event horizon, and found something surprising: the mixed state was thermal! It acted “as if” it is being emitted by some object with temperature “T” which does not depend on what formed the black hole and solely depends on the mass of the black hole. Now, we have the information paradox:
 Physics perspective: Now, once the black hole evaporates, we are left with this mixed thermal there is no way to precisely reconstruct the initial state that formed the black hole: the black hole has taken away information! Once the black hole is gone, the information of what went into the black hole is gone for good. Nobody living in the postblackhole universe could figure out exactly what went into the black hole, even if they had full knowledge of the radiation. Another way to derive a contradiction is that the process of black hole evaporation when combined with the disappearance of the black hole, imply that a pure state has evolved into a mixed state, something which is impossible via unitary time evolution! Pure states only become mixed states whenever we decide to perform a partial trace; they never become mixed because of Schrodinger’s equation which governs the evolution of quantum states.
 CS perspective: We live in a world where only invertible functions are allowed. However, we are given this exotic function – the black hole – which seems to be a genuine random onetomany function. There is no way to determine the input deterministically given the output of the function.
What gives? If the process of black hole evaporation is truly “nonunitary,” it would be a first for physics. We have no way to make sense of quantum mechanics without the assumption of unitary operations and reversibility; hence, it does not seem very conservative to get ride of it.
Physicists don’t know exactly how information is conserved, but they think that if they assume that it does, it will help them figure out something about quantum gravity. Most physicists believe that the process of black hole evaporation should indeed be unitary. The information of what went into the black hole is being released via the radiation in way too subtle for us to currently understand. What does this mean?
 Physics perspective: Somehow, after the black hole is gone, the final state we observe, after tracing over the degrees of freedom taken away by the black hole, is pure and encodes all information about what went inside the black hole. That is:
 CS perspective: Somehow, the exotic black hole function seems random but actually is pseudorandom as well as injective and given the output and enough time, we can decode it and determine the input (think hash functions!).
However, this causes yet another unwanted consequence: the violation of the nocloning theorem!
Xeroxing problem and black hole complementarity
The nocloning theorem simply states that an arbitrary quantum state cannot be copied. In other words, if you have one qubit representing some initial state, no matter what operations you do, you cannot end up with two qubits with the same state you started with. How do our assumptions violate this?
Say you are outside the black hole and send in a qubit with some information (input to the function). You collect the radiation corresponding to the qubit (output of the function) that came out. Now you decode this radiation (output) to determine the state of infalling matter (input). Aha! You have violated the nocloning theorem as you have two copies of the same state: one inside and one outside the black hole.
So wait, again, what gives?
One possible resolution is to postulate that the inside of the black hole just does not exist. However, that doesn’t seem very conservative. According to Einstein’s theory of relativity, locally speaking, there is nothing particularly special about the horizon: hence, one should be able to cross the horizon and move towards the singularity peacefully.
The crucial observation is that for the person who jumped into the black hole, the outside universe may as well not exist; they can not escape. Extending this further, perhaps, somebody on the outside does not believe the interior of the black hole exists and somebody on the inside does not believe the exterior exists and they are both right. This hypothesis, formulated in the early 1990s, has been given the name of Black Hole Complementarity. The word “complementarity” comes from the fact that two observers give different yet complementary views of the world.
In this view, according to someone on the outside, instead of entering the black hole at some finite time, the infalling observer will instead be stopped at some region very close to the horizon, which is quite hot when you get up close. Then, the Hawking radiation coming off of the horizon will hit the observer on its way out, carrying the information about them which has been plastered on the horizon. So the outside observer, who is free to collect this radiation, should be able to reconstruct all the information about the person who went in. Of course, that person will have burned up near the horizon and will be dead.
And from the infalling observer’s perspective, however, they were able to pass peacefully through the black hole and sail on to the singularity. So from their perspective, they live, while from the outside it looks like they died. However, no contradiction can be reached, because nobody has access to both realities.
But why is that? Couldn’t the outside observer see the infalling observer die and then rocket themselves straight into the black hole themselves to meet the alive person once again before they hit the singularity, thus producing a contradiction?
The core idea is that it must take some time for the infalling observer to “thermalize” (equilibriate) on the horizon: enough time for the infalling observer to reach the singularity and hence become completely inaccessible. Calculations do show this to be true. In fact, we can already sense a taste of complexity theory even in this argument: we are assuming that some process is slower than some other process.
In summary, according to the BHC worldview, the information outside the horizon is redundant with the information inside the horizon.
But, in 2012, a new paradox, the Firewall paradox, was introduced by AMPS [3]. This paradox seems to be immune to BHC: the paradox exists even if we assume everything we have discussed till now. The physics principle we violate, in this case, is the monogamy of entanglement.
Monogamy of entanglement and Page time
Before we state the Firewall paradox, we must introduce two key concepts.
Monogamy of entanglement
Monogamy of entanglement is a statement about the maximum entanglement a particle can share with other particles. More precisely, if two particles A and B are maximally entangled with each other, they cannot be at all entanglement with a third particle C. Two maximally entangled particles have saturated both of their “entanglement quotas\”. In order for them to have correlations with other particles, they must decrease their entanglement with each other.
Monogamy of entanglement can be understood as a static version of the nocloning theorem. Here is a short proof sketch of why polygamy of entanglement implies the violation of nocloning theorem.
Let’s take a short detour to explain quantum teleportation:
Say you have three particles A, B, and C with A and B maximally entangled (Bell pair), and C is an arbitrary quantum state:
We can write their total state as:
Rearranging and pairing A and C, the state simplifies to:
which means that if one does a Bell pair measurement on A and C, based on the measurement outcome, we know exactly which state B is projected to and by using rotations can make the state of B equal to the initial state of C. Hence, we teleported quantum information from C to B.
Now, assume that A was maximally entangled to both B and D. Then by doing the same procedure, we could teleport quantum information from C to both B and D and hence, violate the nocloning theorem!
Page time
Named after Don Page, the “Page time” refers to the time when the black hole has emitted enough of its energy in the form of Hawking radiation that its entropy has (approximately) halved. Now the question is, what’s so special about the Page time?
First note that the rank of the density matrix is closely related to its purity (or mixedness). For example, a completely mixed state is the diagonal matrix:
which has maximal rank (). Furthermore, a completely pure state can always be represented as (if we just change the basis and make the first column/row represent ):
which has rank 1.
Imagine we have watched a black hole form and begin emitting Hawking radiation. Say we start collecting this radiation. The density matrix of the radiation will have the form:
where is the total number of qubits in our initial state, is the number of qubits outside (in form of radiation), and is the probability of each state. We are simply tracing over the degrees of freedom inside the black hole (as there are degrees inside the black hole, dimensionality of this space is ).
Don Page proposed the following graph of what he thought entanglement entropy of this density matrix should look like. It is fittingly called the “Page curve.”
 If , rank() = , as there are enough terms in the sum to get a maximally ranked matrix. And hence, we get maximally mixed states. In the beginning, the radiation we collect at early times will still remain heavily entangled with the degrees of freedom near the black hole, and as such the state will look mixed to us because we can not yet observe all the complicated entanglement. As more and more information leaves the black hole in the form of Hawking radiation, we are “tracing out” fewer and fewer of the nearhorizon degrees of freedom. The dimension of our density matrix grows bigger and bigger, and because the outgoing radiation is still so entangled with the nearhorizon degrees of freedom, the density matrix will still have offdiagonal terms which are essentially zero. Hence, the state entropy increases linearly.
 But if , by the same argument, rank() = . Hence, the density matrix becomes more and more pure. Once the black hole’s entropy has reduced by half, the dimension of the Hilbert space we are tracing out finally becomes smaller than the dimension of the Hilbert space we are not tracing out. The offdiagonal terms spring into our density matrix, growing in size and number as the black hole continues to shrink. Finally, once the black hole is gone, we can easily see that all the resulting radiation is in a pure state.
The entanglement entropy of the outgoing radiation finally starts decreasing, as we are finally able to start seeing entanglements between all this seemingly random radiation we have painstakingly collected. Some people like to say that if one could calculate the Page curve from first principles, the information paradox would be solved. Now we are ready to state the firewall paradox.
The Firewall Paradox
Say Alice collects all the Hawking radiation coming out of a black hole. At maybe, about times the Page time, Alice is now able to see significant entanglement in all the radiation she has collected. Alice then dives into the black hole and sees an outgoing Hawking mode escaping. Given the Page curve, we know that knowing this outgoing mode must decrease the entropy of our observed mixed state. In other words, it must make our observed density matrix purer. And hence, be entangled with the particles we have already collected.
(Another way to think about this: let’s say that a random quantum circuit at the horizon scrambles the information in a nontrivial yet injective way in order for radiation particles to encode the information regarding what went inside the black hole. The output qubits of the circuit must be highly entangled due to the random circuit.)
However, given our discussion on Hawking radiation about shortrange entanglement, the outgoing mode must be maximally entangled with an infalling partner mode. This contradicts monogamy of entanglement! The outgoing mode cannot be entangled both with the radiation Alice has already collected and also maximally entangled with the nearby infalling mode!
So, to summarize, what did we do? We started with the existence of black holes and through our game of conservative radicalism, modified how physics works around them in order to make sure the following dear Physics principles are not violated by these special objects:
 Second Law of Thermodynamics
 Objects with entropy emit thermal radiation
 Unitary evolution and reversibility
 Nocloning theorem
 Monogamy of entanglement
And finally, ended with the Firewall paradox.
So, for the last time in this blog post, what gives?
 Firewall solution: The first solution to the paradox is the existence of a firewall at the horizon. The only way to not have the shortrange entanglement discussed is if there is very high energy density at the horizon. However, this violates the “nodrama” theorem and Einstein’s equivalence principle of general relativity which states that locally there should be nothing special about the horizon. If firewalls did exist, an actual wall of fire could randomly appear out of nowhere in front of us right now if a future black hole would have its horizon near us. Hence, this solution is not very popular.
 Remnant solution: One possible resolution would be that the black hole never “poofs” but some quantum gravity effect we do not yet understand stabilizes it instead, allowing for some Plancksized object to stick around? Such an object would be called a “remnant.” The socalled “remnant solution” to the information paradox is not a very popular one. People don’t like the idea of a very tiny, lowmass object holding an absurdly large amount of information.
 No unitary evolution: Perhaps, black holes are special objects which actually lose information! This would mean that black hole physics (the quantum theory of gravity) would be considerably different compared to quantum field theory.
 Computational complexity solution?: Can anyone ever observe this violation? And if not, does that resolve the paradox? This will be covered in our next blog post by Parth and ChiNing.
References
 Scott Aaronson. The complexity of quantum states and transformations: from quantum money to black holes.arXiv preprintarXiv:1607.05256, 2016.
 Daniel Harlow. Jerusalem lectures on black holes and quantum information. Reviews of Modern Physics, 88(1):015002, 2016.
 Ahmed Almheiri, Donald Marolf, Joseph Polchinski, and JamesSully. Black holes: complementarity or firewalls? Journal of HighEnergy Physics, 2013(2):62, 2013.
Introduction to AMP and the Replica Trick
(This post from the lecture by Yueqi Sheng)
In this post, we will talk about detecting phase transitions using
ApproximateMessagePassing (AMP), which is an extension of
BeliefPropagation to “dense” models. We will also discuss the Replica
Symmetric trick, which is a heuristic method of analyzing phase
transitions. We focus on the Rademacher spiked Wigner model (defined
below), and show how both these methods yield the same phrase transition
in this setting.
The Rademacher spiked Wigner model (RSW) is the following. We are given
observations where
(sampled uniformly) is the true signal and is a
GaussianOrthogonalEnsemble (GOE) matrix:
for and
. Here is the signal to noise
ratio. The goal is to approximately recover .
The question here is: how small can be such that it is
impossible to recover anything reasonably correlated with the
groundtruth ? And what do the approximatemessagepassing algorithm
(or the replica method) have to say about this?
To answer the first question, one can think of the task here is to
distinguish vs
. One approach to distinguishing these distributions is to
look at the spectrum of the observation matrix . (In fact, it turns
out that this is an asymptotically optimal distinguisher [1]). The spectrum of behaves as ([2]):
 When , the empirical distribution of eigenvalues in
spiked model still follows the semicircle law, with the top
eigenvalues 
When , we start to see an eigenvalue in the
planted model.
Approximate message passing
This section approximately follows the exposition in [3].
First, note that in the Rademacher spiked Wigner model, the posterior
distribution of the signal conditioned on the observation
is: This
defines a graphicalmodel (or “factorgraph”), over which we can perform
BeliefPropogation to infer the posterior distribution of .
However, in this case the factorgraph is dense (the distribution is a
product of potentials for all
pairs of ).
In the previous blog post, we saw belief propagation works great when the underlying interaction
graph is sparse. Intuitively, this is because is locally tree like,
which allows us to assume each messages are independent random
variables. In dense model, this no longer holds. One can think of dense
model as each node receive a weak signal from all its neighbors.
In the dense model setting, a class of algorithms called Approximate
message passing (AMP) is proposed as an alternative of BP. We will
define AMP for RWM in terms of its state evolution.
State evolution of AMP for Rademacher spiked Wigner model
Recall that in BP, we wish to infer the posterior distributon of
, and the messages we pass between nodes correspond to marginal
probability distribution over values on nodes. In our setting, since the
distributions are over , we can represent distributions by
their expected values. Let denote the
message from to at time . That is, corresponds
to the expected value .
To derive the BP update rules, we want to compute the expectation
of a node , given the
messages for . We can
do this using the posterior distribution of the RWM, ,
which we computed above.
And similarly for .
From the above, we can take expectations over , and express
in terms of
. Doing this (and
using the heuristic assumption that the distribution of is a
product distribution), we find that the BP state update can be written
as:
where the interaction matrix , and
.
Now, Taylor expanding around , we find
since the terms are of order .
At this point, we could try dropping the “nonbacktracking” condition
from the above sum (since the node contributes at most
to the sum anyway), to get the state update:
(note the messages no longer
depend on receiver – so we write in place of ).
However, this simplification turns out not to work for estimating the
signal. The problem is that the “backtracking” terms which we added
amplify over two iterations.
In AMP, we simply perform the above procedure, except we add a
correction term to account for the backtracking issue above. Given ,
for all , the AMP update is:
The correction term corresponds to error introduced by the backtracking
terms. Suppose everything is good until step . We will examine
the influence of backtracking term to a node through length 2 loops.
At time , exert additional influence to
each of it’s neighbor . At time , receive roughly
. Since has magnitude
and we need to sum over all of ’s neighbors,
this error term is to large to ignore. To characterize the exact form of
correction, we simply do a taylor expansion
State evolution of AMP
In this section we attempt to obtain the phase transition of Rademacher
spiked Wigner model via looking at .
We assume that each message could be written as a sum of signal term and
noise term. where
. To the dynamics of AMP (and find its phase
transition), we need to look at how the signal and noise
evolves with .
We do the following simplification: ignore the correction term and
assume each time we obtain an independent noise .
Here, we see that
and .
Note that is essentially proportional to overlap between
ground truth and current belief, since the function keeps the
magnitude of the current beliefs bounded.
For the noise term, each coordinate of is a gaussian random
variable with mean and variance
It was shown in [4] that we can introduce a new
parameter s.t.
As , turns out and
. To study the behavior of
as , it is enough to track the evolution of
.
This heuristic analysis of AMP actually gives a phase transition at
(in fact, the analysis of AMP can be done rigorously as in [5]):
 For : If , w.h.p., thus we have . Taking , we have , which means there AMP solution has no overlap with the ground truth.

For : In this case, AMP’s solution has some correlation with the ground truth.
(Figure from [6])
Replica symmetry trick
Another way of obtaining the phase transition is via a nonrigorous
analytic method called the replica method. Although nonrigorous, this
method from statistical physics has been used to predict the fixed point
of many message passing algorithms and has the advantage of being easy
to simulate. In our case, we will see that we obtain the same phase
transition temperature as AMP above. The method is nonrigorous due to
several assumptions made during the computation.
Outline of replica method
Recall that we are interested in minizing the free energy of a given
system where is
the partition function as before:
and
.
In replica method, is not fixed but a random variable. The
assumption is that as , free energy doesn’t vary with
too much, so we will look at the mean of to approximate free
energy of the system.
is called the free energy density and the goal now is to
compute the free energy density as a function of only , the
temperature of the system.
The replica method is first proposed as a simplification of the
computation of
It is a generally hard problem to compute in a clear way. A
naive attempt of approximate is to simply pull the log out
Unfortunately and are quite different quantities,
at least when temperature is low. Intuitively, is looking at
system with a fixed while in , and are allowed to
fluctuate together. When the temperature is high, doesn’t play a big
roll in system thus they could be close. However, when temperature is
low, there could be a problems. Let ,
,
.
While is hard to compute,
is a much easier quantity. The
replica trick starts from rewriting with moments of :
Recall that for and
, using this we can rewrite in the following
way:
Claim 1. Let
Then,
The idea of replica method is quite simple
 Define a function for s.t. for all such .

Extend analytically to all and take the limit of .
The second step may sound crazy, but for some unexplained reason, it has
been surprisingly effective at making correct predictions.
The term replica comes from the way used to compute
in Claim 1. We expand the th moment
in terms of replicas of the system
For Rademacher spiked Wigner model
In this section, we will see how one can apply the replica trick to
obtain phase transition in the Rademacher spiked Wigner model. Recall
that given a hidden , the observable
where
and .
We are interested in finding the smallest where we can still
recover a solution with some correlation to the ground truth . Note
that is not so important here as doesn’t carry
any information in this case.
Given by the posterior , the system we
set up corresponding to Rademacher spiked Wigner model is the following:
 the system consists of particles and the interactions between
each particle are give by 
the signal to noise ratio as the inverse temperature
.
Following the steps above, we begin by computing
for : Denote where is the
th replica of the system.
We then simplify the above expression with a technical claim.
Claim 2. Let where is a fixed matrix and
is the GOE matrix defined as above. Then,
for some constant depending on distribution of .
Denote . Apply Claim 2 with
, we have
To understand the term inside exponent better, we can rewrite the inner
sum in terms of overlap between replicas:
where the last equality follows from rearranging and switch the inner
and outer summations.
Using a similar trick, we can view the other term as
Note that represents overlaps between the
and th replicas and represents the
overlaps between the th replica and the ground truth vector.
In the end, we get for any integer , (Equation 1):
Our goal becomes to approximate this quantity. Intuitively, if we think
of as indices on a matrices, ,
with , then is the average of i.i.d matrices. So we
expect for w.h.p. In the
remaining part, We find the correct via rewriting Equation 1.
Observe that by introducing a new variable for and
using the property of gaussian intergal (Equation 4):
Replace each by a such integral, we
have (Equation 2):
where is the constant given by introducing gaussian intergals.
To compute the integral in (Equation 2), we need to cheat a little bit and take
before letting . Note that free energy density
is defined as
This is the second assumption made in the replica method and it is
commonly believed that switching the order is okay here. Physically,
this is plausible because we believe intrinsic physical quantities
should not depend on the system size.
Now the Laplace method tells us when , the integral in (Equation 2) is dominated by the max of the exponent.
Theorem 1 (Laplace Method). Let , then
where and is the Hessian of evaluated at the point .
Fix a pair of and apply Laplace method with
what’s left to do is to find the critical point of . Taking the
derivatives gives
where
.
We now need to find a saddle point of where the hessian is PSD. To
do that, we choose to assume the order of the replicas does not matter,
which is refer to as the replica symmetry case. ^{1} One simplest form
of is the following: , and
for some . This also implies that for some
and
Plug this back in to Equation 2 gives: (Equation 3)
To obtain , we only need to deal with the last term in
(Equation 3) as . Using the fact that for all
and using the same trick of introducing new gaussain integral as
in (Equation 4) we have
Using the fact that we want the solution to minimizes free energy,
taking the derivative of the current w.r.t. gives
which matches the fixed point of AMP. Plug in and will give us
. The curve of looks like the Figure below, where
the solid line is the curve of with the given and the
dotted line is the curve given by setting all variables .
References

Turns out for this problem, replica symmetry is the only case. We
will not talk about replica symmetry breaking here, which
intuitively means we partition replicas into groups and recurse. ↩