Windows On Theory

Scott Aaronson blogged in defense of “armchair epidemiology”. Scott makes several points I agree with, but he also advocates that rather than discounting ideas from “contrarians” who have no special expertise in the matter, each one of us should evaluate the input of such people on its merits.

I disagree. I can judge on their merits the validity of a proposed P vs NP proof or a quantum algorithm for SAT, but I have seen time and again smart and educated non-experts misjudge such proposals. As much as I’d like to think otherwise, I would probably be fooled just as easily by a well-presented proposal in area X that “brushes under the rug” subtleties that experts in X would immediately notice.

This is not to say that non experts should stay completely out of the matter. Just like scientific journalists such as Erica Klarreich and Kevin Hartnett of quanta can do a great job of explaining computer science topics to lay audience, so can other well-read people serve as “signal boosters” and highlight works of experts in epidemiology. Journalist Helen Branswell of stat news has been following the novel coronavirus since January 4th.

The difference is that these journalists don’t pretend to see what the experts are missing but rather to highlight and simplify the works that experts are already doing. This is unlike “contrarians” such as Robin Hanson that do their own analysis on a spreadsheet and come up with a “home brewed” policy proposal such as deliberate infection or variolation (with “hero hotels” in which people go to be deliberately infected). I am not saying that such proposals are necessarily wrong, but I am saying that I (or anyone else without the experience in this field) am not qualified to judge them. Even if they did “make sense” to me (they don’t) I would not feel any more confident in judging them than I would in reviewing a paper in astronomy. There is a reason why Wikipedia has a “no original research” policy.

Moreover, the attitude of dismissing expertise can be dangerous, whether it comes in the form of “teach the debate” in the context of evolution, or “ClimateGate” in the context of climate change. Unlike the narrative of few brave “dissenters” or “contrarians”, in the case of COVID-19, experts as well as the world health organization have been literally sounding the alarm (see also timeline, as well as this NPR story on the US response). Yes, some institutions, and especially the U.S., failed in several aspects (most importantly in the early production of testing). But one of the most troubling aspects is the constant sense of “daylight” and distrust between the current U.S. administration and its own medical experts. Moreover, the opinions of people such as law professor Richard Epstein are listened to even when they are far out of their depth. It is one thing to entertain the opinion of non-expert contrarians when we have all the time in the world to debate, discuss and debunk. It’s quite another to do so in the context of a fast-moving health emergency. COVID-19 is an emergency that has medical, social, economical, and technological aspects, but it would best be addressed if each person contributes according to their skill set and collaborates with people of complementary backgrounds.

The new coronavirus upended much of society, including our little corner of it. I believe at this point almost all theorists are teaching and doing research at home, and I thought it would be good to share some of the tools we use for doing so. Below I will describe my setup, but I hope other people share theirs too.

Teaching and virtual whiteboards

I am teaching using Zoom and using an iPad pro with a pencil to simulate a whiteboard. I use a laptop to connect to Zoom and for the camera, laptop, and chat window, and then join the iPad.There are several ways to connect an iPad to a Zoom session:

1. Join the session from the iPad separately using the iPad Zoom app. (To do so you might need to logout of your other account.)
2. Within the Zoom program on your computer you can choose “share screen” and then one of the option is to join an iPad connected to the same wireless network as the laptop/desktop and use “screen mirroring” on the iPad. (You can find the screen mirroring option on the iPad by swiping from the top right corner in a diagonal motion.)
3. Another variant of this is to use a third party app such as Reflector 3. Reflector 3 sets up an airplay server on your PC so you can mirror the iPad screen to it. You can then share the Reflector 3 window from Zoom (see screen shorts below). If you do use Reflector 3, you can remove its annoying iPad like frame.
4. You can use a wired connection, which is either by just connecting through USB (in a Mac) or a complex combination of combining an adapter to take an HDMI signal out of an iPad with an HDMI capture card to stream this signal to the computer.

I use either option 2 or 3. (Might have used 4 if I had a Mac.) The main reason I prefer these to option 1 is because the application I use for a whiteboard – GoodNotes – has a presentation mode that behaves differently when you are connected to an external display or use AirPlay (which is what options 2 and 3 do). In this presentation mode the students don’t see your interface, and so you can Zoom, go to the page selector and more without it disturbing what they see. GoodNotes also has a great “laser pointer”. I set the pages at a landscape orientation, and pre-write a good chunk of what I plan to present before the lecture. I also use the ability to “duplicate pages” to achieve the PowerPoint like effect of gradual reveal.

It is not perfect – I’ve discovered that the screen share sometimes stops refreshing and I need to leave GoodNotes and return to it for it to go back (this seems to works better in Reflector 3 so far for me).

Monitoring the chat window and raised hands in Zoom is non-trivial. It helps a lot that I have a teaching assistant that participates in lecture and notices if I missed something.

Some people say that a “matte screen protector” such as PaperLike makes the iPad more comfortable to write on – haven’t yet tried it. (Update 4/1/2020: I now tried PaperLike and can vouch for it – it greatly improves my handwriting! shipping from the UK did take some time though.)

I have a good Webcam (Logitech Brio) but at the moment I’m not using it since it seems too taxing on my laptop and so I went back to the native webcam. I have a very nice wireless headset/mic combo (Jabra Evolve 75) that I am constantly using and have been very happy with. I particularly like the feature of being able to unmute and mute yourself by raising and lowering the mike.

Research

For research Slack continues to extremely useful. For working jointly on a paper Overleaf is of course great, but for maintaining a shared document it sometimes useful to use simpler platform that are not full fledged LaTeX. Some options include:

• Hackmd.io for markdown (supports LaTeX math)
• Google docs of course. I heard about the Auto-LaTeX plugin but haven’t used it yet.
• Dropbox Paper supports LaTeX natively.

Google JamBoard is an interactive whiteboard, also with an iPad app. I haven’t tried it yet but it seems promising.

Keeping children busy

For many people I imagine childcare is one of the most challenging aspects. At the moment at least the Cambridge Public Schools are not keeping my kids too busy. While probably most of their time is spent in non educational pursuits, we try to also encourage (i.e., bribe/extort/threaten/beg) them to do some learning. If your kids are interested in math, I highly recommend the courses offered by the Art of Problem Solving (they also have a theory connection: one of their books was co-authored by theorist Ravi Boppana). For younger kids you can also try their Beast Academy.

The AOPS program is not free but over the years my kids (especially my 13 year old daughter Alma) have also spent a lot of time on the free and amazing Khan Academy. In fact, last year she was inspired enough by Khan’s JavaScript course to write the following poem which (for obvious reasons) moved me very much:

Tending / Alma Barak

My mind is a desert
of boredom
of blankness
of leaning-back-in-your-chairness
and of simply-staring-at-the-ceilingness
I kick at a crumpled
deserted baby-blonde post-it
with my spotted socked feet
waiting for Aba to come.

We dive when he comes
into the seas of Khan
free-styling our way
into the lakes of Java
we pass codes we already
knew
while loops
for loops
arrays
reminding me of what I’d learned

We frog-kick to an
uncharted
unfamiliar
lagoon of code I don’t know
sometimes I get swept away by currents
of confusion
but my aba, my dad
grabs my hand
and shows me the way through
teaching me
tending to me
washing away the sands of boredom with the sea of Khan.

Shcachar Lovett has put together a new website aggregating information about virtual talks in CS theory: https://cstheorytalks.wordpress.com/

 It has a Google calendar that people can add to their own, and a form to submit a new talk that automatically gets added to the Google calendar. 

This can be a fantastic resource these days that almost no one can travel – please publicize this and also submit to it talks that you are organizing.

From Sandy Irani, FOCS 2020 PC chair:

In light of the very unusual developments in the world due to the spread of Covid-19 and the disruption it is causing to many people in our field, especially those with young children at home, the FOCS PC has decided to push back the final deadline for papers by six days. We would have liked to do more, but we still are trying to stick to our timeline for notification since that date is coordinated with other deadlines in the theory community. In addition, some members of the committee are also affected by this crisis and there is concern that we may not be able to do our job as a committee in a shorter time frame. Pre-registration (including titles and abstracts) is still required by the original deadline. Here are the new dates:

Pre-registration (including titles and abstracts): Thursday April 9, 11:59 PM (EDT)

Final Paper Submission: Wednesday April 15, 11:59 PM (EDT)

Conference url: https://focs2020.cs.duke.edu/

We hope you all stay healthy!

–Sandy Irani, for the FOCS 2020 Committee

Harvard University, as well most other places that I know of will be moving to remote lectures. I just gave the last in-person lecture in my cryptography course. I would appreciate technical suggestions on the best format for teaching remotely. At the moment I plan to use Zoom and log-in from both my laptop (for the video) and from my iPad pro (for the interactive whiteboard).

The one silver lining is that more lectures will be available online. In particular, if you’re teaching algorithms, you might find Eric Vigoda’s videos helpful. (If you know of more sources, please let us know.)

I was hoping that reducing meetings and activities will be good for research, but currently find it hard to concentrate on anything except this train-wreck of a situation. The financial times has a chart that summarizes the progress of the disease in several countries:

The number of confirmed cases grows by about 33% each day. This growth in confirmed cases is partially due to increased testing as cases increase – there is some evidence that the doubling time of the disease (time between infected people to infected) is about 6 days (rather than the 2.4 days that this figure suggest). However, a doubling time of 6 days still means that the number of cases grows 10-fold in a month, and so if there are 10K actual cases in the U.S., today, there would be 100K by mid April and 1M by mid May.

Strong quarantine regimes, contact tracing, and drastically reducing activity and increasing “social distance” can very significantly reduce the base of this exponent. Reducing the base of the exponent is more than simply “delaying the inevitable”. The mortality statistics mask the fact that this can be a very serious illness even for the people who don’t die of it – about 5% of the cases need intensive care (see this Economist article). Spreading the infections over time will enable the healthcare system to handle the increased caseload, which will completely overwhelm it otherwise.

Such steps are clearly much easier to achieve before the number of cases is too large to be manageable, but despite having “advance warning” from other countries, this lesson does not seem at the moment to have sunk in, at least here in the U.S. At the moment no such initiatives are taken at the federal level, the states are doing more but still not enough, and it’s up to private companies and institutions to come up with their own policies. As faculty and citizens there is not much we can do about it except support such decisions even when they are unpopular, and just try to make the remote experience as good as possible for us, our colleagues, and our students.

Gerard Ben Arous, Surya Ganguli, Florent Krzakala and Lenka Zdeborova are organizing a summer school on statistical physics of machine learning on August 2-28, 2020 in Les Houches, France. If you don’t know Les Houches, it apparently looks like this:

They are looking for applications from students, postdocs, and young researchers in physics & math, as well computer scientists. While I am biased (I will be lecturing there too) I think the combination of lecturers, speakers, and audience members will yield a very unique opportunity for interaction across communities, and strongly encourage theoretical computer scientists to apply (which you can from the website). Let me also use this opportunity to remind people again of Tselil Schramm’s blog post where she collected some of the lecture notes from the seminar we ran on physics & computation.

More information about the summer school:

The “Les Houches school of physics”, situated close to Chamonix and the Mont Blanc in the French Alps, has a long history of forming generations of young researchers on the frontiers of their fields. Our school is aimed primarily at the growing audience of theoretical physicists and applied mathematicians interested in machine learning and high-dimensional data analysis, as well as to colleagues from other fields interested in this interface. [my emphasis –Boaz] We will cover basics and frontiers of high-dimensional statistics, machine learning, the theory of computing and learning, and probability theory. We will focus in particular on methods of statistical physics and their results in the context of current questions and theories related to machine learning and neural networks. The school will also cover examples of applications of machine learning methods in physics research, as well as other emerging applications of wide interest. Open questions and directions will be presented as well.

Students, postdocs and young researchers interested to participate in the event are invited to apply on the website http://leshouches2020.krzakala.org/ before March 15, 2020. The capacity of the school is limited, and due to this constraint participants will be selected from the applicants and participants will be required to attend the whole event.

Lecturers:

• Boaz Barak (Harvard): Computational hardness perspectives
• Giulio Biroli (ENS, Paris): High-dimensional dynamics
• Michael Jordan (UC Berkeley): Optimization, diffusion & economics
• Marc Mézard (ENS, Paris): Message-Passing algorithms
• Yann LeCun (Facebook AI, NYU). Challenges and directions in machine learning
• Remi Monasson (ENS, Paris): Statistical physics or learning in neural networks
• Andrea Montanari (Stanford): High-dimensional statistics & neural networks
• Maria Schuld (Univ. KwaZulu Natal & Xanadu): Quantum machine learning
• Haim Sompolinsky (Harvard & Hebrew Univ.): Statistical mechanics of deep neural networks
• Nathan Srebro (TTI-Chicago): Optimization and implicit regularisation
• Miles Stoudenmire (Flatiron, NYC): Tensor network methods
• Pierre Vandergheynst (EPFL, Lausanne): Graph signal processing & neural networks

Invited Speakers (to be completed):

• Christian Borgs (UC Berkeley)
• Jennifer Chayes (UC Berkeley)
• Shirley Ho (Flatiron NYC)
• Levent Sagun (Facebook AI)

This semester I taught another iteration of my “Introduction to Theoretical Computer Science” course, based on my textbook in process. The book was also used in University of Virgnia CS 3102 by David Evans and Nathan Brunelle.

The main differences I made in the text and course since its original version were to make it less “idiosyncratic”: while I still think using programming language terminology is the conceptually “right” way to teach this material, there is a lot to be said for sticking with well-established models. So, I used Boolean circuits as the standard model for finite-input non-uniform computation, and Turing Machines, as the standard model for unbounded-input uniform computation. (I do talk about the equivalent programming languages view of both models, which can be a more useful perspective for some results, and is also easier to work with in code.)

In any course on intro to theoretical CS, there are always beautiful topics that are left on the “cutting room floor”. To partially compensate for that, we had an entirely optional “advanced section” where guest speakers talked about topics such as error correcting codes, circuit lower bounds, communication complexity, interactive proofs, and more. The TA in charge of this section – amazing sophomore named Noah Singer – wrote very detailed lecture notes for this section.

This semester, students in CS 121 could also do an optional project. Many chose to do a video about topics related to the course, here are some examples:

• Helen Huang and Phil Labrum: “Is your brain Turing complete?”
• Dylan Zhou: Candy Crush is NP complete and Mario Kart tour is PSPACE hard.
• Carolyn Ge, Kavya Kopparapu, and Eric Lin: a 4-part series on theory of machine learning including an interview with Les Valiant: What is learnability? Cryptographic limitations, Cryptography and Machine Learning, and future possibilities.
• And finally, Christine Cai, April Chen, Zev Nicolai-Scanio, and Grace Tian produced a rap song “Proof Göds” inspired by the course.

There is much work to still do on both the text and the course. Though the text has improved a lot (we do have 267 closed issues after all) some students still justifiably complained about typos, which can throw off people that are just getting introduced to the topic. I also want to add significantly more solved exercises and examples, since students do find them extremely useful. I need to significantly beef up the NP completeness chapter with more examples of reductions, though I do have Python implementation of several reductions and the Cook Levin theorem.

This type of course is often known as a “great ideas” in computer science, and so in the book I also added a “Big Idea” environment to highlight those. Of course some of those ideas are bigger than others, but I think the list below reflects well the contents of the course:

• If we can represent objects of type T as strings, then we can represent tuples of objects of type T as strings as well.
• A function is not the same as a program. A program computes a function.
• Two models are equivalent in power if they can be used to compute the same set of functions.
• Every finite function can be computed by a large enough Boolean  circuit.
• A program is a piece of text, and so it can be fed as input to other  programs.
• Some functions    cannot be computed by a Boolean circuit using fewer than exponential (in ) number of gates.
• We can precisely define what it means for a function to be computable by any possible algorithm.
• Using equivalence results such as those between Turing and RAM machines, we can “have our cake and eat it too”: We can use a simpler model such as Turing machines when we want to prove something can’t be done, and use a feature-rich model such as RAM machines when we want to prove something can be done.
• There is a  “universal” algorithm that can evaluate arbitrary algorithms on arbitrary inputs.
• There are some functions that can not be computed by any algorithm.
• If a function  is uncomputable we can show that another function is uncomputable by giving a way to reduce the task of computing to computing .
• We can use restricted computational models to bypass limitations such as uncomputability of the Halting problem and Rice’s Theorem. Such models can compute only a restricted subclass of functions, but allow to answer at least some semantic questions on programs.
• A proof is just a string of text whose meaning is given by a verification algorithm.
• The running time of an algorithm is not a number, it is a function of the length of the input.
• For a function and , we can formally define what it means for to be computable in time at most where is the size of the input.
• All “reasonable” computational models are equivalent if we only care about the distinction between  polynomial and exponential. (The book immediately notes quantum computers as a possible exception for this.)
• If we have more time, we can compute more functions.
• By “unrolling the loop” we can transform an algorithm that takes steps to compute into a circuit that uses gates to compute the restriction of to .
• A reduction shows that is “no harder than ” or equivalently that is “no easier than “.
• If a single -complete has a polynomial-time algorithm, then there is such an algorithm for every decision problem that corresponds to the existence of an efficiently-verifiable solution.
• If , we can efficiently solve a fantastic number of decision, search, optimization, counting, and sampling problems from all areas of human endeavors.
• A randomized algorithm outputs the correct value with good probability on every possible input.
• We can amplify the success of randomized algorithms to a value that is arbitrarily close to .
• There is no secrecy without randomness.
• Computational hardness is necessary and sufficient for almost all cryptographic applications.
• Just as we did with classical computation, we can define mathematical models for quantum computation, and represent quantum algorithms as binary strings.
• Quantum computers are not a panacea and are unlikely to solve complete problems, but they can provide exponential speedups to certain structured problems.

These are all ideas that I believe are important for Computer Science undergraduates to be exposed to, but covering all of these does make for a every challenging course, which gets literally mixed reviews from the students, with some loving it and some hating it. (I post all reviews on the course home page.) Running a 200-student class is definitely something that I’m still learning how to do.