Yet another backpropagation tutorial

I am teaching deep learing this week in Harvard's CS 182 (Artificial Intelligence) course. As I'm preparing the back-propagation lecture, Preetum Nakkiran told me about Andrej Karpathy's awesome micrograd package which implements automatic differentiation for scalar variables in very few lines of code. I couldn't resist using this to show how simple back-propagation and stochastic … Continue reading Yet another backpropagation tutorial

Full-replica-symmetry-breaking based algorithms for dummies

One of the fascinating lines of research in recent years has been a convergence between the statistical physics and theoretical computer science points of view on optimization problems.`This blog post is mainly a note to myself (i.e., I'm the "dummy" 😃), trying to work out some basic facts in some of this line of work. … Continue reading Full-replica-symmetry-breaking based algorithms for dummies

Understanding generalization requires rethinking deep learning?

Yamini Bansal, Gal Kaplun, and Boaz Barak (See also paper on arxiv, code on gitlab, upcoming talk by Yamini&Boaz, video of past talk) A central puzzle of deep learning is the question of generalization. In other words, what can we deduce from the training performance of a neural network about its test performance on fresh … Continue reading Understanding generalization requires rethinking deep learning?

SIGACT research highlights – call for nominations

TL;DR: Know of a great recent paper that should be highlighted to the theory community and beyond? Email a nomination to by October 19th. The goal of the SIGACT Research Highlights Committee is to help promotetop computer science theory research via identifying results that are ofhigh quality and broad appeal to the general computer … Continue reading SIGACT research highlights – call for nominations

Highlights of Algorithms (HALG) -free – Aug 31- Sep 2

[Guest post by Yossi Azar] The 5th Highlights of Algorithms conference (HALG 2020) will take place Aug 31- Sep 2, 2020. The Highlights of Algorithms conference is a forum for presenting the highlights of recent developments in algorithms and for discussing potential further advances in this area. The conference will provide a broad picture … Continue reading Highlights of Algorithms (HALG) -free – Aug 31- Sep 2