Deep Double Descent (cross-posted on OpenAI blog)

By Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever This is a lightly edited and expanded version of the following post on the OpenAI blog about the following paper. While I usually don't advertise my own papers on this blog, I thought this might be of interest to theorists, and … Continue reading Deep Double Descent (cross-posted on OpenAI blog)

Introduction to AMP and the Replica Trick

(This post from the lecture by Yueqi Sheng) In this post, we will talk about detecting phase transitions using Approximate-Message-Passing (AMP), which is an extension of Belief-Propagation to “dense” models. We will also discuss the Replica Symmetric trick, which is a heuristic method of analyzing phase transitions. We focus on the Rademacher spiked Wigner model (defined below), … Continue reading Introduction to AMP and the Replica Trick