Simon Kornblith | Understanding neural networks from a representational perspective: effects of width and depth

Deep learning involves complex interactions between data, architecture, and training objectives. In this talk, I will present work toward understanding these interactions through the lens of neural network representations. I will first introduce the challenges of measuring similarity between neural network representations. I will then discuss the use of centered kernel alignment (CKA) as a similarity index between representations, and show that it addresses these challenges. Finally, I will describe our recent work applying CKA and other tools to understand how neural network representations change as networks become wider or deeper.

Meeting ID: 961 0650 8785