Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective

Wu Lin , Felix Dangel , Runa Eschenhagen , Juhan Bae , Richard E. Turner , Alireza Makhzani