Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge

Hanna Wallach , Meera Desai , A. Feder Cooper , Angelina Wang , Chad Atalla , Solon Barocas , Su Lin Blodgett , Alexandra Chouldechova , Emily Corvi , P. Alex Dow , Jean Garcia-Gathright , Alexandra Olteanu , Nicholas J Pangakis , Stefanie Reed , Emily Sheng , Dan Vann , Jennifer Wortman Vaughan , Matthew Vogel , Hannah Washington , Abigail Z. Jacobs