Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition

Zheyang Xiong , Ziyang Cai , John Cooper , Albert Ge , Vasilis Papageorgiou , Zack Sifakis , Angeliki Giannou , Ziqian Lin , Liu Yang , Saurabh Agarwal , Grigorios Chrysos , Samet Oymak , Kangwook Lee , Dimitris Papailiopoulos