Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity

Lu Yin , You Wu , Zhenyu Zhang , Cheng-Yu Hsieh , Yaqing Wang , Yiling Jia , Gen Li , Ajay Kumar Jaiswal , Mykola Pechenizkiy , Yi Liang , Michael Bendersky , Zhangyang Wang , Shiwei Liu