Olatunji Ruwase

3 papers · 2021–2024 · 2 conferences · across top CS/AI conferences

Achievements

🧭 Keyword Pioneer 🌍 Conference Polyglot (2) 🐝 Cross-Pollinator (12) 🌉 Interdisciplinary Bridge

Conferences

NIPS (2) ICLR (1)

Top co-authors

Lei Yang (2) Feng Yan (2) Heyang Qin (2) Yuxiong He (2) Zhewei Yao (2) Samyam Rajbhandari (2) Xiaoxia Wu (2) Zhangyang Wang (1) Beidi Chen (1) Connor Holmes (1)

Keywords

attention mechanism (1) neural network optimization (1) positional encoding (1) long context (1) context fusion (1) batch size (1) gradient similarity (1) rotary positional embedding (1) large language model (1) large scale training (1) adaptive batching (1)

Papers

Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding NIPS 2024

ZeRO++: Extremely Efficient Collective Communication for Large Model Training ICLR 2024

SimiGrad: Fine-Grained Adaptive Batching for Large Scale Training using Gradient Similarity Measurement NIPS 2021