Papers
5,241 papers found
3D-RPE: Enhancing Long-Context Modeling Through 3D Rotary Position Encoding
Xindian Ma, Wenyuan Liu, Peng Zhang et al.
Training with “Paraphrasing the Original Text” Teaches LLM to Better Retrieve in Long-Context Tasks
Yijiong Yu, Yongfeng Huang, Zhixiao Qi et al.
Leveraging Attention to Effectively Compress Prompts for Long-Context LLMs
Yunlong Zhao, Haoran Wu, Bo Xu
Characterizing the Efficiency vs. Accuracy Trade-off for Long-Context NLP Models
Phyllis Ang, Bhuwan Dhingra, Lisa Wu Wills
Making Long-Context Language Models Better Multi-Hop Reasoners
Yanyang Li, Shuo Liang, Michael Lyu et al.
Landmark Embedding: A Chunking-Free Embedding Method For Retrieval Augmented Long-Context Large Language Models
Kun Luo, Zheng Liu, Shitao Xiao et al.
Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Training
Junqing He, Kunhao Pan, Xiaoqun Dong et al.
M4LE: A Multi-Ability Multi-Range Multi-Task Multi-Domain Long-Context Evaluation Benchmark for Large Language Models
Wai-Chung Kwan, Xingshan Zeng, Yufei Wang et al.
DocFinQA: A Long-Context Financial Reasoning Dataset
Varshini Reddy, Rik Koncel-Kedziorski, Viet Dac Lai et al.
Sliding Windows Are Not the End: Exploring Full Ranking with Long-Context Large Language Models
Wenhan Liu, Xinyu Ma, Yutao Zhu et al.
Untie the Knots: An Efficient Data Augmentation Strategy for Long-Context Pre-Training in Language Models
Junfeng Tian, Da Zheng, Yang Chen et al.
What Really Matters in Many-Shot Attacks? An Empirical Study of Long-Context Vulnerabilities in LLMs
Sangyeop Kim, Yohan Lee, Yongwoo Song et al.
LADM: Long-context Training Data Selection with Attention-based Dependency Measurement for LLMs
Jianghao Chen, Junhong Wu, Yangyifan Xu et al.
Hierarchical Document Refinement for Long-context Retrieval-augmented Generation
Jiajie Jin, Xiaoxi Li, Guanting Dong et al.
LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks
Yushi Bai, Shangqing Tu, Jiajie Zhang et al.
LongReward: Improving Long-context Large Language Models with AI Feedback
Jiajie Zhang, Zhongni Hou, Xin Lv et al.
L-CiteEval: A Suite for Evaluating Fidelity of Long-context Models
Zecheng Tang, Keyan Zhou, Juntao Li et al.
How to Train Long-Context Language Models (Effectively)
Tianyu Gao, Alexander Wettig, Howard Yen et al.
Boosting Long-Context Information Seeking via Query-Guided Activation Refilling
Hongjin Qian, Zheng Liu, Peitian Zhang et al.
LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillation
Zican Dong, Junyi Li, Jinhao Jiang et al.
SCOPE: Optimizing Key-Value Cache Compression in Long-context Generation
Jialong Wu, Zhenglin Wang, Linhai Zhang et al.
MoQAE: Mixed-Precision Quantization for Long-Context LLM Inference via Mixture of Quantization-Aware Experts
Wei Tao, Haocheng Lu, Xiaoyang Qu et al.
Scaling up the State Size of RNN LLMs for Long-Context Scenarios
Kai Liu, Jianfei Gao, Kai Chen
MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference
Kunxi Li, Zhonghua Jiang, Zhouzhou Shen et al.
LIFBench: Evaluating the Instruction Following Performance and Stability of Large Language Models in Long-Context Scenarios
Xiaodong Wu, Minhao Wang, Yichen Liu et al.