Quantifying Cross-Lingual Interference: Algorithmic Standardization of Kamtapuri in Large Language Models

AbstractMultilingual Large Language Models (LLMs) often demonstrate impressive zero-shot capabilities on low-resource languages. However, for languages that share a script and significant lexical overlap with a high-resource language (HRL), models may exhibit negative transfer. Focusing on Kamtapuri (Rajbanshi), a distinct low-resource language of North Bengal, we investigate the extent to which SOTA models (e.g., GPT-5.1, Gemini 2.5) preserve distinct dialectal features versus reverting to the dominant language’s norms. We introduce the Kamta-Shibboleth-100 (Benchmark available at: https://github.com/kamtapuri-research/Kamta-Shibboleth-100-BENCHMARK), a diagnostic benchmark derived from a curated 400k-token corpus. Our evaluation reveals a significant discrepancy: while models show high receptive understanding (up to 88% translation accuracy), they exhibit a 0% Syntactic Competence Rate in zero-shot generation of distinct Kamtapuri morphology, compared to 96%+ accuracy on a Standard Bengali control set. Even with 5-shot prompting, syntactic accuracy improves only to 10%, while the Substitution Erasure Rate (SER) reaches 71%, systematically replacing Kamtapuri vocabulary with Bengali cognates. We characterize this behavior not as a lack of knowledge, but as a strong alignment bias toward high-resource standards.