Speech-to-Speech Machine Translation for Dialectal Variations of Hindi

AbstractHindi has many dialects and they are vital to India’s cultural and linguistics heritage. However, many of them have been largely overlooked in modern language technological advancements, primarily due to lack proper resources. In this study, we explore speech-to-speech machine translation (S2ST) for four Hindi dialects, i.e., Awadhi, Bhojpuri, Braj Bhasha, and Magahi. We adopt a cascaded S2ST pipeline comprising of three stages: Automatic Speech Recognition (ASR), Machine Translation (MT), and Text-to-Speech (TTS). We evaluate many recent large language models (LLMs) for dialect-to-Hindi and dialect-to-English translations in zero-shot, few-shot, and chain-of-thought setups. Our comparative analysis offers insights into the current capabilities and limitations of LLM-based approaches for low-resource dialectal S2ST in Indian context.