Quality Estimation for Machine Translation in Low-Resource Settings

Abstract:

Evaluating the quality of Machine Translation (MT) for low-resource languages is a challenging cross-lingual problem in Natural Language Processing. Quality Estimation (QE) aims to predict MT quality in the absence of reference translations. This thesis investigates QE through multilingual modelling, architectural adaptations and efficient learning strategies across nine low-resource language pairs. First, this thesis analyses how multilingual training influences QE performance, showing that combining linguistically related languages yields consistent improvements. The research then examines the suitability of small open-weight large language models (LLMs) for QE with instruction-following approaches. While LLMs show broad generalisation ability, empirical analysis reveals limitations for low-resource languages, including inconsistencies in tokenisation and weaker cross-lingual alignment. Results indicate that prompt-based adaptation alone is insufficient for reliable QE for low-resource languages and motivate architectural refinement. Building on this finding, the thesis proposes ALOPE, a regression-aligned framework for adapting decoder-based LLMs by leveraging informative intermediate Transformer layers embeddings for QE. Combined with lightweight regression heads and parameter-efficient techniques, ALOPE enables smaller open-weight LLMs to achieve strong performance on multiple language pairs while operating within memory and compute constraints. Further, this research extends QE beyond scalar prediction by introducing a dataset enriched with human-annotated weak error descriptions that provide contextual supervision. Building on this, we propose ALOPE-RL, a policy-based error-aware reinforcement learning framework that optimises quantised LLMs using multi-component rewards, achieving state-of-the-art performance and remaining effective under extreme data constraints. This thesis shows that efficient QE for low-resource languages can be performed through linguistically grounded multilingual transfer, regression-aligned adaptation, and error-aware reinforcement learning with limited data and computational resources. All datasets, models, and code developed in this study are publicly released to ensure reproducibility.