LOW-RESOURCE NLP / RESEARCH
ChatgaiyyaLM — Language Adaptation for a Dialect With No Corpus
Chittagonian (চাটগাঁইয়া) has tens of millions of speakers and almost no NLP resources. This is the corpus, the benchmark, and the adapted model built anyway — plus the result that the mined rule system beats every neural arm in one direction, reported rather than buried.
Public demo
20.5 → 65.6 — chrF, Chittagonian → Bangla, n = 120
The problem
The research proposal assumed a Chittagonian corpus existed. It does not — the published resources are four small parallel datasets and two word lists, and after harvesting and deduplicating all of them there are ~12k real Chittagonian sentences, three orders of magnitude short of what continued pretraining needs. So corpus construction under extreme scarcity became the first contribution rather than a preliminary step. The second problem is that nobody can say how dialectal a sentence is: a binary Bangla-vs-Chittagonian classifier hits AUC 0.9986 and is useless for stratification, because 93% of real sentences score above 0.8.
The approach
Chittagonian is not noise over Bangla, it is a systematic phonological transform — so recover the transform and apply it to a large standard-Bangla corpus to synthesize dialectal text at volume. Levenshtein character alignment over 33,815 parallel word pairs yields 656 context-conditioned rewrite rules scored by held-out precision, reproducing documented phonology (ক→গ, প→ফ, খ→হ, আমি→আঁই) with no hand-coded linguistics. Graded dialectness supervision is induced the same way: transducing a sentence at rate p makes it ~p dialectal, so (text@p, p) is a free regression pair — ridge over char n-grams reaches R² 0.68, orders real Bangla 0.112 below real Chittagonian 0.697, and stays monotone in p. A tokenizer study measures the dialect tax directly, and vocabulary augmentation (3,914 mined subwords, mean-of-pieces init) takes Qwen2.5 Chittagonian fertility 5.722 → 2.254. Five adaptation arms then ran on Kaggle T4 ×2 under fp16 autocast with fp32 master weights, because T4 is Turing and has no hardware bf16 — torch.cuda.is_bf16_supported() returns True there anyway, so the guard tests compute capability instead.
The results
On the gold translation track of ChatgaiyyaBench (120 fixed-seed items per direction), the best released adapter reaches chrF 65.64 on Chittagonian→Bangla. The unadapted base model scores 20.54 against 29.51 for copying the input unchanged, so an unadapted LLM is worse than doing nothing. The mined rule transducer reaches 57.94 on Bangla→Chittagonian at zero GPU cost and beats every neural arm in that direction. The model cards report a packaging failure rather than hiding it: an earlier adapter measured 58.46 in its training run but 35.66 when its published weights were re-measured, and is marked superseded. Deduplication removed 35.3% of records because Vashantor test sentences sat in ChatgaiyyaAlap training data; leaving that in would have invalidated every number above. ChatgaiyyaBench ships 13,497 items across 8 tasks, dialectness-stratified and split 7,417 gold / 6,080 silver — reported separately, because silver tracks support relative model comparison and are not a substitute for native-speaker evaluation.
Stack: PyTorch, transformers, PEFT / LoRA, Qwen2.5, Hugging Face Hub, scikit-learn, React 19, Vite