NLP / CLASSIFICATION
Kotodama 言霊 — Emotion Intelligence Where the Colour Is the Output
Type how you feel and a fine-tuned transformer reads it across six emotions while an ink-spirit renders the result. Every surface is achromatic ink and washi paper — the only saturated colour on screen is set from the live prediction, so the interface cannot look confident about an emotion the model didn't return.
Public demo
0.75 — out-of-register macro-F1, was 0.37
The problem
The benchmark for this task is 20,000 English tweets in one narrow register: 98.7% contain the word 'feel' and 37.9% literally begin with 'i feel'. A model trained only on it learns the register rather than the emotion and scores well anyway. The failure that exposed it: 'I think I am not a good person' came back joy at 99% — the model matched 'good', missed the negation, and had seen exactly one training example in 16,000 containing 'good person'.
The approach
Four rungs, each a real step rather than a jump: TF-IDF + logistic regression (0.8328 macro-F1, 5 s), a BiLSTM with learned embeddings (4.7M params, 0.8805), DistilBERT on tweets only (0.8951), and RoBERTa trained on dair-ai plus mapped GoEmotions. Macro-F1 is the headline, not accuracy — joy is 9.4× the size of surprise, so a joy-only classifier scores 33.5% accuracy while being worthless — and inverse-frequency class weights normalised to mean 1 make one surprise example worth about nine joy examples in the gradient. GoEmotions supplies the phrasing variety and real negation the benchmark lacks; only unambiguous single-label rows that map cleanly into the six classes are used, with multi-label rows dropped rather than arbitrarily resolved. The dair-ai test split is never touched, so the combined model stays directly comparable to the tweet-only one.
The results
The shipped model scores slightly lower on the benchmark than the tweet-only DistilBERT — 0.8907 vs 0.8951, a bootstrap over 2,000 resamples putting the difference at 95% CI [−0.026, +0.016], indistinguishable from zero — and it is still the one that ships, because out-of-register macro-F1 went 0.3691 → 0.7468 and the negation challenge set went 15% → 46%. 'I think I am not a good person' now returns sadness at 84.8%. The EDA predicted the residual errors before the model existed: ranking each class's vocabulary by z-scored weighted log-odds produced three falsifiable predictions about which classes would blur — surprise↔fear, love→joy, anger↔fear — and all three survive into the fine-tuned model, which argues those errors belong to the label set rather than to capacity. Per-user isolation is enforced in exactly one function, so the personal dashboard and the admin overview run identical aggregation code and cannot drift; the admin surface returns 404 rather than 403 so it never confirms its own existence, and those tests were mutation-checked by sabotaging the scoping predicate. CI enforces that the label map stays byte-identical across ml/, backend/ and frontend/, because that ordering is the checkpoint's logit order — a silent reorder would return anger as fear with no error anywhere.
Stack: React 19, Vite, TypeScript, Tailwind v4, FastAPI, SQLAlchemy 2, RoBERTa, Firebase, Postgres, HF Spaces