MANUSCRIPT IN PREPARATION

FairShift-PFN

Fairness side-effects of label-free test-time adaptation in tabular foundation models

Research question: When a tabular foundation model is adapted to a shifted deployment population without labels, what happens to fairness — and which effects of that adaptation can be identified, bounded or certified without target labels?
Problem: Label-free test-time adaptation is usually judged by accuracy alone. A change that raises balanced accuracy can widen the gap between groups, and an accuracy-only no-regression check will not see it.
Dataset: Public data downloaded programmatically: ACS PUMS via folktables (Income, Employment, PublicCoverage, TravelTime, Mobility), UCI Diabetes 130-US Hospitals, HMDA loan-application records, and BRFSS 2018 (screened only), plus a synthetic shift generator.
Setup: Three tabular foundation models under real and synthetic deployment shifts. Experimental protocols were frozen before runs (pre-registration files in the repository); deviations are logged. A label-efficient gate (v3.2) requests labels only for rows whose decision the adaptation flips.

Results

0 — utility-harm runs of the gate on every real set (pre-registered) (source: README.md)
ρ 0.55–0.94 — label-free estimate vs true equalized-odds change, held-out real sets (source: README.md)
~10³ — labels needed to certify the ~1 pp gains on real shifts (budget tested: ≤ 400) (source: README.md)

The gate does not capture the roughly 1-point gains available on real shifts at budgets of 400 labels or fewer.
A pre-registered criterion (C3) failed on every real arm where it applied; this is reported in the repository rather than dropped.
An improved fairness metric is not evidence of ethical or legal fairness.
The revision rounds in the repository are simulated internal reviews, not peer review. Not submitted.

Sole author: research question, theory, library, experiments, pre-registration and manuscript.

Code and manuscript are in a private repository; available to reviewers on request.