MANUSCRIPT IN PREPARATION

InsightVLM

Naming the statistical finding a chart shows — and checking it against the plotted data

Research question

Can a vision model state what a chart's data shows — bimodality, heteroscedasticity, a level shift, a non-significant group difference — and be scored against a statistical test run on the exact data that was plotted?

Problem

Chart datasets such as FigureQA, DVQA, PlotQA and ChartQA pair charts with questions and answers; ChartGen and ChartCoder pair them with plotting code; CharXiv tests description and reasoning over scientific figures. Their supervision describes or queries the chart. None attaches a statement about the data that has been checked to be true, so none can score whether a model's stated finding is correct.

Dataset

InsightVLM-20K: 20,000 charts in 26 chart types, 20 finding classes, exactly 1,000 per class split 700/100/200. A label is issued only when a named test (Shapiro–Wilk, Hartigan's dip, Mann–Kendall, Breusch–Pagan, Kruskal–Wallis, …) confirms it on exactly the plotted array and no other class matches; re-running the tests reproduces 99.95% of labels. 6,083 charts come from 654 real public tables (OpenML, World Bank, Our World in Data, UCI, vega-datasets, seaborn-data) in source-disjoint splits, so real-data generalisation is measured separately from synthetic. Licence: CC BY 4.0 for the corpus; real-source charts keep their table's licence; synthetic charts CC0.

Setup

ResNet-50, ResNet-101 and DenseNet-121 classifiers at 384 px, trained on Kaggle (2× T4) with a supplementary real-chart pool and renderer augmentation (matplotlib, Plotly, randomised styles). The deployed ensemble averages ResNet-101 and DenseNet-121 with equal weights; every selection was made on the validation split and the held-out test split was scored once. Test charts are also re-drawn with Plotly and with Vega-Lite, a library never seen in training.

Results

• 86.7% — real test charts (n = 1,192), 95% CI [84.6, 88.6] (source: results/stats_report.json)
• 46.8% — best zero-shot VLM on the same charts (Qwen3-VL-8B) (source: results/stats_report.json)
• 71.6% — charts re-drawn by a plotting library never seen in training (source: results/stats_report.json)

Limitations

• Distribution shape is the weakest family (0.763 on real charts). Accuracy is lowest where real examples are rare: heavy tails 0.385 (n = 26), seasonality 0.462 (n = 13), left skew 0.474 (n = 19).
• The deployed models are CNN classifiers over a fixed 20-class taxonomy. They name a finding; they do not generate the reasoning or cite numbers.
• Vega-Lite re-renders lose about 15 points against matplotlib.
• Not peer reviewed. The dataset is hosted on Kaggle; a public archive with a DOI is planned but does not exist yet.

Contribution

Team project — Group Hocrux, Phitron AI/ML Thesis Track. A K M Asifuzzaman is the first-listed author on the manuscript draft and maintains the repository and the public demo. The repository does not document a per-member split of the work, so none is claimed here.

The demo is public and runs in the browser. Code, manuscript draft and thesis are in a private repository; available to reviewers on request.