VISION-LANGUAGE / RESEARCH
InsightVLM — Statistically Verified Findings From Chart Images
A 20,000-chart dataset where every label is a statistical finding — bimodality, heteroscedasticity, a level shift — confirmed by a named test on the exact data plotted, and a CNN ensemble that names the finding on 86.7% of real charts against 46.8% for the best zero-shot vision-language model.
Public demo · Team project — Group Hocrux (5 members), Phitron AI/ML Thesis Track
86.7% — real test charts; best zero-shot VLM 46.8%
The problem
Chart datasets pair images with questions, captions or plotting code. None states what the data shows and checks that statement against the data, so no benchmark can score whether a model's reading of a chart is true.
The approach
Labels are produced by statistical tests, not annotators: a finding is assigned only when a named test (Shapiro–Wilk, Hartigan's dip, Mann–Kendall, Breusch–Pagan, Kruskal–Wallis, …) confirms it on the plotted array and no other class matches. 6,083 charts come from 654 real public tables in source-disjoint splits. ResNet and DenseNet classifiers are trained with renderer augmentation across three plotting libraries and ensembled on the validation split; the test split is scored once, against zero-shot Qwen-VL baselines, with bootstrap CIs and exact McNemar tests.
The results
86.7% [84.6, 88.6] on real test charts and 71.6% on charts re-drawn by Vega-Lite, a library never seen in training, against 46.8% for Qwen3-VL-8B zero-shot. A rendering-leak study found that version-1 time-series charts drew marks only for the matching class; removing them dropped one model from 0.957 to 0.311, so every reported number uses the corrected version 2. Manuscript in preparation; not peer reviewed.
Stack: PyTorch, ResNet-101, DenseNet-121, Qwen-VL baselines, matplotlib · Plotly · Vega-Lite, ONNX Runtime Web, Kaggle 2×T4