Skip to content
AI

AI Charts Need to Remember What the Reader Selected

Flint-interactive makes AI charts easier to explore. TENS examines how selections, filters and resets should stay attached to the answers that follow.

Share Email
Colorful software code displayed on a computer monitor
Source code displayed on a computer monitor. Photo: Markus Spiske via Wikimedia Commons. CC0 1.0 Universal Public Domain Dedication. Center-cropped from 5,760 × 3,840 pixels to 16:9 and resized to 2,400 × 1,350 pixels; no generative or content-altering edits.

A chart can respond perfectly to a click while the answer built from that click is wrong. That distinction matters as AI assistants move from drawing pictures of data to letting people explore those pictures and ask follow-up questions. Microsoft Research’s October 22 account of Flint-interactive puts that transition into focus: making a chart easier to operate also makes its changing state part of the evidence an assistant must understand.

Microsoft describes a system that translates gestures into meaningful events and applies higher-level updates to the display. Its examples connect selections across charts, tables and text. The company argues that moving rendering dependencies into a compiler can reduce the work agents spend assembling and repairing interactive graphics. These are the developers’ claims, rather than independently measured gains reported by TENS.

Our analysis is that the most consequential test lies at the handoff between exploration and explanation. A reader should be able to establish which records were selected, what changed on screen and which of those changes the assistant used when answering. A polished interaction alone cannot establish that chain.

Highlighting and filtering ask different questions

Vega-Lite’s official documentation provides a useful comparison. It describes parameters that turn user input into data queries and can control appearance, filtering or the extent of an axis. Its examples distinguish filtering records from changing the displayed scale. It also documents a default under which an empty selection predicate includes all data values, with an option to alter that behavior.

Those distinctions reveal why a visually similar gesture can carry different analytical meanings. Consider a hypothetical dashboard of delivery times. Highlighting one depot might leave the overall average unchanged; filtering to that depot could recalculate the average for a smaller group. Zooming might simply enlarge part of the display. An assistant that treats all three operations as equivalent could describe the wrong population without any obvious rendering error.

TENS proposes a visible selection receipt alongside each follow-up answer: the selected group, active filters, time range and number of included records. The receipt should distinguish a change in emphasis from a change in the calculation. This is a proposed evaluation method, not a claim that either project currently supplies that complete feature.

Resetting a chart should reset the explanation

The empty-selection case deserves its own test. In the delivery example, clearing a depot selection might restore the full dataset, preserve a previous filter or leave no records available, depending on the application’s design. Each behavior can be intentional. The failure would be an answer that silently assumes a different behavior from the one the interface implements.

Our proposed review would run the same question before selection, during selection and after reset. Compare the included record identifiers at each stage, rather than judging correctness from colors or labels alone. Then check whether the answer’s stated population matches those records. A reset control should restore a documented analytical state, not merely remove the visible highlight.

This comparison also helps separate two responsibilities. A charting system can expose precise selection information, while the surrounding assistant can still ignore or misread it. Conversely, a careful assistant cannot reliably recover a filter that the application never passes along. Evaluation should identify which boundary lost the meaning before assigning credit or blame to the model.

Measure the saved work after checking the result

Microsoft’s public repository dates Flint 1.0’s introduction of interactive capabilities to October 19. The October 22 research article supplies the current explanation of that work; it should not be mistaken for a separate software release date. The repository describes a shared specification that compiles into chart formats and an interface for linking interaction state with application behavior.

That architecture suggests an opportunity for repeatable checks. Save the input dataset, chart specification and sequence of selections with an accepted answer. A reviewer could then compare the intended analytical population with the population actually used. The goal would be to preserve the meaning of the exploration, even when the display is rebuilt or shown in a different setting.

For a practical comparison, TENS would measure time to a verified answer across a fixed set of tasks, including selection, reset and contradictory filters. Count corrections and abandoned attempts as well as completed charts. Faster generation would be valuable, but a workflow that transfers more checking work to the reader should not receive the same efficiency credit as one that reduces total effort.

The sources reviewed do not establish that Flint-interactive outperforms alternatives on that end-to-end measure. They establish a design direction and documented mechanisms worth examining. The next useful evidence would connect those mechanisms to fewer interpretation errors and less review time. As AI charts become conversational, the important question is whether their answers remain attached to the data the reader actually chose.

Image: Illustrative software photograph, not a Flint screenshot. Photo by Markus Spiske via Wikimedia Commons. CC0 1.0 Universal Public Domain Dedication. Center-cropped to 16:9 and resized; no generative or content-altering edits.