Fred Hohman
/ CSE Ph.D. Student at GT

TeleGam: Combining Visualization and Verbalization for Interpretable Machine Learning

Fred Hohman*, Arjun Srinivasan*, Steven Drucker

The TeleGam user interface contains (A) a model selector and parameters for the visualizations and verbalizations. (B) The Global Model View displays model feature-level verbalizations of GAM shape function charts that describe a feature's overall impact on model predictions. (C) The Local Instance View displays two data instance's waterfall charts, an explanatory visualization that shows the cumulative sum of the contribution each feature has on the final prediction. Alongside are instance-level verbalizations that, when brushed, highlight in orange the corresponding marks of the visualization that the verbalization refers to. (D) Settings to interactively tune verbalization generation thresholds.

Abstract

While machine learning (ML) continues to find success in solving previously-thought hard problems, interpreting and exploring ML models remains challenging. Recent work has shown that visualizations are a powerful tool to aid debugging, analyzing, and interpreting ML models. However, depending on the complexity of the model (e.g., number of features), interpreting these visualizations can be difficult and may require additional expertise. Alternatively, textual descriptions, or verbalizations, can be a simple, yet effective way to communicate or summarize key aspects about a model, such as the overall trend in a model’s predictions or comparisons between pairs of data instances. With the potential benefits of visualizations and verbalizations in mind, we explore how the two can be combined to aid ML interpretability. Specifically, we present a prototype system, TeleGam, that demonstrates how visualizations and verbalizations can collectively support interactive exploration of ML models, for example, generalized additive models (GAMs). We describe TeleGam’s interface and underlying heuristics to generate the verbalizations. We conclude by discussing how TeleGam can serve as a platform to conduct future studies for understanding user expectations and designing novel interfaces for interpretable ML.

Citation

TeleGam: Combining Visualization and Verbalization for Interpretable Machine Learning
Fred Hohman*, Arjun Srinivasan*, Steven Drucker
IEEE Visualization Conference (VIS). Vancouver, Canada, 2019.
Project Demo PDF Preview Code BibTeX * Authors contributed equally

BibTeX


@article{hohman2019telegam,
  title={TeleGam: Combining Visualization and Verbalization for Interpretable Machine Learning,
  author={Hohman, Fred and Srinivasan, Arjun and Drucker, Steven M.},
  journal={IEEE Visualization Conference (VIS)},
  year={2019},
  publisher={IEEE},
  url={https://poloclub.github.io/telegam/}
}