Explanipedia

Traceable Texts and Their Effects: A Study of Summary-Source Links in AI-Generated Summaries Open

Hita Kambhamettu, J. A. Flores, Andrew Head · 2025

FreeForm: Flexibly Augmenting Formulas with Synchronized Markup and Graphical Edits Open

Jeffrey Tao, Litao Yan, Jessica Shi, M. D. Ginsberg, Andrew Head · 2025

QED in Context: An Observation Study of Proof Assistant Users Open

Jessica Shi, Cassia Torczon, Harrison Goldstein, Benjamin C. Pierce, Andrew Head · 2025

Interactive theorem provers, or proof assistants, are important tools across many areas of computer science and mathematics, but even experts find them challenging to use effectively. To improve their design, we need a deeper, user-centric…

Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation Open

Yue Yang, Ajay Patel, Matt Deitke, Tanmay Gupta, Luca Weihs , et al. · 2025

Reasoning about images with rich text, such as charts and documents, is a critical application of vision-language models (VLMs). However, VLMs often struggle in these domains due to the scarcity of diverse text-rich vision-language data. T…

Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation Open

Yue Yang, Ajay Patel, Matt Deitke, Tanmay Gupta, Luca Weihs , et al. · 2025

Tyche: Making Sense of PBT Effectiveness Open

Harrison Goldstein, Jeffrey Tao, Zac Hatfield-Dodds, Benjamin C. Pierce, Andrew Head · 2024

Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models Open

Matt Deitke, Christopher Clark, Sang-Ho Lee, R. S. Tripathi, Yue Yang , et al. · 2024

Today's most advanced vision-language models (VLMs) remain proprietary. The strongest open-weight models rely heavily on synthetic data from proprietary VLMs to achieve good performance, effectively distilling these closed VLMs into open o…

Traceable Text: Deepening Reading of AI-Generated Summaries with Phrase-Level Provenance Links Open

Hita Kambhamettu, José de Jesús Vargas Flores, Andrew Head · 2024

As AI-generated summaries proliferate, how can we help people understand the veracity of those summaries? In this short paper, we design a simple interaction primitive, traceable text, to support critical examination of generated summaries…

Accelerating Scientific Paper Skimming with Augmented Intelligence Through Customizable Faceted Highlights Open

Raymond Fok, Luca Soldaini, Cassidy Trier, Erin Bransom, Kelsey MacMillan , et al. · 2024

Scholars need to keep up with an exponentially increasing flood of scientific papers. To aid this challenge, we introduce Scim , a novel intelligent interface that helps scholars skim papers to rapidly review and gain a cursory understandi…

Explainable Notes: Examining How to Unlock Meaning in Medical Notes with Interactivity and Artificial Intelligence Open

Hita Kambhamettu, Danaë Metaxa, Kevin B. Johnson, Andrew Head · 2024

Medical progress notes have recently become available to patients at an unprecedented scale. Progress notes offer patients insight into their care that they cannot find elsewhere. That said, reading a note requires patients to contend with…

Ivie: Lightweight Anchored Explanations of Just-Generated Code Open

Litao Yan, Alyssa Hwang, Zhiyuan Wu, Andrew Head · 2024

Programming assistants have reshaped the experience of programming into one\nwhere programmers spend less time writing and more time critically examining\ncode. In this paper, we explore how programming assistants can be extended to\naccel…

Grounded Intuition of GPT-Vision's Abilities with Scientific Images Open

Alyssa Hwang, Andrew Head, Chris Callison-Burch · 2023

GPT-Vision has impressed us on a range of vision-language tasks, but it comes with the familiar new challenge: we have little idea of its capabilities and limitations. In our study, we formalize a process that many have instinctively been …

FFL: A Language and Live Runtime for Styling and Labeling Typeset Math Formulas Open

Zhiyuan Wu, Jiening Li, Kevin Ma, Hita Kambhamettu, Andrew Head · 2023

As interest grows in learning math concepts in fields like data science and machine learning, it is becoming more important to help broad audiences engage with math notation. In this paper, we explore how authoring tools can help authors b…

CALYPSO: LLMs as Dungeon Master's Assistants Open

Andrew Zhu, Lara J. Martin, Andrew Head, Chris Callison-Burch · 2023

The role of a Dungeon Master, or DM, in the game Dungeons & Dragons is to perform multiple tasks simultaneously. The DM must digest information about the game setting and monsters, synthesize scenes to present to other players, and respond…

CALYPSO: LLMs as Dungeon Masters' Assistants Open

Andrew Zhu, Lara J. Martin, Andrew Head, Chris Callison-Burch · 2023

The role of a Dungeon Master, or DM, in the game Dungeons & Dragons is to perform multiple tasks simultaneously. The DM must digest information about the game setting and monsters, synthesize scenes to present to other players, and respond…

Rewriting the Script: Adapting Text Instructions for Voice Interaction Open

Alyssa Hwang, Natasha Oza, Chris Callison-Burch, Andrew Head · 2023

Voice assistants have sharply risen in popularity in recent years, but their use has been limited mostly to simple applications like music, hands-free search, or control of internet-of-things devices. What would it take for voice assistant…

Rewriting the Script: Adapting Text Instructions for Voice Interaction Open

Alyssa Hwang, Natasha Oza, Chris Callison-Burch, Andrew Head · 2023

Voice assistants have sharply risen in popularity in recent years, but their use has been limited mostly to simple applications like music, hands-free search, or control of internet-of-things devices. What would it take for voice assistant…

Complex Mathematical Symbol Definition Structures: A Dataset and Model for Coordination Resolution in Definition Extraction Open

Anna Martin-Boyle, Andrew Head, Kyle Lo, Risham Sidhu, Marti A. Hearst , et al. · 2023

Mathematical symbol definition extraction is important for improving scholarly reading interfaces and scholarly information extraction (IE). However, the task poses several challenges: math symbols are difficult to process as they are not …

CiteSee: Augmenting Citations in Scientific Papers with Persistent and Personalized Historical Context Open

Joseph Chee Chang, Amy X. Zhang, Jonathan Bragg, Andrew Head, Kyle Lo , et al. · 2023

When reading a scholarly article, inline citations help researchers contextualize the current article and discover relevant prior work. However, it can be challenging to prioritize and make sense of the hundreds of citations encountered du…

<span>Paper Plain</span> : Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processing Open

Tal August, Lucy Lu Wang, Jonathan Bragg, Marti A. Hearst, Andrew Head , et al. · 2023

When seeking information not covered in patient-friendly documents, healthcare consumers may turn to the research literature. Reading medical papers, however, can be a challenging experience. To improve access to medical papers, we explore…

Scim: Intelligent Skimming Support for Scientific Papers Open

Raymond Fok, Hita Kambhamettu, Luca Soldaini, Jonathan Bragg, Kyle Lo , et al. · 2023

Scholars need to keep up with an exponentially increasing flood of scientific papers. To aid this challenge, we introduce Scim, a novel intelligent interface that helps experienced researchers skim – or rapidly review – a paper to attain a…

The Semantic Reader Project: Augmenting Scholarly Documents through AI-Powered Interactive Reading Interfaces Open

Kyle Lo, Joseph Chee Chang, Andrew Head, Jonathan Bragg, Amy X. Zhang , et al. · 2023

Scholarly publications are key to the transfer of knowledge from scholars to others. However, research papers are information-dense, and as the volume of the scientific literature grows, the need for new technology to support the reading p…

Scim: Intelligent Skimming Support for Scientific Papers Open

Raymond Fok, Hita Kambhamettu, Luca Soldaini, Jonathan Bragg, Kyle Lo , et al. · 2022

Researchers need to keep up with immense literatures, though it is time-consuming and difficult to do so. In this paper, we investigate the role that intelligent interfaces can play in helping researchers skim papers, that is, rapidly revi…

Math Augmentation: How Authors Enhance the Readability of Formulas using Novel Visual Design Practices Open

Andrew Head, Amber Xie, Marti A. Hearst · 2022

With the increasing growth and impact of machine learning and other math-intensive fields, it is more important than ever to broaden access to mathematical notation. Can new visual and interactive displays help a wider readership successfu…

Paper Plain: Making Medical Research Papers Approachable to Healthcare Consumers with Natural Language Processing Open

Tal August, Lucy Lu Wang, Jonathan Bragg, Marti A. Hearst, Andrew Head , et al. · 2022

When seeking information not covered in patient-friendly documents, like medical pamphlets, healthcare consumers may turn to the research literature. Reading medical papers, however, can be a challenging experience. To improve access to me…

Augmenting Scientific Papers with Just-in-Time, Position-Sensitive Definitions of Terms and Symbols Open

Andrew Head, Kyle Lo, Dongyeop Kang, Raymond Fok, Sam Skjonsberg , et al. · 2021

Despite the central importance of research papers to scientific progress, they can be difficult to read. Comprehension is often stymied when the information needed to understand a passage resides somewhere else: in another section, or in a…

Fine-grained lineage for safer notebook interactions Open

Stephen Macke, Hongpu Gong, Doris Jung‐Lin Lee, Andrew Head, Doris Xin , et al. · 2021

Computational notebooks have emerged as the platform of choice for data science and analytical workflows, enabling rapid iteration and exploration. By keeping intermediate program state in memory and segmenting units of execution into so-c…

Modeling Mathematical Notation Semantics in Academic Papers Open

Hwiyeol Jo, Dongyeop Kang, Andrew Head, Marti A. Hearst · 2021

Natural language models often fall short when understanding and generating mathematical notation. What is not clear is whether these shortcomings are due to fundamental limitations of the models, or the absence of appropriate tasks. In thi…

Document-Level Definition Detection in Scholarly Documents: Existing Models, Error Analyses, and Future Directions Open

Dongyeop Kang, Andrew Head, Risham Sidhu, Kyle Lo, Daniel S. Weld , et al. · 2020

The task of definition detection is important for scholarly papers, because papers often make use of technical terminology that may be unfamiliar to readers. Despite prior work on definition detection, current approaches are far from being…

Augmenting Scientific Papers with Just-in-Time, Position-Sensitive\n Definitions of Terms and Symbols Open

Andrew Head, Kyle Lo, Dongyeop Kang, Raymond Fok, Sam Skjonsberg , et al. · 2020

Despite the central importance of research papers to scientific progress,\nthey can be difficult to read. Comprehension is often stymied when the\ninformation needed to understand a passage resides somewhere else: in another\nsection, or i…

Andrew Head YOU? Author Swipe