Neil T. Heffernan
YOU?
Author Swipe
View article: From belief to evidence: simpler immediate feedback improves language learning and confidence in semi-open-ended questions
From belief to evidence: simpler immediate feedback improves language learning and confidence in semi-open-ended questions Open
Semi-open-ended questions (SOE) are commonly used in online foreign language assignments; however, the effects of online feedback on these question types have received limited research attention. To address this gap, the present study empl…
View article: E-TRIALS: Empowering Data-Driven Decisions to Enhance Computer-Based Learning Platforms
E-TRIALS: Empowering Data-Driven Decisions to Enhance Computer-Based Learning Platforms Open
Computer-based learning platforms (CBLPs) have become a common medium in schools, transforming how students learn and interact with educational content. However, researchers still lack adequate tools to address the diverse set of challenge…
View article: The Karp Dataset
The Karp Dataset Open
Understanding the mathematical reasoning capabilities of Large Language Models (LLMs) is a central topic in the study of artificial intelligence. This new domain necessitates the creation of datasets of reasoning tasks for both training an…
View article: DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Images
DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Images Open
In real-world settings, vision language models (VLMs) should robustly handle naturalistic, noisy visual content as well as domain-specific language and concepts. For example, K-12 educators using digital learning platforms may need to exam…
View article: Automated Feedback in Math Education: A Comparative Analysis of LLMs for Open-Ended Responses
Automated Feedback in Math Education: A Comparative Analysis of LLMs for Open-Ended Responses Open
The effectiveness of feedback in enhancing learning outcomes is well documented within Educational Data Mining (EDM). Various prior research has explored methodologies to enhance the effectiveness of feedback. Recent developments in Large …
View article: Investigating the Robustness of Knowledge Tracing Models in the Presence of Student Conceptual Drift
Investigating the Robustness of Knowledge Tracing Models in the Presence of Student Conceptual Drift Open
Knowledge Tracing (KT) has been an established problem in the educational data mining field for decades,and it is commonly assumed that the underlying learning process being modeled remains static. Giventhe ever changing landscape of onlin…
View article: Investigating the Robustness of Knowledge Tracing Models in the Presence of Student Conceptual Drift
Investigating the Robustness of Knowledge Tracing Models in the Presence of Student Conceptual Drift Open
Knowledge Tracing (KT) has been an established problem in the educational data mining field for decades,and it is commonly assumed that the underlying learning process being modeled remains static. Giventhe ever changing landscape of onlin…
View article: Multiple Choice vs. Fill-In Problems: The Trade-off Between Scalability and Learning
Multiple Choice vs. Fill-In Problems: The Trade-off Between Scalability and Learning Open
Learning experience designers consistently balance the trade-off between open and close-ended activities. The growth and scalability of Computer Based Learning Platforms (CBLPs) have only magnified the importance of these design trade-offs…
View article: Expert Features for a Student Support Recommendation Contextual Bandit Algorithm
Expert Features for a Student Support Recommendation Contextual Bandit Algorithm Open
Contextual multi-armed bandits have previously been used to personalize student support messages given to learners by supplying a model with relevant context about the user, problem, and available student supports. In this work, we propose…
View article: The Effect of Assistance on Gamers: Assessing The Impact of On-Demand Hints & Feedback Availability on Learning for Students Who Game the System
The Effect of Assistance on Gamers: Assessing The Impact of On-Demand Hints & Feedback Availability on Learning for Students Who Game the System Open
Gaming the system, characterized by attempting to progress through a learning activity without engaging in essential learning behaviors, remains a persistent problem in computer-based learning platforms. This paper examines a simple interv…
View article: Language and Mathematics Learning: A Comparative Study of Digital Learning Platforms
Language and Mathematics Learning: A Comparative Study of Digital Learning Platforms Open
This paper presents a conceptual exploration of how Digital Learning Platforms (DLPs) can be utilized to investigate the impact of language clarity, precision, engagement, and contextual relevance on mathematics learning from word problems…
View article: Generative AI for Education (GAIED): Advances, Opportunities, and Challenges
Generative AI for Education (GAIED): Advances, Opportunities, and Challenges Open
This survey article has grown out of the GAIED (pronounced "guide") workshop organized by the authors at the NeurIPS 2023 conference. We organized the GAIED workshop as part of a community-building effort to bring together researchers, edu…
View article: Immediate text-based feedback timing on foreign language online assignments: How immediate should immediate feedback be?
Immediate text-based feedback timing on foreign language online assignments: How immediate should immediate feedback be? Open
Immediate feedback has been considered a cornerstone of online language learning platforms. However, a closer reading of relevant research reveals that the definition of the term “immediate feedback” is inconsistent. Furthermore, findings …
View article: How to Open Science: Analyzing the Open Science Statement Compliance of the Learning @ Scale Conference
How to Open Science: Analyzing the Open Science Statement Compliance of the Learning @ Scale Conference Open
There have been numerous efforts documenting the effects of open science in existing papers; however, these efforts typically only consider the author's analyses and supplemental materials from the papers. While understanding the current r…
View article: How to Open Science: Promoting Principles and Reproducibility Practices within the Learning @ Scale Community
How to Open Science: Promoting Principles and Reproducibility Practices within the Learning @ Scale Community Open
Across the past decade, open science has increased in momentum, making research more openly available and reproducible. In addition, learning at scale systems have been developed to collect and apply models, features and reports to better …
View article: Investigating the Impact of Skill-Related Videos on Online Learning
Investigating the Impact of Skill-Related Videos on Online Learning Open
Many online learning platforms and MOOCs incorporate some amount of video-based content into their platform, but there are few randomized controlled experiments that evaluate the effectiveness of the different methods of video integration.…
View article: How Common are Common Wrong Answers? Crowdsourcing Remediation at Scale
How Common are Common Wrong Answers? Crowdsourcing Remediation at Scale Open
Solving mathematical problems is cognitively complex, involving strategy formulation, solution development, and the application of learned concepts. However, gaps in students' knowledge or weakly grasped concepts can lead to errors. Teache…
View article: Auto-scoring Student Responses with Images in Mathematics
Auto-scoring Student Responses with Images in Mathematics Open
Teachers often rely on the use of a range of open-ended problems to assess students' understanding of mathematical concepts. Beyond traditional conceptions of student open-ended work, commonly in the form of textual short-answer or essay r…
View article: Modeling and Analyzing Scorer Preferences in Short-Answer Math Questions
Modeling and Analyzing Scorer Preferences in Short-Answer Math Questions Open
Automated scoring of student responses to open-ended questions, including short-answer questions, has great potential to scale to a large number of responses. Recent approaches for automated scoring rely on supervised learning, i.e., train…
View article: How to Open Science: Debugging Reproducibility within the Educational Data Mining Conference
How to Open Science: Debugging Reproducibility within the Educational Data Mining Conference Open
Despite increased efforts to assess the adoption rates of open science and robustness of reproducibility in sub-disciplines of education technology, there is a lack of understanding of why some research is not reproducible. Prior work has …
View article: How to Open Science: Promoting Principles and Reproducibility Practices within the Educational Data Mining Community
How to Open Science: Promoting Principles and Reproducibility Practices within the Educational Data Mining Community Open
Across the past decade, open science has increased in momentum, making research more openly available and reproducible. Educational data mining, as a subfield of education technology, has been expanding in scope as well, developing and pro…
View article: Knowledge Tracing Over Time: A Longitudinal Analysis
Knowledge Tracing Over Time: A Longitudinal Analysis Open
The use of Bayesian Knowledge Tracing (BKT) models in predicting student learning and mastery, especially in mathematics, is a well-established and proven approach in learning analytics. In this work, we report on our analysis examining th…
View article: Effective Evaluation of Online Learning Interventions with Surrogate Measures
Effective Evaluation of Online Learning Interventions with Surrogate Measures Open
There is a growing need to empirically evaluate the quality of online instructional interventions at scale. In response, some online learning platforms have begun to implement rapid A/B testing of instructional interventions. In these scen…
View article: Using Auxiliary Data to Boost Precision in the Analysis of A/B Tests on an Online Educational Platform: New Data and New Results
Using Auxiliary Data to Boost Precision in the Analysis of A/B Tests on an Online Educational Platform: New Data and New Results Open
Randomized A/B tests within online learning platforms represent an exciting direction in learning sciences. With minimal assumptions, they allow causal effect estimation without confounding bias and exact statistical inference even in smal…
View article: A Bandit You Can Trust
A Bandit You Can Trust Open
This work proposes Dynamic Linear Epsilon-Greedy, a novel contextual multi-armed bandit algorithm that can adaptively assign personalized content to users while enabling unbiased statistical analysis. Traditional A/B testing and reinforcem…
View article: Using Auxiliary Data to Boost Precision in the Analysis of A/B Tests on an Online Educational Platform: New Data and New Results
Using Auxiliary Data to Boost Precision in the Analysis of A/B Tests on an Online Educational Platform: New Data and New Results Open
Randomized A/B tests within online learning platforms represent an exciting direction in learning sciences. With minimal assumptions, they allow causal effect estimation without confounding bias and exact statistical inference even in smal…
View article: Modeling and Analyzing Scorer Preferences in Short-Answer Math Questions
Modeling and Analyzing Scorer Preferences in Short-Answer Math Questions Open
Automated scoring of student responses to open-ended questions, including short-answer questions, has great potential to scale to a large number of responses. Recent approaches for automated scoring rely on supervised learning, i.e., train…
View article: How to Open Science: A Principle and Reproducibility Review of the Learning Analytics and Knowledge Conference
How to Open Science: A Principle and Reproducibility Review of the Learning Analytics and Knowledge Conference Open
Within the field of education technology, learning analytics has increased in popularity over the past decade. Researchers conduct experiments and develop software, building on each other's work to create more intricate systems. In paralle…