Aaron Halfaker
YOU?
Author Swipe
View article: ORES-Inspect: A technology probe for machine learning audits on enwiki
ORES-Inspect: A technology probe for machine learning audits on enwiki Open
Auditing the machine learning (ML) models used on Wikipedia is important for ensuring that vandalism-detection processes remain fair and effective. However, conducting audits is challenging because stakeholders have diverse priorities and …
View article: Wikibench: Community-Driven Data Curation for AI Evaluation on Wikipedia
Wikibench: Community-Driven Data Curation for AI Evaluation on Wikipedia Open
AI tools are increasingly deployed in community contexts. However, datasets used to evaluate AI are typically created by developers and annotators outside a given community, which can yield misleading conclusions about AI performance. How …
View article: Summaries, Highlights, and Action items: Design, implementation and evaluation of an LLM-powered meeting recap system
Summaries, Highlights, and Action items: Design, implementation and evaluation of an LLM-powered meeting recap system Open
Meetings play a critical infrastructural role in coordinating work. The recent surge of hybrid and remote meetings in computer-mediated spaces has led to new problems (e.g., more time spent in less engaging meetings) and new opportunities …
View article: On Improving Summarization Factual Consistency from Natural Language Feedback
On Improving Summarization Factual Consistency from Natural Language Feedback Open
Yixin Liu, Budhaditya Deb, Milagro Teruel, Aaron Halfaker, Dragomir Radev, Ahmed Hassan Awadallah. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.
View article: Logical Transformers: Infusing Logical Structures into Pre-Trained Language Models
Logical Transformers: Infusing Logical Structures into Pre-Trained Language Models Open
Borui Wang, Qiuyuan Huang, Budhaditya Deb, Aaron Halfaker, Liqun Shao, Daniel McDuff, Ahmed Hassan Awadallah, Dragomir Radev, Jianfeng Gao. Findings of the Association for Computational Linguistics: ACL 2023. 2023.
View article: On Improving Summarization Factual Consistency from Natural Language Feedback
On Improving Summarization Factual Consistency from Natural Language Feedback Open
Despite the recent progress in language generation models, their outputs may not always meet user expectations. In this work, we study whether informational feedback in natural language can be leveraged to improve generation quality and us…
View article: Automatically Labeling Low Quality Content on Wikipedia By Leveraging Patterns in Editing Behaviors
Automatically Labeling Low Quality Content on Wikipedia By Leveraging Patterns in Editing Behaviors Open
Wikipedia articles aim to be definitive sources of encyclopedic content. Yet, only 0.6% of Wikipedia articles have high quality according to its quality scale due to insufficient number of Wikipedia editors and enormous number of articles.…
View article: Automatically Labeling Low Quality Content on Wikipedia by Leveraging Patterns in Editing Behaviors
Automatically Labeling Low Quality Content on Wikipedia by Leveraging Patterns in Editing Behaviors Open
Wikipedia articles aim to be definitive sources of encyclopedic content. Yet, only 0.6% of Wikipedia articles have high quality according to its quality scale due to insufficient number of Wikipedia editors and enormous number of articles.…
View article: Who Did What: Editor Role Identification in Wikipedia
Who Did What: Editor Role Identification in Wikipedia Open
Understanding the social roles played by contributors to online communities can facilitate the process of task routing. In this work, we develop new techniques to find roles in Wikipedia based on editors' low-level edit types and investiga…
View article: Defense Mechanism or Socialization Tactic? Improving Wikipedia’s Notifications to Rejected Contributors
Defense Mechanism or Socialization Tactic? Improving Wikipedia’s Notifications to Rejected Contributors Open
Unlike traditional firms, open collaborative systems rely on volunteers to operate, and many communities struggle to maintain enough contributors to ensure the quality and quantity of content. However, Wikipedia has historically faced the …
View article: Wikipedia ORES Explorer: Visualizing Trade-offs For Designing Applications With Machine Learning API
Wikipedia ORES Explorer: Visualizing Trade-offs For Designing Applications With Machine Learning API Open
With the growing industry applications of Artificial Intelligence (AI) systems, pre-trained models and APIs have emerged and greatly lowered the barrier of building AI-powered products. However, novice AI application designers often strugg…
View article: Effects of Algorithmic Flagging on Fairness
Effects of Algorithmic Flagging on Fairness Open
Online community moderators often rely on social signals such as whether or not a user has an account or a profile page as clues that users may cause problems. Reliance on these clues can lead to "overprofiling'' bias when moderators focus…
View article: Replication data for: Effects of algorithmic flagging on fairness: Quasi-experimental evidence from Wikipedia
Replication data for: Effects of algorithmic flagging on fairness: Quasi-experimental evidence from Wikipedia Open
Code overview The code is all in code.tar.gz. Identifying thresholds and cutoffs over time Pretty much all in identify_cutoffs.py. Iterates git repository, parses wmf-config/InitializeSettings.php. Interprets historical json versions. Buil…
View article: ORES
ORES Open
Algorithmic systems---from rule-based bots to machine learning classifiers---have a long history of supporting the essential work of content moderation and other curation work in peer production projects. From counter-vandalism to task rou…
View article: Effects of algorithmic flagging on fairness: quasi-experimental evidence from Wikipedia
Effects of algorithmic flagging on fairness: quasi-experimental evidence from Wikipedia Open
Online community moderators often rely on social signals such as whether or not a user has an account or a profile page as clues that users may cause problems. Reliance on these clues can lead to overprofiling bias when moderators focus on…
View article: Effects of algorithmic flagging on fairness: quasi-experimental evidence from Wikipedia
Effects of algorithmic flagging on fairness: quasi-experimental evidence from Wikipedia Open
Online community moderators often rely on social signals such as whether or not a user has an account or a profile page as clues that users may cause problems. Reliance on these clues can lead to overprofiling bias when moderators focus on…
View article: Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems
Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems Open
On Wikipedia, sophisticated algorithmic tools are used to assess the quality\nof edits and take corrective actions. However, algorithms can fail to solve the\nproblems they were designed for if they conflict with the values of communities\…
View article: Wikipedia Articles and Associated WikiProject Templates
Wikipedia Articles and Associated WikiProject Templates Open
== wikiproject_to_template.halfak_20191202.yaml ==The mapping of the canonical names of WikiProjects to all the templates that might be used to tag an article with this WikiProject that was used for generating this dump. For instance, the …
View article: ORES: Lowering Barriers with Participatory Machine Learning in Wikipedia
ORES: Lowering Barriers with Participatory Machine Learning in Wikipedia Open
Algorithmic systems---from rule-based bots to machine learning classifiers---have a long history of supporting the essential work of content moderation and other curation work in peer production projects. From counter-vandalism to task rou…
View article: Not at Home on the Range: Peer Production and the Urban/Rural Divide
Not at Home on the Range: Peer Production and the Urban/Rural Divide Open
Wikipedia articles about places, OpenStreetMap features, and other forms of peer-produced content have become critical sources of geographic knowledge for humans and intelligent technologies. In this paper, we explore the effectiveness of …
View article: PreCall: A Visual Interface for Threshold Optimization in ML Model Selection
PreCall: A Visual Interface for Threshold Optimization in ML Model Selection Open
Machine learning systems are ubiquitous in various kinds of digital applications and have a huge impact on our everyday life. But a lack of explainability and interpretability of such systems hinders meaningful participation by people, esp…
View article: PreCall: A Visual Interface for Threshold Optimization in ML Model Selection
PreCall: A Visual Interface for Threshold Optimization in ML Model Selection Open
Machine learning systems are ubiquitous in various kinds of digital applications and have a huge impact on our everyday life. But a lack of explainability and interpretability of such systems hinders meaningful participation by people, esp…
View article: Wikimedia App View
Wikimedia App View Open
Each row represents a page view within the official Wikimedia mobile App. This dataset was gather by randomly sampling 100k unique application IDs from the request logs and then gathered all of the requests made by those application IDs.
View article: Wikimedia Desktop View
Wikimedia Desktop View Open
Each row represents a page view to the Desktop version of Wikimedia's sites (e.g. wikipedia, wiktionary, etc.) This dataset was gathered through the use of a snippet of Javascript that used a cookie to randomly sample readers at 1/1000. 7.…
View article: Wikimedia Mobile View
Wikimedia Mobile View Open
Each row represents a page view to the Mobile version of Wikimedia's sites (e.g. wikipedia, wiktionary, etc.). This dataset was gather by randomly sampling 100k unique IPs from the request logs and then gathering all of the requests made b…
View article: Cyclopath Route Get
Cyclopath Route Get Open
Each row represents a route search request submitted by a logged-in user via cyclopath.org.
View article: League of Legends Game
League of Legends Game Open
This dataset contains start & end timestamps for League of Legends games played via the Duowan plugin. We randomly sampled 100k user_ids and gathered all games associated with those users.
View article: Activity Sessions datasets
Activity Sessions datasets Open
This article contains a set of datasets used to demonstrate a strong regularity in inter-activity time. See the paper: User Session Identification Based on Strong Regularities in Inter-activity Time http://arxiv.org/abs/1411.2878 Abstract …
View article: (English) Wikipedia Edit
(English) Wikipedia Edit Open
Each row represents an edit saved by a user to a page in Wikipedia. 100k users who saved at least one edit were randomly sampled and the timestamps of all of their revisions were retrieved.
View article: Open Street Map Changeset
Open Street Map Changeset Open
This open-source alternative mapping service also publishes regular database dumps. We downloaded a full history dump of OSM contributions as of 24 February 2014, restricting this to the North American region as defined by Geofabrik 6 , wh…