Explanipedia

Query Refinement Prompts for Closed-Book Long-Form QA Open

Reinald Kim Amplayo, Kellie Webster, Michael J. Collins, Dipanjan Das, Shashi Narayan · 2023

Reinald Kim Amplayo, Kellie Webster, Michael Collins, Dipanjan Das, Shashi Narayan. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models Open

Bernd Bohnet, Vinh Q. Tran, Pat Verga, Roee Aharoni, Daniel Andor , et al. · 2022

Large language models (LLMs) have shown impressive results while requiring little or no direct supervision. Further, there is mounting evidence that LLMs may have potential in information-seeking scenarios. We believe the ability of an LLM…

Query Refinement Prompts for Closed-Book Long-Form Question Answering Open

Reinald Kim Amplayo, Kellie Webster, Michael J. Collins, Dipanjan Das, Shashi Narayan · 2022

Large language models (LLMs) have been shown to perform well in answering questions and in producing long-form texts, both in few-shot closed-book settings. While the former can be validated using well-known evaluation metrics, the latter …

Flexible text generation for counterfactual fairness probing Open

Zee Fryer, Vera Axelrod, Ben Packer, Alex Beutel, Jilin Chen , et al. · 2022

A common approach for testing fairness issues in text-based classifiers is through the use of counterfactuals: does the classifier output change if a sensitive attribute in the input is changed? Existing counterfactual generation methods t…

Flexible text generation for counterfactual fairness probing Open

Zee Fryer, Vera Axelrod, Ben Packer, Alex Beutel, Jilin Chen , et al. · 2022

A common approach for testing fairness issues in text-based classifiers is through the use of counterfactuals: does the classifier output change if a sensitive attribute in the input is changed? Existing counterfactual generation methods t…

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts Open

Nan Du, Yanping Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin , et al. · 2021

Scaling language models with more data, compute and parameters has driven significant progress in natural language processing. For example, thanks to scaling, GPT-3 was able to achieve strong results on in-context learning tasks. However, …

Towards Deconfounding the Influence of Subject's Demographic Characteristics in Question Answering. Open

Maharshi Gor, Kellie Webster, Jordan Boyd‐Graber · 2021

Question Answering (QA) tasks are used as benchmarks of general machine intelligence. Therefore, robust QA evaluation is critical, and metrics should indicate how models will answer any question. However, major QA datasets have skewed dist…

Toward Deconfounding the Influence of Entity Demographics for Question Answering Accuracy Open

Maharshi Gor, Kellie Webster, Jordan Boyd‐Graber · 2021

The goal of question answering (QA) is to answer any question. However, major QA datasets have skewed distributions over gender, profession, and nationality. Despite that skew, model accuracy analysis reveals little evidence that accuracy …

How to Write a Bias Statement: Recommendations for Submissions to the\n Workshop on Gender Bias in NLP Open

Christian Hardmeier, Marta R. Costa‐jussà, Kellie Webster, Will Radford, Su Lin Blodgett · 2021

At the Workshop on Gender Bias in NLP (GeBNLP), we'd like to encourage\nauthors to give explicit consideration to the wider aspects of bias and its\nsocial implications. For the 2020 edition of the workshop, we therefore\nrequested that al…

How to Write a Bias Statement: Recommendations for Submissions to the Workshop on Gender Bias in NLP Open

Christian Hardmeier, Marta R. Costa‐jussà, Kellie Webster, Will Radford, Su Lin Blodgett · 2021

At the Workshop on Gender Bias in NLP (GeBNLP), we'd like to encourage authors to give explicit consideration to the wider aspects of bias and its social implications. For the 2020 edition of the workshop, we therefore requested that all a…

They, Them, Theirs: Rewriting with Gender-Neutral English Open

Tony Sun, Kellie Webster, Apurva Shah, William Yang Wang, Melvin Johnson · 2021

Responsible development of technology involves applications being inclusive of the diverse set of users they hope to support. An important part of this is understanding the many ways to refer to a person and being able to fluently change b…

Toward Deconfounding the Effect of Entity Demographics for Question Answering Accuracy Open

Maharshi Gor, Kellie Webster, Jordan Boyd‐Graber · 2021

The goal of question answering (QA) is to answer _any_ question. However, major QA datasets have skewed distributions over gender, profession, and nationality. Despite that skew, an analysis of model accuracy reveals little evidence that a…

Underspecification Presents Challenges for Credibility in Modern Machine Learning Open

Alexander D’Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi , et al. · 2020

ML models often exhibit unexpectedly poor behavior when they are deployed in real-world domains. We identify underspecification as a key reason for these failures. An ML pipeline is underspecified when it can return many predictors with eq…

Measuring and Reducing Gendered Correlations in Pre-trained Models Open

Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler , et al. · 2020

Pre-trained models have revolutionized natural language understanding. However, researchers have found they can encode artifacts undesired in many applications, such as professions correlating with one gender more than another. We explore …

Type B Reflexivization as an Unambiguous Testbed for Multilingual\n Multi-Task Gender Bias Open

Ana Valeria González, Maria Barrett, Peter Juel Henrichsen, Kellie Webster, Anders Søgaard · 2020

The one-sided focus on English in previous studies of gender bias in NLP\nmisses out on opportunities in other languages: English challenge datasets such\nas GAP and WinoGender highlight model preferences that are "hallucinatory",\ne.g., d…

Scalable Cross Lingual Pivots to Model Pronoun Gender for Translation Open

Kellie Webster, Emily Pitler · 2020

Machine translation systems with inadequate document understanding can make errors when translating dropped or neutral pronouns into languages with gendered pronouns (e.g., English). Predicting the underlying gender of these pronouns is di…

Social Biases in NLP Models as Barriers for Persons with Disabilities Open

Ben Hutchinson, Vinodkumar Prabhakaran, Emily Denton, Kellie Webster, Yu Zhong , et al. · 2020

Building equitable and inclusive NLP technologies demands consideration of whether and how social attitudes are represented in ML models. In particular, representations encoded in models often inadvertently perpetuate undesirable social bi…

Automatically Identifying Gender Issues in Machine Translation using\n Perturbations Open

Hila Gonen, Kellie Webster · 2020

The successful application of neural methods to machine translation has\nrealized huge quality advances for the community. With these improvements, many\nhave noted outstanding challenges, including the modeling and treatment of\ngendered …

Type B Reflexivization as an Unambiguous Testbed for Multilingual Multi-Task Gender Bias Open

Ana Valeria González, Maria Barrett, Peter Juel Henrichsen, Kellie Webster, Anders Søgaard · 2020

The one-sided focus on English in previous studies of gender bias in NLP misses out on opportunities in other languages: English challenge datasets such as GAP and WinoGender highlight model preferences that are "hallucinatory", e.g., disa…

Automatically Identifying Gender Issues in Machine Translation using Perturbations Open

Hila Gonen, Kellie Webster · 2020

The successful application of neural methods to machine translation has realized huge quality advances for the community. With these improvements, many have noted outstanding challenges, including the modeling and treatment of gendered lan…

Social Biases in NLP Models as Barriers for Persons with Disabilities Open

Ben Hutchinson, Vinodkumar Prabhakaran, Emily Denton, Kellie Webster, Yu Zhong , et al. · 2020

Building equitable and inclusive NLP technologies demands consideration of whether and how social attitudes are represented in ML models. In particular, representations encoded in models often inadvertently perpetuate undesirable social bi…

Gendered Ambiguous Pronoun (GAP) Shared Task at the Gender Bias in NLP Workshop 2019 Open

Kellie Webster, Marta R. Costa‐jussà, Christian Hardmeier, Will Radford · 2019

The 1st ACL workshop on Gender Bias in Natural Language Processing included a shared task on gendered ambiguous pronoun (GAP) resolution. This task was based on the coreference challenge defined in Webster et al. (2018), designed to benchm…

Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns Open

Kellie Webster, Marta Vilar Recasens, Vera Axelrod, Jason Baldridge · 2018

Coreference resolution is an important task for natural language understanding, and the resolution of ambiguous pronouns a longstanding challenge. Nonetheless, existing corpora do not capture ambiguous pronouns in sufficient volume or dive…

Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns Open

Kellie Webster, Marta Vilar Recasens, Vera Axelrod, Jason Baldridge · 2018

Coreference resolution is an important task for natural language understanding, and the resolution of ambiguous pronouns a longstanding challenge. Nonetheless, existing corpora do not capture ambiguous pronouns in sufficient volume or dive…

A Challenge Set and Methods for Noun-Verb Ambiguity Open

Ali Elkahky, Kellie Webster, Daniel Andor, Emily Pitler · 2018

English part-of-speech taggers regularly make egregious errors related to noun-verb ambiguity, despite having achieved 97%+ accuracy on the WSJ Penn Treebank since 2002. These mistakes have been difficult to quantify and make taggers less …

Improved Coreference Resolution Using Cognitive Insights Open

Kellie Webster · 2016

Coreference resolution is the task of extracting referential expressions, or mentions, in text and clustering these by the entity or concept they refer to. The sustained research interest in the task reflects the richness of reference expr…

Using mention accessibility to improve coreference resolution Open

Kellie Webster, Joel Nothman · 2016

Modern coreference resolution systems require linguistic and general knowledge typically sourced from costly, manually curated resources.Despite their intuitive appeal, results have been mixed.In this work, we instead implement fine-graine…

Kellie Webster YOU? Author Swipe