David Esiobu
YOU?
Author Swipe
View article: Lost in Inference: Rediscovering the Role of Natural Language Inference for Large Language Models
Lost in Inference: Rediscovering the Role of Natural Language Inference for Large Language Models Open
In the recent past, a popular way of evaluating natural language understanding (NLU), was to consider a model's ability to perform natural language inference (NLI) tasks. In this paper, we investigate if NLI tasks, that are rarely used for…
View article: Evaluation data contamination in LLMs: how do we measure it and (when) does it matter?
Evaluation data contamination in LLMs: how do we measure it and (when) does it matter? Open
Hampering the interpretation of benchmark scores, evaluation data contamination has become a growing concern in the evaluation of LLMs, and an active area of research studies its effects. While evaluation data contamination is easily under…
View article: ROBBIE: Robust Bias Evaluation of Large Generative Language Models
ROBBIE: Robust Bias Evaluation of Large Generative Language Models Open
As generative large language models (LLMs) grow more performant and prevalent, we must develop comprehensive enough tools to measure and improve their fairness. Different prompt-based datasets can be used to measure social bias across mult…
View article: Llama 2: Open Foundation and Fine-Tuned Chat Models
Llama 2: Open Foundation and Fine-Tuned Chat Models Open
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dial…
View article: A Theory on Adam Instability in Large-Scale Machine Learning
A Theory on Adam Instability in Large-Scale Machine Learning Open
We present a theory for the previously unexplained divergent behavior noticed in the training of large language models. We argue that the phenomenon is an artifact of the dominant optimization algorithm used for training, called Adam. We o…
View article: ROBBIE: Robust Bias Evaluation of Large Generative Language Models
ROBBIE: Robust Bias Evaluation of Large Generative Language Models Open
David Esiobu, Xiaoqing Tan, Saghar Hosseini, Megan Ung, Yuchen Zhang, Jude Fernandes, Jane Dwivedi-Yu, Eleonora Presani, Adina Williams, Eric Smith. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 20…