Benjamin Roth
YOU?
Author Swipe
View article: One and a Half Year of ChatGPT: An Umbrella Review of Large Language Model (LLM) Perception Across Time and Research Fields
One and a Half Year of ChatGPT: An Umbrella Review of Large Language Model (LLM) Perception Across Time and Research Fields Open
Large language models (LLMs) like ChatGPT have emerged as transformative tools across various research fields. To illustrate the development of perceived benefits and concerns of LLMs across research fields, this umbrella review synthesize…
View article: Agree, Disagree, Explain: Decomposing Human Label Variation in NLI through the Lens of Explanations
Agree, Disagree, Explain: Decomposing Human Label Variation in NLI through the Lens of Explanations Open
Natural Language Inference datasets often exhibit human label variation. To better understand these variations, explanation-based approaches analyze the underlying reasoning behind annotators' decisions. One such approach is the LiTEx taxo…
View article: Exploring prompts to elicit memorization in masked language model-based named entity recognition
Exploring prompts to elicit memorization in masked language model-based named entity recognition Open
The possibility of identifying specific information about the training data a language model memorized poses a privacy risk. In this study, we analyze the ability of prompts to detect training data memorization in six masked language model…
View article: The Impact of Graph Structure, Cluster Centroid and Text Review Embeddings on Recommendation Methods
The Impact of Graph Structure, Cluster Centroid and Text Review Embeddings on Recommendation Methods Open
It is generally accepted that collaborative information is important for the performance of recommender systems. It is also generally accepted that if this information is sparser, it impacts recommendation systems negatively. Various appro…
View article: Helpful assistant or fruitful facilitator? Investigating how personas affect language model behavior
Helpful assistant or fruitful facilitator? Investigating how personas affect language model behavior Open
One way to steer generations from large language models (LLM) is to assign a persona: a role that describes how the user expects the LLM to behave (e.g., a helpful assistant, a teacher, a woman). This paper investigates how personas affect…
View article: Streamlining and Accelerating the Molecular Tumor Board Process at the University Medical Center Hamburg-Eppendorf
Streamlining and Accelerating the Molecular Tumor Board Process at the University Medical Center Hamburg-Eppendorf Open
As the first open-source and extendable solution for standardized MTB documentation, MONOCLE enables wider adoption by other medical centers.
View article: Knowledge Connector: Decision support system for multiomics-based precision oncology
Knowledge Connector: Decision support system for multiomics-based precision oncology Open
Precision cancer medicine aims to improve patient outcomes by providing individually tailored recommendations for clinical management based on the evaluation of biological disease profiles in multidisciplinary molecular tumor boards (MTBs)…
View article: Influences on LLM Calibration: A Study of Response Agreement, Loss Functions, and Prompt Styles
Influences on LLM Calibration: A Study of Response Agreement, Loss Functions, and Prompt Styles Open
Calibration, the alignment between model confidence and prediction accuracy, is critical for the reliable deployment of large language models (LLMs). Existing works neglect to measure the generalization of their methods to other prompt sty…
View article: Influences on LLM Calibration: A Study of Response Agreement, Loss Functions, and Prompt Styles
Influences on LLM Calibration: A Study of Response Agreement, Loss Functions, and Prompt Styles Open
View article: Influence-driven Curriculum Learning for Pre-training on Limited Data
Influence-driven Curriculum Learning for Pre-training on Limited Data Open
View article: Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance
Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance Open
View article: RecombiText: Compositional Data Augmentation for Enhancing LLM Pre-Training Datasets in Low-Resource Scenarios
RecombiText: Compositional Data Augmentation for Enhancing LLM Pre-Training Datasets in Low-Resource Scenarios Open
View article: Specification overfitting in artificial intelligence
Specification overfitting in artificial intelligence Open
Machine learning (ML) and artificial intelligence (AI) approaches are often criticized for their inherent bias and for their lack of control, accountability, and transparency. Consequently, regulatory bodies struggle with containing this t…
View article: From Calculation to Adjudication: Examining LLM judges on Mathematical Reasoning Tasks
From Calculation to Adjudication: Examining LLM judges on Mathematical Reasoning Tasks Open
To reduce the need for human annotations, large language models (LLMs) have been proposed as judges of the quality of other candidate models. The performance of LLM judges is typically evaluated by measuring the correlation with human judg…
View article: An Evaluation of Explanation Methods for Black-Box Detectors of Machine-Generated Text
An Evaluation of Explanation Methods for Black-Box Detectors of Machine-Generated Text Open
The increasing difficulty to distinguish language-model-generated from human-written text has led to the development of detectors of machine-generated text (MGT). However, in many contexts, a black-box prediction is not sufficient, it is e…
View article: To Know or Not To Know? Analyzing Self-Consistency of Large Language Models under Ambiguity
To Know or Not To Know? Analyzing Self-Consistency of Large Language Models under Ambiguity Open
One of the major aspects contributing to the striking performance of large language models (LLMs) is the vast amount of factual knowledge accumulated during pre-training. Yet, many LLMs suffer from self-inconsistency, which raises doubts a…
View article: Black-box Model Ensembling for Textual and Visual Question Answering via Information Fusion
Black-box Model Ensembling for Textual and Visual Question Answering via Information Fusion Open
A diverse range of large language models (LLMs), e.g., ChatGPT, and visual question answering (VQA) models, e.g., BLIP, have been developed for solving textual and visual question answering tasks. However, fine-tuning these models is eithe…
View article: Helpful assistant or fruitful facilitator? Investigating how personas affect language model behavior
Helpful assistant or fruitful facilitator? Investigating how personas affect language model behavior Open
One way to personalize and steer generations from large language models (LLM) is to assign a persona: a role that describes how the user expects the LLM to behave (e.g., a helpful assistant, a teacher, a woman). This paper investigates how…
View article: Analysing zero-shot temporal relation extraction on clinical notes using temporal consistency
Analysing zero-shot temporal relation extraction on clinical notes using temporal consistency Open
This paper presents the first study for temporal relation extraction in a zero-shot setting focusing on biomedical text. We employ two types of prompts and five LLMs (GPT-3.5, Mixtral, Llama 2, Gemma, and PMC-LLaMA) to obtain responses abo…
View article: Text-Guided Alternative Image Clustering
Text-Guided Alternative Image Clustering Open
Traditional image clustering techniques only find a single grouping within visual data. In particular, they do not provide a possibility to explicitly define multiple types of clustering. This work explores the potential of large vision-la…
View article: The Impact of Cluster Centroid and Text Review Embeddings on Recommendation Methods
The Impact of Cluster Centroid and Text Review Embeddings on Recommendation Methods Open
Recommendation systems often neglect global patterns that can be provided by clusters of similar items or even additional information such as text. Therefore, we study the impact of integrating clustering embeddings, review embeddings, and…
View article: Exploring prompts to elicit memorization in masked language model-based named entity recognition
Exploring prompts to elicit memorization in masked language model-based named entity recognition Open
Training data memorization in language models impacts model capability (generalization) and safety (privacy risk). This paper focuses on analyzing prompts' impact on detecting the memorization of 6 masked language model-based named entity …
View article: Specification Overfitting in Artificial Intelligence
Specification Overfitting in Artificial Intelligence Open
Machine learning (ML) and artificial intelligence (AI) approaches are often criticized for their inherent bias and for their lack of control, accountability, and transparency. Consequently, regulatory bodies struggle with containing this t…
View article: Specification Overfitting in Artificial Intelligence
Specification Overfitting in Artificial Intelligence Open
Machine learning (ML) and artificial intelligence (AI) approaches are often criticized for their inherent bias and for their lack of control, accountability, and transparency. Consequently, regulatory bodies struggle with containing this t…
View article: Counterfactual Reasoning with Knowledge Graph Embeddings
Counterfactual Reasoning with Knowledge Graph Embeddings Open
Knowledge graph embeddings (KGEs) were originally developed to infer true but missing facts in incomplete knowledge repositories. In this paper, we link knowledge graph completion and counterfactual reasoning via our new task CFKGR. We mod…
View article: Text-Guided Image Clustering
Text-Guided Image Clustering Open
Image clustering divides a collection of images into meaningful groups, typically interpreted post-hoc via human-given annotations. Those are usually in the form of text, begging the question of using text as an abstraction for image clust…
View article: Linking Danish Parser Output to a Central Word Repository:From Morphosemantic Disambiguation to Unique Identifiers
Linking Danish Parser Output to a Central Word Repository:From Morphosemantic Disambiguation to Unique Identifiers Open
View article: Counterfactual Reasoning with Knowledge Graph Embeddings
Counterfactual Reasoning with Knowledge Graph Embeddings Open
View article: Text-Guided Image Clustering
Text-Guided Image Clustering Open
View article: Functionality learning through specification instructions
Functionality learning through specification instructions Open
Test suites assess natural language processing models' performance on specific functionalities: cases of interest involving model robustness, fairness, or particular linguistic capabilities. This paper introduces specification instructions…