Leif Azzopardi
YOU?
Author Swipe
View article: All Claims Are Equal, but Some Claims Are More Equal Than Others: Importance-Sensitive Factuality Evaluation of LLM Generations
All Claims Are Equal, but Some Claims Are More Equal Than Others: Importance-Sensitive Factuality Evaluation of LLM Generations Open
Existing methods for evaluating the factuality of large language model (LLM) responses treat all claims as equally important. This results in misleading evaluations when vital information is missing or incorrect as it receives the same wei…
View article: POW: Political Overton Windows of Large Language Models
POW: Political Overton Windows of Large Language Models Open
Political bias in Large Language Models (LLMs) presents a growing concern for the responsible deployment of AI systems. Traditional audits often attempt to locate a model's political position as a point estimate, masking the broader set of…
View article: Improving the Reusability of Conversational Search Test Collections
Improving the Reusability of Conversational Search Test Collections Open
Incomplete relevance judgments limit the reusability of test collections. When new systems are compared to previous systems that contributed to the pool, they often face a disadvantage. This is due to pockets of unjudged documents (called …
View article: Conversational Gold: Evaluating Personalized Conversational Search System using Gold Nuggets
Conversational Gold: Evaluating Personalized Conversational Search System using Gold Nuggets Open
The rise of personalized conversational search systems has been driven by advancements in Large Language Models (LLMs), enabling these systems to retrieve and generate answers for complex information needs. However, the automatic evaluatio…
View article: PRISM: A Methodology for Auditing Biases in Large Language Models
PRISM: A Methodology for Auditing Biases in Large Language Models Open
Auditing Large Language Models (LLMs) to discover their biases and preferences is an emerging challenge in creating Responsible Artificial Intelligence (AI). While various methods have been proposed to elicit the preferences of such models…
View article: Evaluation of Attribution Bias in Generator-Aware Retrieval-Augmented Large Language Models
Evaluation of Attribution Bias in Generator-Aware Retrieval-Augmented Large Language Models Open
Attributing answers to source documents is an approach used to enhance the verifiability of a model's output in retrieval augmented generation (RAG). Prior work has mainly focused on improving and evaluating the attribution quality of larg…
View article: TREC iKAT 2023: A Test Collection for Evaluating Conversational and Interactive Knowledge Assistants
TREC iKAT 2023: A Test Collection for Evaluating Conversational and Interactive Knowledge Assistants Open
Conversational information seeking has evolved rapidly in the last few years with the development of Large Language Models (LLMs), providing the basis for interpreting and responding in a naturalistic manner to user requests. The extended …
View article: Report on the Search Futures Workshop at ECIR 2024
Report on the Search Futures Workshop at ECIR 2024 Open
The First Search Futures Workshop, in conjunction with the Fourty-sixth European Conference on Information Retrieval (ECIR) 2024, looked into the future of search to ask questions such as: • How can we harness the power of generative AI to…
View article: Can We Use Large Language Models to Fill Relevance Judgment Holes?
Can We Use Large Language Models to Fill Relevance Judgment Holes? Open
Incomplete relevance judgments limit the re-usability of test collections. When new systems are compared against previous systems used to build the pool of judged documents, they often do so at a disadvantage due to the ``holes'' in test c…
View article: A Conceptual Framework for Conversational Search and Recommendation: Conceptualizing Agent-Human Interactions During the Conversational Search Process
A Conceptual Framework for Conversational Search and Recommendation: Conceptualizing Agent-Human Interactions During the Conversational Search Process Open
The conversational search task aims to enable a user to resolve information needs via natural language dialogue with an agent. In this paper, we aim to develop a conceptual framework of the actions and intents of users and agents explainin…
View article: Measuring the retrievability of digital library content using analytics data
Measuring the retrievability of digital library content using analytics data Open
Digital libraries aim to provide value to users by housing content that is accessible and searchable. Often such access is afforded through external web search engines. In this article, we measure how easily digital library content can be …
View article: Measuring Bias in a Ranked List using Term-based Representations
Measuring Bias in a Ranked List using Term-based Representations Open
In most recent studies, gender bias in document ranking is evaluated with the NFaiRR metric, which measures bias in a ranked list based on an aggregation over the unbiasedness scores of each ranked document. This perspective in measuring t…
View article: Seeking Socially Responsible Consumers: Exploring the Intention-Search-Behaviour Gap
Seeking Socially Responsible Consumers: Exploring the Intention-Search-Behaviour Gap Open
The increasing prominence of "Socially Responsible Consumers"has brought about a heightened focus on the ethical, environmental, social, and ideological dimensions influencing product purchasing decisions. Despite this emphasis, studies ha…
View article: The Influence of Presentation and Performance on User Satisfaction
The Influence of Presentation and Performance on User Satisfaction Open
The effectiveness of an IR system is gauged not just by its ability to retrieve relevant results but also by how it presents these results to users; an engaging presentation often correlates with increased user satisfaction. While existing…
View article: Uncharted Territory: Understanding Exploratory Search Behaviours in Literature Reviews
Uncharted Territory: Understanding Exploratory Search Behaviours in Literature Reviews Open
In the realm of Information Seeking and Retrieval (ISR), searching the literature for relevant references in the context of academic work, such as theses or publications, is widely recognised as an exploratory search task. This task become…
View article: Ranking Heterogeneous Search Result Pages using the Interactive Probability Ranking Principle
Ranking Heterogeneous Search Result Pages using the Interactive Probability Ranking Principle Open
The Probability Ranking Principle (PRP) ranks search results based on their expected utility derived solely from document contents, often overlooking the nuances of presentation and user interaction. However, with the evolution of Search E…
View article: TREC iKAT 2023: The Interactive Knowledge Assistance Track Overview
TREC iKAT 2023: The Interactive Knowledge Assistance Track Overview Open
Conversational Information Seeking has evolved rapidly in the last few years with the development of Large Language Models providing the basis for interpreting and responding in a naturalistic manner to user requests. iKAT emphasizes the c…
View article: Retrievability Bias Estimation Using Synthetically Generated Queries
Retrievability Bias Estimation Using Synthetically Generated Queries Open
Ranking with pre-trained language models (PLMs) has shown to be highly effective for various Information Retrieval tasks. Previous studies investigated the performance of these models in terms of effectiveness and efficiency. However, ther…
View article: What the Dickens: Post-mortem privacy and intergenerational trust
What the Dickens: Post-mortem privacy and intergenerational trust Open
The paper argues that protecting post-mortem privacy is not solely beneficial for the deceased and their relatives but enables intergenerational data-sharing. However, legal approaches alone are unlikely to generate the trust required and …
View article: Revealing Cumulative Risks in Online Personal Information: A Data Narrative Study
Revealing Cumulative Risks in Online Personal Information: A Data Narrative Study Open
When pieces from an individual's personal information available online are connected over time and across multiple platforms, this more complete digital trace can give unintended insights into their life and opinions. In a data narrative i…