Dan Iter
YOU?
Author Swipe
View article: Generative Enrichment via NFT and Synthesis (GENESIS): A Multi-Perspective AI-Driven Protocol for Secure Medical Analysis
Generative Enrichment via NFT and Synthesis (GENESIS): A Multi-Perspective AI-Driven Protocol for Secure Medical Analysis Open
Knowledge-intensive tasks, such as open-domain question answering (QA), require access to a large amount of world or domain knowledge. A common approach for knowledge-intensive tasks is to employ a retrieve-then-read pipeline that first re…
View article: Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone Open
We introduce phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3.5…
View article: The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions
The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions Open
Recent progress in Large Language Models (LLMs) has produced models that exhibit remarkable performance across a variety of NLP tasks. However, it remains unclear whether the existing focus of NLP research accurately captures the genuine r…
View article: Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models
Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models Open
Large language models (LLMs) can perform a wide range of tasks by following natural language instructions, without the necessity of task-specific fine-tuning. Unfortunately, the performance of LLMs is greatly influenced by the quality of t…
View article: In-Context Demonstration Selection with Cross Entropy Difference
In-Context Demonstration Selection with Cross Entropy Difference Open
Large language models (LLMs) can use in-context demonstrations to improve performance on zero-shot tasks. However, selecting the best in-context examples is challenging because model performance can vary widely depending on the selected ex…
View article: LMGQS: A Large-scale Dataset for Query-focused Summarization
LMGQS: A Large-scale Dataset for Query-focused Summarization Open
Query-focused summarization (QFS) aims to extract or generate a summary of an input document that directly answers or is relevant to a given query. The lack of large-scale datasets in the form of documents, queries, and summaries has hinde…
View article: InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT
InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT Open
While large models such as GPT-3 demonstrate exceptional performance in zeroshot and fewshot summarization tasks, their extensive serving and fine-tuning costs hinder their utilization in various applications. Conversely, previous studies …
View article: Automatic Prompt Optimization with "Gradient Descent" and Beam Search
Automatic Prompt Optimization with "Gradient Descent" and Beam Search Open
Large Language Models (LLMs) have shown impressive performance as general purpose agents, but their abilities remain highly dependent on prompts which are hand written with onerous trial-and-error effort. We propose a simple and nonparamet…
View article: G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment Open
The quality of texts generated by natural language generation (NLG) systems is hard to measure automatically. Conventional reference-based metrics, such as BLEU and ROUGE, have been shown to have relatively low correlation with human judgm…
View article: How Does In-Context Learning Help Prompt Tuning?
How Does In-Context Learning Help Prompt Tuning? Open
Fine-tuning large language models is becoming ever more impractical due to their rapidly-growing scale. This motivates the use of parameter-efficient adaptation methods such as prompt tuning (PT), which adds a small number of tunable embed…
View article: Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models
Auto-Instruct: Automatic Instruction Generation and Ranking for Black-Box Language Models Open
Large language models (LLMs) can perform a wide range of tasks by following natural language instructions, without the necessity of task-specific fine-tuning. Unfortunately, the performance of LLMs is greatly influenced by the quality of t…
View article: The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions
The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions Open
Siru Ouyang, Shuohang Wang, Yang Liu, Ming Zhong, Yizhu Jiao, Dan Iter, Reid Pryzant, Chenguang Zhu, Heng Ji, Jiawei Han. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.
View article: G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment
G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment Open
The quality of texts generated by natural language generation (NLG) systems is hard to measure automatically. Conventional reference-based metrics, such as BLEU and ROUGE, have been shown to have relatively low correlation with human judgm…
View article: InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT
InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT Open
While large models such as GPT-3 demonstrate exceptional performance in zeroshot and fewshot summarization tasks, their extensive serving and fine-tuning costs hinder their utilization in various applications. Conversely, previous studies …
View article: LMGQS: A Large-scale Dataset for Query-focused Summarization
LMGQS: A Large-scale Dataset for Query-focused Summarization Open
Query-focused summarization (QFS) aims to extract or generate a summary of an input document that directly answers or is relevant to a given query. The lack of large-scale datasets in the form of documents, queries, and summaries has hinde…
View article: In-Context Demonstration Selection with Cross Entropy Difference
In-Context Demonstration Selection with Cross Entropy Difference Open
Large language models (LLMs) can use in-context demonstrations to improve performance on zero-shot tasks. However, selecting the best in-context examples is challenging because model performance can vary widely depending on the selected ex…
View article: Automatic Prompt Optimization with “Gradient Descent” and Beam Search
Automatic Prompt Optimization with “Gradient Descent” and Beam Search Open
Large Language Models (LLMs) have shown impressive performance as general purpose agents, but their abilities remain highly dependent on prompts which are hand written with onerous trial-and-error effort. We propose a simple and nonparamet…
View article: The Trade-offs of Domain Adaptation for Neural Language Models
The Trade-offs of Domain Adaptation for Neural Language Models Open
This work connects language model adaptation with concepts of machine learning theory. We consider a training setup with a large out-of-domain set and a small in-domain set. We derive how the benefit of training a model on either set depen…
View article: Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference
Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference Open
Performing event and entity coreference resolution across documents vastly increases the number of candidate mentions, making it intractable to do the full $n^2$ pairwise comparisons. Existing approaches simplify by considering coreference…
View article: The Trade-offs of Domain Adaptation for Neural Language Models
The Trade-offs of Domain Adaptation for Neural Language Models Open
This work connects language model adaptation with concepts of machine learning theory. We consider a training setup with a large out-of-domain set and a small in-domain set. We derive how the benefit of training a model on either set depen…
View article: On the Complementarity of Data Selection and Fine Tuning for Domain Adaptation
On the Complementarity of Data Selection and Fine Tuning for Domain Adaptation Open
Domain adaptation of neural networks commonly relies on three training phases: pretraining, selected data training and then fine tuning. Data selection improves target domain generalization by training further on pretraining data identifie…
View article: Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference
Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference Open
Performing event and entity coreference resolution across documents vastly increases the number of candidate mentions, making it intractable to do the full $n^2$ pairwise comparisons. Existing approaches simplify by considering coreference…
View article: Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models
Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models Open
Recent models for unsupervised representation learning of text have employed a number of techniques to improve contextual word representations but have put little focus on discourse-level representations. We propose CONPONO, an inter-sente…
View article: Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models
Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models Open
Recent models for unsupervised representation learning of text have employed a number of techniques to improve contextual word representations but have put little focus on discourse-level representations. We propose Conpono, an inter-sente…
View article: Entity Attribute Relation Extraction with Attribute-Aware Embeddings
Entity Attribute Relation Extraction with Attribute-Aware Embeddings Open
Entity-attribute relations are a fundamental component for building large-scale knowledge bases, which are widely employed in modern search engines. However, most such knowledge bases are manually curated, covering only a small fraction of…
View article: Automatic Detection of Incoherent Speech for Diagnosing Schizophrenia
Automatic Detection of Incoherent Speech for Diagnosing Schizophrenia Open
Schizophrenia is a mental disorder which afflicts an estimated 0.7% of adults world wide. It affects many areas of mental function, often evident from incoherent speech. Diagnosing schizophrenia relies on subjective judgments resulting in …
View article: FrameIt: Ontology Discovery for Noisy User-Generated Text
FrameIt: Ontology Discovery for Noisy User-Generated Text Open
A common need of NLP applications is to extract structured data from text corpora in order to perform analytics or trigger an appropriate action. The ontology defining the structure is typically application dependent and in many cases it i…
View article: Socratic Learning: Augmenting Generative Models to Incorporate Latent Subsets in Training Data
Socratic Learning: Augmenting Generative Models to Incorporate Latent Subsets in Training Data Open
A challenge in training discriminative models like neural networks is obtaining enough labeled training data. Recent approaches use generative models to combine weak supervision sources, like user-defined heuristics or knowledge bases, to …
View article: Socratic Learning: Correcting Misspecified Generative Models using Discriminative Models
Socratic Learning: Correcting Misspecified Generative Models using Discriminative Models Open
A challenge in training discriminative models like neural networks is obtaining enough labeled training data. Recent approaches use generative models to combine weak supervision sources, like user-defined heuristics or knowledge bases, to …