Jaime Carbonell
YOU?
Author Swipe
View article: Xlnet: Generalized Autoregressive Pretraining for Language Understanding
Xlnet: Generalized Autoregressive Pretraining for Language Understanding Open
With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregressive language modeling. However, relying on corrupting th…
View article: CAWET: Context-Aware Worst-Case Execution Time Estimation Using Transformers
CAWET: Context-Aware Worst-Case Execution Time Estimation Using Transformers Open
This paper presents CAWET, a hybrid worst-case program timing estimation technique. CAWET identifies the longest execution path using static techniques, whereas the worst-case execution time (WCET) of basic blocks is predicted using an adv…
View article: Language Technologies for Humanitarian Aid
Language Technologies for Humanitarian Aid Open
Humanitarian aid missions, whether emergency famine relief, establishment of medical clinics, or missions in conjunction with peace-keeping operations, require on-demand communication with the indigenous population. If such operations take…
View article: Document Representation and Query Expansion Models for Blog Recommendation
Document Representation and Query Expansion Models for Blog Recommendation Open
We explore several different document representation models and two query expansion models for the task of recommending blogs to a user in response to a query. Blog relevance ranking differs from traditional document ranking in ad-hocinfor…
View article: StructSum: Summarization via Structured Representations
StructSum: Summarization via Structured Representations Open
ive text summarization aims at compressing the information of a long source document into a rephrased, condensed summary. Despite advances in modeling techniques, abstractive summarization models still suffer from several key challenges: (…
View article: Efficient Meta Lifelong-Learning with Limited Memory
Efficient Meta Lifelong-Learning with Limited Memory Open
Current natural language processing models work well on a single task, yet they often fail to continuously learn new tasks without forgetting previous ones as they are re-trained throughout their lifetime, a challenge known as lifelong lea…
View article: Harnessing Code Switching to Transcend the Linguistic Barrier
Harnessing Code Switching to Transcend the Linguistic Barrier Open
Code mixing (or code switching) is a common phenomenon observed in social-media content generated by a linguistically diverse user-base. Studies show that in the Indian sub-continent, a substantial fraction of social media posts exhibit co…
View article: Voice for the Voiceless: Active Sampling to Detect Comments Supporting the Rohingyas
Voice for the Voiceless: Active Sampling to Detect Comments Supporting the Rohingyas Open
The Rohingya refugee crisis is one of the biggest humanitarian crises of modern times with more than 700,000 Rohingyas rendered homeless according to the United Nations High Commissioner for Refugees. While it has received sustained press …
View article: Improving Candidate Generation for Low-resource Cross-lingual Entity Linking
Improving Candidate Generation for Low-resource Cross-lingual Entity Linking Open
Cross-lingual entity linking (XEL) is the task of finding referents in a target-language knowledge base (KB) for mentions extracted from source-language texts. The first step of (X)EL is candidate generation, which retrieves a list of plau…
View article: StructSum: Incorporating Latent and Explicit Sentence Dependencies for Single Document Summarization.
StructSum: Incorporating Latent and Explicit Sentence Dependencies for Single Document Summarization. Open
Traditional preneural approaches to single document summarization relied on modeling the intermediate structure of a document before generating the summary. In contrast, the current state of the art neural summarization models do not prese…
View article: Efficient Meta Lifelong-Learning with Limited Memory
Efficient Meta Lifelong-Learning with Limited Memory Open
Current natural language processing models work well on a single task, yet they often fail to continuously learn new tasks without forgetting previous ones as they are re-trained throughout their lifetime, a challenge known as lifelong lea…
View article: Soft Gazetteers for Low-Resource Named Entity Recognition
Soft Gazetteers for Low-Resource Named Entity Recognition Open
Traditional named entity recognition models use gazetteers (lists of entities) as features to improve performance. Although modern neural network models do not require such hand-crafted features for strong performance, recent work has demo…
View article: Mining Insights from Large-Scale Corpora Using Fine-Tuned Language Models
Mining Insights from Large-Scale Corpora Using Fine-Tuned Language Models Open
Mining insights from large volume of social media texts with minimal supervision is a highly challenging Natural Language Processing (NLP) task. While Language Models' (LMs) efficacy in several downstream tasks is well-studied, assessing t…
View article: Hope Speech Detection: A Computational Analysis of the Voice of Peace
Hope Speech Detection: A Computational Analysis of the Voice of Peace Open
The recent Pulwama terror attack (February 14, 2019, Pulwama, Kashmir) triggered a chain of escalating events between India and Pakistan adding another episode to their 70-year-old dispute over Kashmir. The present era of ubiquitious socia…
View article: Optimizing Data Usage via Differentiable Rewards
Optimizing Data Usage via Differentiable Rewards Open
To acquire a new skill, humans learn better and faster if a tutor, based on their current knowledge level, informs them of how much attention they should pay to particular content or practice problems. Similarly, a machine learning model c…
View article: Cross-lingual Alignment vs Joint Training: A Comparative Study and A Simple Unified Framework
Cross-lingual Alignment vs Joint Training: A Comparative Study and A Simple Unified Framework Open
Learning multilingual representations of text has proven a successful method for many cross-lingual transfer learning tasks. There are two main paradigms for learning such representations: (1) alignment, which maps different independently …
View article: Voice for the Voiceless: Active Sampling to Detect Comments Supporting\n the Rohingyas
Voice for the Voiceless: Active Sampling to Detect Comments Supporting\n the Rohingyas Open
The Rohingya refugee crisis is one of the biggest humanitarian crises of\nmodern times with more than 600,000 Rohingyas rendered homeless according to\nthe United Nations High Commissioner for Refugees. While it has received\nsustained pre…
View article: Learning Rhyming Constraints using Structured Adversaries
Learning Rhyming Constraints using Structured Adversaries Open
Existing recurrent neural language models often fail to capture higher-level structure present in text: for example, rhyming patterns present in poetry. Much prior work on poetry generation uses manually defined constraints which are satis…
View article: CMU-01 at the SIGMORPHON 2019 Shared Task on Crosslinguality and Context\n in Morphology
CMU-01 at the SIGMORPHON 2019 Shared Task on Crosslinguality and Context\n in Morphology Open
This paper presents the submission by the CMU-01 team to the SIGMORPHON 2019\ntask 2 of Morphological Analysis and Lemmatization in Context. This task\nrequires us to produce the lemma and morpho-syntactic description of each token\nin a s…
View article: Gradient-Based Inference for Networks with Output Constraints
Gradient-Based Inference for Networks with Output Constraints Open
Practitioners apply neural networks to increasingly complex problems in natural language processing, such as syntactic parsing and semantic role labeling that have rich output structures. Many such structured-prediction problems require de…
View article: Domain Adaptation of Neural Machine Translation by Lexicon Induction
Domain Adaptation of Neural Machine Translation by Lexicon Induction Open
It has been previously noted that neural machine translation (NMT) is very sensitive to domain shift. In this paper, we argue that this is a dual effect of the highly lexicalized nature of NMT, resulting in failure for sentences with large…
View article: Data-Driven Approach to Multiple-Source Domain Adaptation.
Data-Driven Approach to Multiple-Source Domain Adaptation. Open
A key problem in domain adaptation is determining what to transfer across different domains. We propose a data-driven method to represent these changes across multiple source domains and perform unsupervised domain adaptation. We assume th…
View article: Low-Dimensional Density Ratio Estimation for Covariate Shift Correction.
Low-Dimensional Density Ratio Estimation for Covariate Shift Correction. Open
Covariate shift is a prevalent setting for supervised learning in the wild when the training and test data are drawn from different time periods, different but related domains, or via different sampling strategies. This paper addresses a t…
View article: Learning Rhyming Constraints using Structured Adversaries
Learning Rhyming Constraints using Structured Adversaries Open
Harsh Jhamtani, Sanket Vaibhav Mehta, Jaime Carbonell, Taylor Berg-Kirkpatrick. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processin…
View article: CMU-01 at the SIGMORPHON 2019 Shared Task on Crosslinguality and Context in Morphology
CMU-01 at the SIGMORPHON 2019 Shared Task on Crosslinguality and Context in Morphology Open
This paper presents the submission by the CMU-01 team to the SIGMORPHON 2019 task 2 of Morphological Analysis and Lemmatization in Context. This task requires us to produce the lemma and morpho-syntactic description of each token in a sequ…
View article: A Little Annotation does a Lot of Good: A Study in Bootstrapping Low-resource Named Entity Recognizers
A Little Annotation does a Lot of Good: A Study in Bootstrapping Low-resource Named Entity Recognizers Open
Most state-of-the-art models for named entity recognition (NER) rely on the availability of large amounts of labeled data, making them challenging to extend to new, lower-resourced languages. However, there are now several proposed approac…
View article: Domain Adaptation of Neural Machine Translation by Lexicon Induction
Domain Adaptation of Neural Machine Translation by Lexicon Induction Open
It has been previously noted that neural machine translation (NMT) is very sensitive to domain shift. In this paper, we argue that this is a dual effect of the highly lexicalized nature of NMT, resulting in failure for sentences with large…
View article: Transformer-XL: Attentive Language Models beyond a Fixed-Length Context
Transformer-XL: Attentive Language Models beyond a Fixed-Length Context Open
Transformers have a potential of learning longer-term dependency, but are limited by a fixed-length context in the setting of language modeling. We propose a novel neural architecture Transformer-XL that enables learning dependency beyond …