Leon Derczynski
YOU?
Author Swipe
NLP Security and Ethics, in the Wild Open
As NLP models are used by a growing number of end-users, an area of increasing importance is NLP Security (NLPSec): assessing the vulnerability of models to malicious attacks and developing comprehensive countermeasures against them. While…
View article: Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models
Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models Open
As inference-time scaling becomes critical for enhanced reasoning capabilities, it is increasingly becoming important to build models that are efficient to infer. We introduce Nemotron-H, a family of 8B and 56B/47B hybrid Mamba-Transformer…
View article: Efficient Methods for Natural Language Processing: A Survey
Efficient Methods for Natural Language Processing: A Survey Open
Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data; however, using only scale to improve performance means that resource consumption also grows. Such resources inc…
Importing Phantoms: Measuring LLM Package Hallucination Vulnerabilities Open
Large Language Models (LLMs) have become an essential tool in the programmer's toolkit, but their tendency to hallucinate code can be used by malicious actors to introduce vulnerabilities to broad swathes of the software supply chain. In t…
View article: Summon a demon and bind it: A grounded theory of LLM red teaming
Summon a demon and bind it: A grounded theory of LLM red teaming Open
Engaging in the deliberate generation of abnormal outputs from Large Language Models (LLMs) by attacking them is a novel human activity. This paper presents a thorough exposition of how and why people perform such attacks, defining LLM red…
NLP Security and Ethics, in the Wild Open
As NLP models are used by a growing number of end-users, an area of increasing importance is NLP Security (NLPSec): assessing the vulnerability of models to malicious attacks and developing comprehensive countermeasures against them. While…
View article: Nemotron-4 340B Technical Report
Nemotron-4 340B Technical Report Open
We release the Nemotron-4 340B model family, including Nemotron-4-340B-Base, Nemotron-4-340B-Instruct, and Nemotron-4-340B-Reward. Our models are open access under the NVIDIA Open Model License Agreement, a permissive model license that al…
garak: A Framework for Security Probing Large Language Models Open
As Large Language Models (LLMs) are deployed and integrated into thousands of applications, the need for scalable evaluation of how models respond to adversarial attacks grows rapidly. However, LLM security is a moving target: models produ…
View article: Introducing v0.5 of the AI Safety Benchmark from MLCommons
Introducing v0.5 of the AI Safety Benchmark from MLCommons Open
This paper introduces v0.5 of the AI Safety Benchmark, which has been created by the MLCommons AI Safety Working Group. The AI Safety Benchmark has been designed to assess the safety risks of AI systems that use chat-tuned language models.…
Summon a Demon and Bind it: A Grounded Theory of LLM Red Teaming Open
Engaging in the deliberate generation of abnormal outputs from Large Language Models (LLMs) by attacking them is a novel human activity. This paper presents a thorough exposition of how and why people perform such attacks, defining LLM red…
View article: Surveying (Dis)Parities and Concerns of Compute Hungry NLP Research
Surveying (Dis)Parities and Concerns of Compute Hungry NLP Research Open
Many recent improvements in NLP stem from the development and use of large pre-trained language models (PLMs) with billions of parameters. Large model sizes makes computational cost one of the main limiting factors for training and evaluat…
View article: Assessing Language Model Deployment with Risk Cards
Assessing Language Model Deployment with Risk Cards Open
This paper introduces RiskCards, a framework for structured assessment and documentation of risks associated with an application of language models. As with all language, text generated by language models can be harmful, or used to bring a…
View article: Efficient Methods for Natural Language Processing: A Survey
Efficient Methods for Natural Language Processing: A Survey Open
Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data; however, using only scale to improve performance means that resource consumption also grows. Such resources inc…
RWKV: Reinventing RNNs for the Transformer Era Open
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Gv, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon,…
Anchoring Fine-tuning of Sentence Transformer with Semantic Label Information for Efficient Truly Few-shot Classification Open
Few-shot classification is a powerful technique, but training requires substantial computing power and data. We propose an efficient method with small model sizes and less training data with only 2-8 training instances per class. Our propo…
TeamAmpa at SemEval-2023 Task 3: Exploring Multilabel and Multilingual RoBERTa Models for Persuasion and Framing Detection Open
This paper describes our submission to the SemEval 2023 Task 3 on two subtasks: detecting persuasion techniques and framing. Both subtasks are multi-label classification problems. We present a set of experiments, exploring how to get robus…
Foreword to NEJLT Volume 8, 2022 Open
An introduction to the Northern European Journal of Language Technology in 2022
View article: Efficient Methods for Natural Language Processing: A Survey
Efficient Methods for Natural Language Processing: A Survey Open
Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data; however, using only scale to improve performance means that resource consumption also grows. Such resources inc…
Training a T5 Using Lab-sized Resources Open
Training large neural language models on large datasets is resource- and time-intensive. These requirements create a barrier to entry, where those with fewer resources cannot build competitive models. This paper presents various techniques…
Sparse Probability of Agreement Open
Measuring inter-annotator agreement is important for annotation tasks, but many metrics require a fully-annotated set of data, where all annotators annotate all samples. We define Sparse Probability of Agreement, SPA, which estimates the p…
Set Interdependence Transformer: Set-to-Sequence Neural Networks for Permutation Learning and Structure Prediction Open
The task of learning to map an input set onto a permuted sequence of its elements is challenging for neural networks. Set-to-sequence problems occur in natural language processing, computer vision and structure prediction, where interactio…
The ITU Faroese Pairs Dataset Open
This article documents a dataset of sentence pairs between Faroese and Danish, produced at ITU Copenhagen. The data covers tranlsation from both source languages, and is intended for use as training data for machine translation systems in …
Set Interdependence Transformer: Set-to-Sequence Neural Networks for Permutation Learning and Structure Prediction Open
The task of learning to map an input set onto a permuted sequence of its elements is challenging for neural networks. Set-to-sequence problems occur in natural language processing, computer vision and structure prediction, where interactio…
Bridging the Domain Gap for Stance Detection for the Zulu language Open
Misinformation has become a major concern in recent last years given its spread across our information sources. In the past years, many NLP tasks have been introduced in this area, with some systems reaching good results on English languag…
Handling and Presenting Harmful Text in NLP Research Open
Text data can pose a risk of harm. However, the risks are not fully understood, and how to handle, present, and discuss harmful text in a safe way remains an unresolved issue in the NLP community. We provide an analytical framework categor…
SHAJ: Albanian hate speech & abusive language Open
SHAJ is an annotated Albanian dataset for hate speech and offensive speech that has been constructed from user-generated content on various social media platforms. Its annotation follows the hierarchical schema introduced in OffensEval.Pap…
DanFEVER: claim verification dataset for Danish Open
DanFEVER is a dataset intended for claim verification in Danish. The dataset builds upon the task framing of the well known (English) FEVER fact extraction and verification challenge. DanFEVER can be used for creating models for detecting …