Prémkumar Dévanbu
YOU?
Author Swipe
View article: CoDocBench: A Dataset for Code-Documentation Alignment in Software Maintenance
CoDocBench: A Dataset for Code-Documentation Alignment in Software Maintenance Open
One of the central tasks in software maintenance is being able to understand and develop code changes. Thus, given a natural language description of the desired new operation of a function, an agent (human or AI) might be asked to generate…
View article: Calibration and Correctness of Language Models for Code ICSE Artifact
Calibration and Correctness of Language Models for Code ICSE Artifact Open
Machine learning models are widely used, but can also often be wrong. Users would benefit from a reliable indication of whether a given output from a given model should be trusted, so a rational decision can be made whether to use the outp…
View article: Vision Paper: Proof-Carrying Code Completions
Vision Paper: Proof-Carrying Code Completions Open
View article: Can LLMs Replace Manual Annotation of Software Engineering Artifacts?
Can LLMs Replace Manual Annotation of Software Engineering Artifacts? Open
Experimental evaluations of software engineering innovations, e.g., tools and processes, often include human-subject studies as a component of a multi-pronged strategy to obtain greater generalizability of the findings. However, human-subj…
View article: Trojans in Large Language Models of Code: A Critical Review through a Trigger-Based Taxonomy
Trojans in Large Language Models of Code: A Critical Review through a Trigger-Based Taxonomy Open
Large language models (LLMs) have provided a lot of exciting new capabilities in software development. However, the opaque nature of these models makes them difficult to reason about and inspect. Their opacity gives rise to potential secur…
View article: Calibration of Large Language Models on Code Summarization
Calibration of Large Language Models on Code Summarization Open
A brief, fluent, and relevant summary can be helpful during program comprehension; however, such a summary does require significant human effort to produce. Often, good summaries are unavailable in software projects, which makes maintenanc…
View article: Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)
Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization) Open
Large Language Models (LLM) are a new class of computation engines, "programmed" via prompt engineering. Researchers are still learning how to best "program" these LLMs to help developers. We start with the intuition that developers tend t…
View article: RepairAgent: An Autonomous, LLM-Based Agent for Program Repair
RepairAgent: An Autonomous, LLM-Based Agent for Program Repair Open
Automated program repair has emerged as a powerful technique to mitigate the impact of software bugs on system reliability and user experience. This paper introduces RepairAgent, the first work to address the program repair challenge throu…
View article: Robustness, Security, Privacy, Explainability, Efficiency, and Usability of Large Language Models for Code
Robustness, Security, Privacy, Explainability, Efficiency, and Usability of Large Language Models for Code Open
Large language models for code (LLM4Code), which demonstrate strong performance (e.g., high accuracy) in processing source code, have significantly transformed software engineering. Many studies separately investigate the non-functional pr…
View article: Studying LLM Performance on Closed- and Open-source Data
Studying LLM Performance on Closed- and Open-source Data Open
Large Language models (LLMs) are finding wide use in software engineering practice. These models are extremely data-hungry, and are largely trained on open-source (OSS) code distributed with permissive licenses. In terms of actual use howe…
View article: Towards Understanding What Code Language Models Learned
Towards Understanding What Code Language Models Learned Open
Pre-trained language models are effective in a variety of natural language tasks, but it has been argued their capabilities fall short of fully learning meaning or understanding language. To understand the extent to which language models c…
View article: Better patching using LLM prompting, via Self-Consistency
Better patching using LLM prompting, via Self-Consistency Open
Large Language models (LLMs) can be induced to solve non-trivial problems with "few-shot" prompts including illustrative problem-solution examples. Now if the few-shots also include "chain of thought" (CoT) explanations, which are of the f…
View article: A Survey of Trojans in Neural Models of Source Code: Taxonomy and Techniques
A Survey of Trojans in Neural Models of Source Code: Taxonomy and Techniques Open
In this work, we study literature in Explainable AI and Safe AI to understand poisoning of neural models of code. In order to do so, we first establish a novel taxonomy for Trojan AI for code, and present a new aspect-based classification …
View article: AI Safety Subproblems for Software Engineering Researchers
AI Safety Subproblems for Software Engineering Researchers Open
In this 4-page manuscript we discuss the problem of long-term AI Safety from a Software Engineering (SE) research viewpoint. We briefly summarize long-term AI Safety, and the challenge of avoiding harms from AI as systems meet or exceed hu…
View article: Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)
Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization) Open
Large Language Models (LLM) are a new class of computation engines, "programmed" via prompt engineering. We are still learning how to best "program" these LLMs to help developers. We start with the intuition that developers tend to conscio…
View article: Large Language Models and Simple, Stupid Bugs
Large Language Models and Simple, Stupid Bugs Open
With the advent of powerful neural language models, AI-based systems to assist developers in coding tasks are becoming widely available; Copilot is one such system. Copilot uses Codex, a large language model (LLM), to complete code conditi…
View article: Extending Source Code Pre-Trained Language Models to Summarise Decompiled Binaries
Extending Source Code Pre-Trained Language Models to Summarise Decompiled Binaries Open
Reverse engineering binaries is required to understand and analyse programs for which the source code is unavailable. Decompilers can transform the largely unreadable binaries into a more readable source code-like representation. However, …
View article: CAPYBARA: Decompiled Binary Functions and Related Summaries
CAPYBARA: Decompiled Binary Functions and Related Summaries Open
CAPYBARA This dataset is published as part of the paper: "Extending Source Code Pre-Trained Language Models to Summarise Decompiled Binaries". It includes both the training/evaluation data as well as the raw data. The data…
View article: CAPYBARA: Decompiled Binary Functions and Related Summaries
CAPYBARA: Decompiled Binary Functions and Related Summaries Open
CAPYBARA This dataset is published as part of the paper: "Extending Source Code Pre-Trained Language Models to Summarise Decompiled Binaries". It includes both the training/evaluation data as well as the raw data. The data…
View article: FlexType: A Plug-and-Play Framework for Type Inference Models
FlexType: A Plug-and-Play Framework for Type Inference Models Open
Types in TypeScript play an important role in the correct usage of variables and APIs. Type errors such as variable or function misuse can be avoided with explicit type annotations. In this work, we introduce FlexType, an IDE extension tha…
View article: Few-shot training LLMs for project-specific code-summarization
Few-shot training LLMs for project-specific code-summarization Open
Very large language models (LLMs), such as GPT-3 and Codex have achieved state-of-the-art performance on several natural-language tasks, and show great promise also for code. A particularly exciting aspect of LLMs is their knack for few-sh…
View article: Few-shot training LLMs for project-specific code-summarization
Few-shot training LLMs for project-specific code-summarization Open
Very large language models (LLMs), such as GPT-3 and Codex have achieved state-of-the-art performance on several natural-language tasks, and show great promise also for code. A particularly exciting aspect of LLMs is their knack for few-sh…
View article: NatGen: Generative pre-training by "Naturalizing" source code
NatGen: Generative pre-training by "Naturalizing" source code Open
Pre-trained Generative Language models (e.g. PLBART, CodeT5, SPT-Code) for source code yielded strong results on several tasks in the past few years, including code generation and translation. These models have adopted varying pre-training…
View article: Learning code summarization from a small and local dataset
Learning code summarization from a small and local dataset Open
Foundation models (e.g., CodeBERT, GraphCodeBERT, CodeT5) work well for many software engineering tasks. These models are pre-trained (using self-supervision) with billions of code tokens, and then fine-tuned with hundreds of thousands of …
View article: ManyTypes4TypeScript
ManyTypes4TypeScript Open
In this paper, we present ManyTypes4TypeScript, a very large corpus for training and evaluating machine-learning models for sequence-based type inference in TypeScript. The dataset includes over 9 million type annotations, across 13,953 pr…
View article: Multilingual training for software engineering
Multilingual training for software engineering Open
Well-trained machine-learning models, which leverage large amounts of open-source software data, have now become an interesting approach to automating many software engineering tasks. Several SE tasks have all been subject to this approach…
View article: ManyTypes4TypeScript: A Comprehensive TypeScript Dataset for Sequence-Based Type Inference
ManyTypes4TypeScript: A Comprehensive TypeScript Dataset for Sequence-Based Type Inference Open
In this paper, we present ManyTypes4TypeScript, a very large corpus for training and evaluating machine-learning models for sequence-based type inference in TypeScript. The dataset includes over 9 million type annotations, across 13,953 pr…
View article: ManyTypes4TypeScript: A Comprehensive TypeScript Dataset for Sequence-Based Type Inference
ManyTypes4TypeScript: A Comprehensive TypeScript Dataset for Sequence-Based Type Inference Open
In this paper, we present ManyTypes4TypeScript, a very large corpus for training and evaluating machine-learning models for sequence-based type inference in TypeScript. The dataset includes over 9 million type annotations, across 13,953 pr…
View article: ManyTypes4TypeScript: A Comprehensive TypeScript Dataset for Sequence-Based Type Inference
ManyTypes4TypeScript: A Comprehensive TypeScript Dataset for Sequence-Based Type Inference Open
In this paper, we present ManyTypes4TypeScript, a very large corpus for training and evaluating machine-learning models for sequence-based type inference in TypeScript. The dataset includes over 9 million type annotations, across 13,953 pr…
View article: Learning to Find Usages of Library Functions in Optimized Binaries
Learning to Find Usages of Library Functions in Optimized Binaries Open
Much software, whether beneficent or malevolent, is distributed only as\nbinaries, sans source code. Absent source code, understanding binaries'\nbehavior can be quite challenging, especially when compiled under higher levels\nof compiler …