Alexander Bezzubov
YOU?
Author Swipe
Diff-XYZ: A Benchmark for Evaluating Diff Understanding Open
Reliable handling of code diffs is central to agents that edit and refactor repositories at scale. We introduce Diff-XYZ, a compact benchmark for code-diff understanding with three supervised tasks: apply (old code $+$ diff $\rightarrow$ n…
Challenge on Optimization of Context Collection for Code Completion Open
The rapid advancement of workflows and methods for software engineering using AI emphasizes the need for a systematic evaluation and analysis of their ability to leverage information from entire projects, particularly in large code bases. …
Towards Realistic Evaluation of Commit Message Generation by Matching Online and Offline Settings Open
When a Commit Message Generation (CMG) system is integrated into the IDEs and other products at JetBrains, we perform online evaluation based on user acceptance of the generated messages. However, performing online experiments with every c…
All You Need Is Logs: Improving Code Completion by Learning from Anonymous IDE Usage Logs Open
In this work, we propose an approach for collecting completion usage logs from the users in an IDE and using them to train a machine learning based model for ranking completion candidates. We developed a set of features that describe compl…