James Diffenderfer
YOU?
Author Swipe
View article: BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models
BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models Open
Data-driven molecular discovery leverages artificial intelligence/machine learning (AI/ML) and generative modeling to filter and design novel molecules. Discovering novel molecules requires accurate out-of-distribution (OOD) predictions, b…
View article: LLM Unlearning Reveals a Stronger-Than-Expected Coreset Effect in Current Benchmarks
LLM Unlearning Reveals a Stronger-Than-Expected Coreset Effect in Current Benchmarks Open
Large language model unlearning has become a critical challenge in ensuring safety and controlled model behavior by removing undesired data-model influences from the pretrained model while preserving general utility. Significant recent eff…
View article: TruthPrInt: Mitigating LVLM Object Hallucination Via Latent Truthful-Guided Pre-Intervention
TruthPrInt: Mitigating LVLM Object Hallucination Via Latent Truthful-Guided Pre-Intervention Open
Object Hallucination (OH) has been acknowledged as one of the major trustworthy challenges in Large Vision-Language Models (LVLMs). Recent advancements in Large Language Models (LLMs) indicate that internal states, such as hidden states, e…
View article: ZFP: A compressed array representation for numerical computations
ZFP: A compressed array representation for numerical computations Open
HPC trends favor algorithms and implementations that reduce data motion relative to FLOPS. We investigate the use of lossy compressed data arrays in place of traditional IEEE floating point arrays to store the primary data of calculations.…
View article: SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning
SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning Open
Large Language Models (LLMs) have highlighted the necessity of effective unlearning mechanisms to comply with data regulations and ethical AI practices. LLM unlearning aims at removing undesired data influences and associated model capabil…
View article: End-to-End Mesh Optimization of a Hybrid Deep Learning Black-Box PDE Solver
End-to-End Mesh Optimization of a Hybrid Deep Learning Black-Box PDE Solver Open
Deep learning has been widely applied to solve partial differential equations (PDEs) in computational fluid dynamics. Recent research proposed a PDE correction framework that leverages deep learning to correct the solution obtained by a PD…
View article: Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies
Adversarial Robustness Limits via Scaling-Law and Human-Alignment Studies Open
This paper revisits the simple, long-studied, yet still unsolved problem of making image classifiers robust to imperceptible perturbations. Taking CIFAR10 as an example, SOTA clean accuracy is about $100$%, but SOTA robustness to $\ell_{\i…
View article: Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression
Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression Open
Compressing high-capability Large Language Models (LLMs) has emerged as a favored strategy for resource-efficient inferences. While state-of-the-art (SoTA) compression methods boast impressive advancements in preserving benign task perform…
View article: GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations Open
As Large Language Models (LLMs) are integrated into critical real-world applications, their strategic and logical reasoning abilities are increasingly crucial. This paper evaluates LLMs' reasoning abilities in competitive environments thro…
View article: ReTA: Recursively Thinking Ahead to Improve the Strategic Reasoning of Large Language Models
ReTA: Recursively Thinking Ahead to Improve the Strategic Reasoning of Large Language Models Open
Jinhao Duan, Shiqi Wang, James Diffenderfer, Lichao Sun, Tianlong Chen, Bhavya Kailkhura, Kaidi Xu. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog…
View article: When Bio-Inspired Computing meets Deep Learning: Low-Latency, Accurate, & Energy-Efficient Spiking Neural Networks from Artificial Neural Networks
When Bio-Inspired Computing meets Deep Learning: Low-Latency, Accurate, & Energy-Efficient Spiking Neural Networks from Artificial Neural Networks Open
Bio-inspired Spiking Neural Networks (SNN) are now demonstrating comparable accuracy to intricate convolutional neural networks (CNN), all while delivering remarkable energy and latency efficiency when deployed on neuromorphic hardware. In…
View article: DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training
DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training Open
Zeroth-order (ZO) optimization has become a popular technique for solving machine learning (ML) problems when first-order (FO) information is difficult or impossible to obtain. However, the scalability of ZO optimization remains an open pr…
View article: Neural Image Compression: Generalization, Robustness, and Spectral Biases
Neural Image Compression: Generalization, Robustness, and Spectral Biases Open
Recent advances in neural image compression (NIC) have produced models that are starting to outperform classic codecs. While this has led to growing excitement about using NIC in real-world applications, the successful adoption of any mach…
View article: Efficient Multi-Prize Lottery Tickets: Enhanced Accuracy, Training, and Inference Speed
Efficient Multi-Prize Lottery Tickets: Enhanced Accuracy, Training, and Inference Speed Open
Recently, Diffenderfer and Kailkhura proposed a new paradigm for learning compact yet highly accurate binary neural networks simply by pruning and quantizing randomly weighted full precision neural networks. However, the accuracy of these …
View article: Models Out of Line: A Fourier Lens on Distribution Shift Robustness
Models Out of Line: A Fourier Lens on Distribution Shift Robustness Open
Improving the accuracy of deep neural networks (DNNs) on out-of-distribution (OOD) data is critical to an acceptance of deep learning (DL) in real world applications. It has been observed that accuracies on in-distribution (ID) versus OOD …
View article: Zeroth-Order SciML: Non-intrusive Integration of Scientific Software with Deep Learning
Zeroth-Order SciML: Non-intrusive Integration of Scientific Software with Deep Learning Open
Using deep learning (DL) to accelerate and/or improve scientific workflows can yield discoveries that are otherwise impossible. Unfortunately, DL models have yielded limited success in complex scientific domains due to large data requireme…
View article: A Framework for Error-Bounded Approximate Computing, with an Application to Dot Products
A Framework for Error-Bounded Approximate Computing, with an Application to Dot Products Open
Approximate computing techniques, which trade off the computation accuracy of an algorithm for better performance and energy efficiency, have been successful in reducing computation and power costs in several domains. However, error sensit…
View article: Benchmarking Test-Time Unsupervised Deep Neural Network Adaptation on Edge Devices
Benchmarking Test-Time Unsupervised Deep Neural Network Adaptation on Edge Devices Open
The prediction accuracy of the deep neural networks (DNNs) after deployment at the edge can suffer with time due to shifts in the distribution of the new data. To improve robustness of DNNs, they must be able to update themselves to enhanc…
View article: HPAC
HPAC Open
As we approach the limits of Moore's law, researchers are exploring new paradigms for future high-performance computing (HPC) systems. Approximate computing has gained traction by promising to deliver substantial computing power. However, …
View article: NPASA: An algorithm for nonlinear programming -- Motivation and Global Convergence
NPASA: An algorithm for nonlinear programming -- Motivation and Global Convergence Open
In this paper, we present a two phase method for solving nonlinear programming problems called Nonlinear Polyhedral Active Set Algorithm (NPASA) that has global and local convergence guarantees under reasonable assumptions. The first phase…
View article: NPASA: An algorithm for nonlinear programming -- Local Convergence
NPASA: An algorithm for nonlinear programming -- Local Convergence Open
In this paper, we provide local convergence analysis for the two phase Nonlinear Polyhedral Active Set Algorithm (NPASA) designed to solve nonlinear programs. In particular, we establish local quadratic convergence of the primal iterates a…
View article: A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness
A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness Open
Successful adoption of deep learning (DL) in the wild requires models to be: (1) compact, (2) accurate, and (3) robust to distributional shifts. Unfortunately, efforts towards simultaneously meeting these requirements have mostly been unsu…
View article: A Winning Hand: Compressing Deep Networks Can Improve\n Out-Of-Distribution Robustness
A Winning Hand: Compressing Deep Networks Can Improve\n Out-Of-Distribution Robustness Open
Successful adoption of deep learning (DL) in the wild requires models to be:\n(1) compact, (2) accurate, and (3) robust to distributional shifts.\nUnfortunately, efforts towards simultaneously meeting these requirements have\nmostly been u…
View article: QDOT: Quantized Dot Product Kernel for Approximate High-Performance Computing
QDOT: Quantized Dot Product Kernel for Approximate High-Performance Computing Open
Approximate computing techniques have been successful in reducing computation and power costs in several domains. However, error sensitive applications in high-performance computing are unable to benefit from existing approximate computing…
View article: Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network
Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network Open
Recently, Frankle & Carbin (2019) demonstrated that randomly-initialized dense networks contain subnetworks that once found can be trained to reach test accuracy comparable to the trained dense network. However, finding these high performi…
View article: Stability Analysis of Inline ZFP Compression for Floating-Point Data in Iterative Methods
Stability Analysis of Inline ZFP Compression for Floating-Point Data in Iterative Methods Open
Currently, the dominating constraint in many high performance computing applications is data capacity and bandwidth, in both inter-node communications and even more-so in on-node data motion. A new approach to address this limitation is to…
View article: Variable Precision Computing
Variable Precision Computing Open
This report summarizes the activities and major accomplishments of the Variable Precision Computing Strategic Initiative project. The overarching goal of this project was to initiate and promote a new paradigm in High Performance Computing…