Susmit Jha
YOU?
Author Swipe
View article: Privacy Preserving In-Context-Learning Framework for Large Language Models
Privacy Preserving In-Context-Learning Framework for Large Language Models Open
Large language models (LLMs) have significantly transformed natural language understanding and generation, but they raise privacy concerns due to potential exposure of sensitive information. Studies have highlighted the risk of information…
View article: Spatio-Temporal Pruning for Compressed Spiking Large Language Models
Spatio-Temporal Pruning for Compressed Spiking Large Language Models Open
Large Language Models (LLMs) present significant challenges for deployment in energy-constrained environments due to their large model sizes and high inference latency. Spiking Neural Networks (SNNs), inspired by the sparse event-driven ne…
View article: Predicting Secure Messaging Traffic in Clinical Settings
Predicting Secure Messaging Traffic in Clinical Settings Open
Asynchronous text-based communication, secure messaging, has become one of the preferred modes of communication despite its potential to disrupt workflow and increase burden. Identifying peak communication can guide interventions to reduce…
View article: Scalable Bayesian Low-Rank Adaptation of Large Language Models via Stochastic Variational Subspace Inference
Scalable Bayesian Low-Rank Adaptation of Large Language Models via Stochastic Variational Subspace Inference Open
Despite their widespread use, large language models (LLMs) are known to hallucinate incorrect information and be poorly calibrated. This makes the uncertainty quantification of these models of critical importance, especially in high-stakes…
View article: On the Evaluation of Engineering Artificial General Intelligence
On the Evaluation of Engineering Artificial General Intelligence Open
We discuss the challenges and propose a framework for evaluating engineering artificial general intelligence (eAGI) agents. We consider eAGI as a specialization of artificial general intelligence (AGI), deemed capable of addressing a broad…
View article: Safety Monitoring for Learning-Enabled Cyber-Physical Systems in Out-of-Distribution Scenarios
Safety Monitoring for Learning-Enabled Cyber-Physical Systems in Out-of-Distribution Scenarios Open
View article: Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding
Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding Open
We introduce a novel approach for calibrating uncertainty quantification (UQ) tailored for multi-modal large language models (LLMs). Existing state-of-the-art UQ methods rely on consistency among multiple responses generated by the LLM on …
View article: AGENT: An Aerial Vehicle Generation and Design Tool Using Large Language Models
AGENT: An Aerial Vehicle Generation and Design Tool Using Large Language Models Open
Computer-aided design (CAD) is a promising application area for emerging artificial intelligence methods. Traditional workflows for cyberphysical systems create detailed digital models which can be evaluated by physics simulators in order …
View article: TeleLoRA: Teleporting Model-Specific Alignment Across LLMs
TeleLoRA: Teleporting Model-Specific Alignment Across LLMs Open
Mitigating Trojans in Large Language Models (LLMs) is one of many tasks where alignment data is LLM specific, as different LLMs have different Trojan triggers and trigger behaviors to be removed. In this paper, we introduce TeleLoRA (Telep…
View article: Debugging and Runtime Analysis of Neural Networks with VLMs (A Case Study)
Debugging and Runtime Analysis of Neural Networks with VLMs (A Case Study) Open
Debugging of Deep Neural Networks (DNNs), particularly vision models, is very challenging due to the complex and opaque decision-making processes in these networks. In this paper, we explore multi-modal Vision-Language Models (VLMs), such …
View article: Calibration and Correctness of Language Models for Code ICSE Artifact
Calibration and Correctness of Language Models for Code ICSE Artifact Open
Machine learning models are widely used, but can also often be wrong. Users would benefit from a reliable indication of whether a given output from a given model should be trusted, so a rational decision can be made whether to use the outp…
View article: Polysemantic Dropout: Conformal OOD Detection for Specialized LLMs
Polysemantic Dropout: Conformal OOD Detection for Specialized LLMs Open
View article: Addressing Uncertainty in LLMs to Enhance Reliability in Generative AI
Addressing Uncertainty in LLMs to Enhance Reliability in Generative AI Open
In this paper, we present a dynamic semantic clustering approach inspired by the Chinese Restaurant Process, aimed at addressing uncertainty in the inference of Large Language Models (LLMs). We quantify uncertainty of an LLM on a given que…
View article: Second-Order Forward-Mode Automatic Differentiation for Optimization
Second-Order Forward-Mode Automatic Differentiation for Optimization Open
This paper introduces a second-order hyperplane search, a novel optimization step that generalizes a second-order line search from a line to a $k$-dimensional hyperplane. This, combined with the forward-mode stochastic gradient method, yie…
View article: Concept-based Analysis of Neural Networks via Vision-Language Models
Concept-based Analysis of Neural Networks via Vision-Language Models Open
The analysis of vision-based deep neural networks (DNNs) is highly desirable but it is very challenging due to the difficulty of expressing formal specifications for vision tasks and the lack of efficient verification procedures. In this p…
View article: Task-Agnostic Detector for Insertion-Based Backdoor Attacks
Task-Agnostic Detector for Insertion-Based Backdoor Attacks Open
Textual backdoor attacks pose significant security threats. Current detection approaches, typically relying on intermediate feature representation or reconstructing potential triggers, are task-specific and less effective beyond sentence c…
View article: Non-Markovian Quantum Control via Model Maximum Likelihood Estimation and Reinforcement Learning
Non-Markovian Quantum Control via Model Maximum Likelihood Estimation and Reinforcement Learning Open
Reinforcement Learning (RL) techniques have been increasingly applied in optimizing control systems. However, their application in quantum systems is hampered by the challenge of performing closed-loop control due to the difficulty in meas…
View article: Direct Amortized Likelihood Ratio Estimation
Direct Amortized Likelihood Ratio Estimation Open
We introduce a new amortized likelihood ratio estimator for likelihood-free simulation-based inference (SBI). Our estimator is simple to train and estimates the likelihood ratio using a single forward pass of the neural estimator. Our appr…
View article: math-PVS: A Large Language Model Framework to Map Scientific Publications to PVS Theories
math-PVS: A Large Language Model Framework to Map Scientific Publications to PVS Theories Open
As artificial intelligence (AI) gains greater adoption in a wide variety of applications, it has immense potential to contribute to mathematical discovery, by guiding conjecture generation, constructing counterexamples, assisting in formal…
View article: Neuro Symbolic Reasoning for Planning: Counterexample Guided Inductive Synthesis using Large Language Models and Satisfiability Solving
Neuro Symbolic Reasoning for Planning: Counterexample Guided Inductive Synthesis using Large Language Models and Satisfiability Solving Open
Generative large language models (LLMs) with instruct training such as GPT-4 can follow human-provided instruction prompts and generate human-like responses to these prompts. Apart from natural language responses, they have also been found…
View article: Neural Stochastic Differential Equations for Robust and Explainable Analysis of Electromagnetic Unintended Radiated Emissions
Neural Stochastic Differential Equations for Robust and Explainable Analysis of Electromagnetic Unintended Radiated Emissions Open
We present a comprehensive evaluation of the robustness and explainability of ResNet-like models in the context of Unintended Radiated Emission (URE) classification and suggest a new approach leveraging Neural Stochastic Differential Equat…
View article: TIJO: Trigger Inversion with Joint Optimization for Defending Multimodal Backdoored Models
TIJO: Trigger Inversion with Joint Optimization for Defending Multimodal Backdoored Models Open
We present a Multimodal Backdoor Defense technique TIJO (Trigger Inversion using Joint Optimization). Recent work arXiv:2112.07668 has demonstrated successful backdoor attacks on multimodal models for the Visual Question Answering task. Th…
View article: AircraftVerse: A Large-Scale Multimodal Dataset of Aerial Vehicle Designs
AircraftVerse: A Large-Scale Multimodal Dataset of Aerial Vehicle Designs Open
Dataset accompanying code and paper: AircraftVerse: A Large-Scale Multimodal Dataset of Aerial Vehicle Designs We present AircraftVerse, a publicly available aerial vehicle design dataset…
View article: AircraftVerse: A Large-Scale Multimodal Dataset of Aerial Vehicle Designs
AircraftVerse: A Large-Scale Multimodal Dataset of Aerial Vehicle Designs Open
We present AircraftVerse, a publicly available aerial vehicle design dataset. Aircraft design encompasses different physics domains and, hence, multiple modalities of representation. The evaluation of these cyber-physical system (CPS) desi…
View article: Measuring Classification Decision Certainty and Doubt
Measuring Classification Decision Certainty and Doubt Open
Quantitative characterizations and estimations of uncertainty are of fundamental importance in optimization and decision-making processes. Herein, we propose intuitive scores, which we call certainty and doubt, that can be used in both a B…
View article: On the Robustness of AlphaFold: A COVID-19 Case Study
On the Robustness of AlphaFold: A COVID-19 Case Study Open
Protein folding neural networks (PFNNs) such as AlphaFold predict remarkably accurate structures of proteins compared to other approaches. However, the robustness of such networks has heretofore not been explored. This is particularly rele…
View article: Principles of Robust Learning and Inference for IoBTs
Principles of Robust Learning and Inference for IoBTs Open
The Internet of Battlefield Things (IoBTs) operate in an adversarial rapidly-evolving environment, necessitating fast, robust and resilient decision-making. The success of machine learning, in particular deep learning methods, can improve …
View article: Design of Unmanned Air Vehicles Using Transformer Surrogate Models
Design of Unmanned Air Vehicles Using Transformer Surrogate Models Open
Computer-aided design (CAD) is a promising new area for the application of artificial intelligence (AI) and machine learning (ML). The current practice of design of cyber-physical systems uses the digital twin methodology, wherein the actu…
View article: CODiT: Conformal Out-of-Distribution Detection in Time-Series Data
CODiT: Conformal Out-of-Distribution Detection in Time-Series Data Open
Machine learning models are prone to making incorrect predictions on inputs that are far from the training distribution. This hinders their deployment in safety-critical applications such as autonomous vehicles and healthcare. The detectio…
View article: Inferring and Conveying Intentionality: Beyond Numerical Rewards to Logical Intentions
Inferring and Conveying Intentionality: Beyond Numerical Rewards to Logical Intentions Open
Shared intentionality is a critical component in developing conscious AI agents capable of collaboration, self-reflection, deliberation, and reasoning. We formulate inference of shared intentionality as an inverse reinforcement learning pr…