Mohammad Sadegh Akhondzadeh
YOU?
Author Swipe
View article: KurTail : Kurtosis-based LLM Quantization
KurTail : Kurtosis-based LLM Quantization Open
One of the challenges of quantizing a large language model (LLM) is the presence of outliers. Outliers often make uniform quantization schemes less effective, particularly in extreme cases such as 4-bit quantization. We introduce KurTail, …
View article: KurTail : Kurtosis-based LLM Quantization
KurTail : Kurtosis-based LLM Quantization Open
View article: Robust Yet Efficient Conformal Prediction Sets
Robust Yet Efficient Conformal Prediction Sets Open
Conformal prediction (CP) can convert any model's output into prediction sets guaranteed to include the true label with any user-specified probability. However, same as the model itself, CP is vulnerable to adversarial test examples (evasi…
View article: Probing Graph Representations
Probing Graph Representations Open
Today we have a good theoretical understanding of the representational power of Graph Neural Networks (GNNs). For example, their limitations have been characterized in relation to a hierarchy of Weisfeiler-Lehman (WL) isomorphism tests. Ho…
View article: Evaluating Sparse Interpretable Word Embeddings for Biomedical Domain
Evaluating Sparse Interpretable Word Embeddings for Biomedical Domain Open
Word embeddings have found their way into a wide range of natural language processing tasks including those in the biomedical domain. While these vector representations successfully capture semantic and syntactic word relations, hidden pat…