Johan A. K. Suykens
YOU?
Author Swipe
View article: Rethinking PCA Through Duality
Rethinking PCA Through Duality Open
Motivated by the recently shown connection between self-attention and (kernel) principal component analysis (PCA), we revisit the fundamentals of PCA. Using the difference-of-convex (DC) framework, we present several novel formulations and…
View article: Generative Kernel Spectral Clustering
Generative Kernel Spectral Clustering Open
Modern clustering approaches often trade interpretability for performance, particularly in deep learning-based methods. We present Generative Kernel Spectral Clustering (GenKSC), a novel model combining kernel spectral clustering with gene…
View article: A Dual Perspective of Reinforcement Learning for Imposing Policy Constraints
A Dual Perspective of Reinforcement Learning for Imposing Policy Constraints Open
View article: Generative Kernel Spectral Clustering
Generative Kernel Spectral Clustering Open
View article: Learning in Feature Spaces via Coupled Covariances: Asymmetric Kernel SVD and Nyström method
Learning in Feature Spaces via Coupled Covariances: Asymmetric Kernel SVD and Nyström method Open
In contrast with Mercer kernel-based approaches as used e.g., in Kernel Principal Component Analysis (KPCA), it was previously shown that Singular Value Decomposition (SVD) inherently relates to asymmetric kernels and Asymmetric Kernel Sin…
View article: Learning Analysis of Kernel Ridgeless Regression with Asymmetric Kernel Learning
Learning Analysis of Kernel Ridgeless Regression with Asymmetric Kernel Learning Open
Ridgeless regression has garnered attention among researchers, particularly in light of the ``Benign Overfitting'' phenomenon, where models interpolating noisy samples demonstrate robust generalization. However, kernel ridgeless regression…
View article: HeNCler: Node Clustering in Heterophilous Graphs via Learned Asymmetric Similarity
HeNCler: Node Clustering in Heterophilous Graphs via Learned Asymmetric Similarity Open
Clustering nodes in heterophilous graphs is challenging as traditional methods assume that effective clustering is characterized by high intra-cluster and low inter-cluster connectivity. To address this, we introduce HeNCler-a novel approa…
View article: SolNet: Open-source deep learning models for photovoltaic power forecasting across the globe
SolNet: Open-source deep learning models for photovoltaic power forecasting across the globe Open
Deep learning models have gained increasing prominence in recent years in the field of solar pho-tovoltaic (PV) forecasting. One drawback of these models is that they require a lot of high-quality data to perform well. This is often infeas…
View article: A Dual Perspective of Reinforcement Learning for Imposing Policy Constraints
A Dual Perspective of Reinforcement Learning for Imposing Policy Constraints Open
Model-free reinforcement learning methods lack an inherent mechanism to impose behavioural constraints on the trained policies. Although certain extensions exist, they remain limited to specific types of constraints, such as value constrai…
View article: Unsupervised Neighborhood Propagation Kernel Layers for Semi-supervised Node Classification
Unsupervised Neighborhood Propagation Kernel Layers for Semi-supervised Node Classification Open
We present a deep Graph Convolutional Kernel Machine (GCKM) for semi-supervised node classification in graphs. The method is built of two main types of blocks: (i) We introduce unsupervised kernel machine layers propagating the node featur…
View article: Sparsity via Sparse Group $k$-max Regularization
Sparsity via Sparse Group $k$-max Regularization Open
For the linear inverse problem with sparsity constraints, the $l_0$ regularized problem is NP-hard, and existing approaches either utilize greedy algorithms to find almost-optimal solutions or to approximate the $l_0$ regularization with i…
View article: Self-Attention through Kernel-Eigen Pair Sparse Variational Gaussian Processes
Self-Attention through Kernel-Eigen Pair Sparse Variational Gaussian Processes Open
While the great capability of Transformers significantly boosts prediction accuracy, it could also yield overconfident predictions and require calibrated uncertainty estimation, which can be commonly tackled by Gaussian processes (GPs). Ex…
View article: Can overfitted deep neural networks in adversarial training generalize? -- An approximation viewpoint
Can overfitted deep neural networks in adversarial training generalize? -- An approximation viewpoint Open
Adversarial training is a widely used method to improve the robustness of deep neural networks (DNNs) over adversarial perturbations. However, it is empirically observed that adversarial training on over-parameterized networks often suffer…
View article: Nonlinear functional regression by functional deep neural network with kernel embedding
Nonlinear functional regression by functional deep neural network with kernel embedding Open
Recently, deep learning has been widely applied in functional data analysis (FDA) with notable empirical success. However, the infinite dimensionality of functional data necessitates an effective dimension reduction approach for functional…
View article: CoRe-Sleep: A Multimodal Fusion Framework for Time Series Robust to Imperfect Modalities
CoRe-Sleep: A Multimodal Fusion Framework for Time Series Robust to Imperfect Modalities Open
Sleep abnormalities can have severe health consequences. Automated sleep staging, i.e. labelling the sequence of sleep stages from the patient's physiological recordings, could simplify the diagnostic process. Previous work on automated sl…
View article: Deep Kernel Principal Component Analysis for multi-level feature learning
Deep Kernel Principal Component Analysis for multi-level feature learning Open
View article: Accelerated sparse Kernel Spectral Clustering for large scale data clustering problems
Accelerated sparse Kernel Spectral Clustering for large scale data clustering problems Open
An improved version of the sparse multiway kernel spectral clustering (KSC) is presented in this brief. The original algorithm is derived from weighted kernel principal component (KPCA) analysis formulated within the primal-dual least-squa…
View article: Enhancing Kernel Flexibility via Learning Asymmetric Locally-Adaptive Kernels
Enhancing Kernel Flexibility via Learning Asymmetric Locally-Adaptive Kernels Open
The lack of sufficient flexibility is the key bottleneck of kernel-based learning that relies on manually designed, pre-given, and non-trainable kernels. To enhance kernel flexibility, this paper introduces the concept of Locally-Adaptive-…
View article: Low-Rank Multitask Learning based on Tensorized SVMs and LSSVMs
Low-Rank Multitask Learning based on Tensorized SVMs and LSSVMs Open
Multitask learning (MTL) leverages task-relatedness to enhance performance. With the emergence of multimodal data, tasks can now be referenced by multiple indices. In this paper, we employ high-order tensors, with each mode corresponding t…
View article: Multi-view kernel PCA for time series forecasting
Multi-view kernel PCA for time series forecasting Open
View article: A Dual Formulation for Probabilistic Principal Component Analysis
A Dual Formulation for Probabilistic Principal Component Analysis Open
In this paper, we characterize Probabilistic Principal Component Analysis in Hilbert spaces and demonstrate how the optimal solution admits a representation in dual space. This allows us to develop a generative framework for kernel methods…
View article: Unbalanced Optimal Transport: A Unified Framework for Object Detection
Unbalanced Optimal Transport: A Unified Framework for Object Detection Open
During training, supervised object detection tries to correctly match the predicted bounding boxes and associated classification scores to the ground truth. This is essential to determine which predictions are to be pushed towards which so…
View article: Increasing Performance And Sample Efficiency With Model-agnostic Interactive Feature Attributions
Increasing Performance And Sample Efficiency With Model-agnostic Interactive Feature Attributions Open
Model-agnostic feature attributions can provide local insights in complex ML models. If the explanation is correct, a domain expert can validate and trust the model's decision. However, if it contradicts the expert's knowledge, related wor…
View article: Explaining the Model and Feature Dependencies by Decomposition of the Shapley Value
Explaining the Model and Feature Dependencies by Decomposition of the Shapley Value Open
Shapley values have become one of the go-to methods to explain complex models to end-users. They provide a model agnostic post-hoc explanation with foundations in game theory: what is the worth of a player (in machine learning, a feature v…
View article: Nonlinear SVD with Asymmetric Kernels: feature learning and asymmetric Nyström method
Nonlinear SVD with Asymmetric Kernels: feature learning and asymmetric Nyström method Open
Asymmetric data naturally exist in real life, such as directed graphs. Different from the common kernel methods requiring Mercer kernels, this paper tackles the asymmetric kernel-based learning problem. We describe a nonlinear extension of…
View article: Combining Primal and Dual Representations in Deep Restricted Kernel Machines Classifiers
Combining Primal and Dual Representations in Deep Restricted Kernel Machines Classifiers Open
In the context of deep learning with kernel machines, the deep Restricted Kernel Machine (DRKM) framework allows multiple levels of kernel PCA (KPCA) and Least-Squares Support Vector Machines (LSSVM) to be combined into a deep architecture…
View article: Extending Kernel PCA through Dualization: Sparsity, Robustness and Fast Algorithms
Extending Kernel PCA through Dualization: Sparsity, Robustness and Fast Algorithms Open
The goal of this paper is to revisit Kernel Principal Component Analysis (KPCA) through dualization of a difference of convex functions. This allows to naturally extend KPCA to multiple objective functions and leads to efficient gradient-b…
View article: Primal-Attention: Self-attention through Asymmetric Kernel SVD in Primal Representation
Primal-Attention: Self-attention through Asymmetric Kernel SVD in Primal Representation Open
Recently, a new line of works has emerged to understand and improve self-attention in Transformers by treating it as a kernel machine. However, existing works apply the methods for symmetric kernels to the asymmetric self-attention, result…
View article: Duality in Multi-View Restricted Kernel Machines
Duality in Multi-View Restricted Kernel Machines Open
We propose a unifying setting that combines existing restricted kernel machine methods into a single primal-dual multi-view framework for kernel principal component analysis in both supervised and unsupervised settings. We derive the prima…
View article: CoRe-Sleep: A Multimodal Fusion Framework for Time Series Robust to Imperfect Modalities
CoRe-Sleep: A Multimodal Fusion Framework for Time Series Robust to Imperfect Modalities Open
Sleep abnormalities can have severe health consequences. Automated sleep staging, i.e. labelling the sequence of sleep stages from the patient's physiological recordings, could simplify the diagnostic process. Previous work on automated sl…