Matthias Fey
YOU?
Author Swipe
View article: Let’s Talk AI with AI Expert Matthias Fey
Let’s Talk AI with AI Expert Matthias Fey Open
Scale is hopefully not all you need. My personal AI mission: Productionizing Graph Neural Networks – a set of models with strong inductive bias that require less data and are easier to interpret.
View article: TGM: a Modular and Efficient Library for Machine Learning on Temporal Graphs
TGM: a Modular and Efficient Library for Machine Learning on Temporal Graphs Open
Well-designed open-source software drives progress in Machine Learning (ML) research. While static graph ML enjoys mature frameworks like PyTorch Geometric and DGL, ML for temporal graphs (TG), networks that evolve over time, lacks compara…
View article: PyG 2.0: Scalable Learning on Real World Graphs
PyG 2.0: Scalable Learning on Real World Graphs Open
PyG (PyTorch Geometric) has evolved significantly since its initial release, establishing itself as a leading framework for Graph Neural Networks. In this paper, we present Pyg 2.0 (and its subsequent minor versions), a comprehensive updat…
View article: Relational Graph Transformer
Relational Graph Transformer Open
Relational Deep Learning (RDL) is a promising approach for building state-of-the-art predictive models on multi-table relational data by representing it as a heterogeneous temporal graph. However, commonly used Graph Neural Network models …
View article: ContextGNN: Beyond Two-Tower Recommendation Systems
ContextGNN: Beyond Two-Tower Recommendation Systems Open
Recommendation systems predominantly utilize two-tower architectures, which evaluate user-item rankings through the inner product of their respective embeddings. However, one key limitation of two-tower models is that they learn a pair-agn…
View article: RelBench: A Benchmark for Deep Learning on Relational Databases
RelBench: A Benchmark for Deep Learning on Relational Databases Open
We present RelBench, a public benchmark for solving predictive tasks over relational databases with graph neural networks. RelBench provides databases and tasks spanning diverse domains and scales, and is intended to be a foundational infr…
View article: FASTEN: Fast GPU-accelerated Segmented Matrix Multiplication for Heterogenous Graph Neural Networks
FASTEN: Fast GPU-accelerated Segmented Matrix Multiplication for Heterogenous Graph Neural Networks Open
This paper introduces FASTEN, a cutting-edge library developed to address the computational challenges inherent in Heterogeneous Graph Neural Networks (HGNNs). The key focus of FASTEN is the optimization of segmented matrix multiplication,…
View article: From Similarity to Superiority: Channel Clustering for Time Series Forecasting
From Similarity to Superiority: Channel Clustering for Time Series Forecasting Open
Time series forecasting has attracted significant attention in recent decades. Previous studies have demonstrated that the Channel-Independent (CI) strategy improves forecasting performance by treating different channels individually, whil…
View article: PyTorch Frame: A Modular Framework for Multi-Modal Tabular Learning
PyTorch Frame: A Modular Framework for Multi-Modal Tabular Learning Open
We present PyTorch Frame, a PyTorch-based framework for deep learning over multi-modal tabular data. PyTorch Frame makes tabular deep learning easy by providing a PyTorch-based data structure to handle complex tabular data, introducing a m…
View article: Relational Deep Learning: Graph Representation Learning on Relational Databases
Relational Deep Learning: Graph Representation Learning on Relational Databases Open
Much of the world's most valued data is stored in relational databases and data warehouses, where the data is organized into many tables connected by primary-foreign key relations. However, building machine learning models using this data …
View article: Temporal Graph Benchmark for Machine Learning on Temporal Graphs
Temporal Graph Benchmark for Machine Learning on Temporal Graphs Open
We present the Temporal Graph Benchmark (TGB), a collection of challenging and diverse benchmark datasets for realistic, reproducible, and robust evaluation of machine learning models on temporal graphs. TGB datasets are of large scale, sp…
View article: 4 Structured Data
4 Structured Data Open
Parameter sharing is a key technique in various state-of-the-art machine learning approaches. The underlying idea is simple yet effective. Given a highly overparametrized model whose input data obeys some repetitive structure, multiple sub…
View article: Deconvolution of 1D NMR spectra: A deep learning-based approach
Deconvolution of 1D NMR spectra: A deep learning-based approach Open
View article: Weisfeiler and Leman go Machine Learning: The Story so far
Weisfeiler and Leman go Machine Learning: The Story so far Open
In recent years, algorithms and neural architectures based on the Weisfeiler--Leman algorithm, a well-known heuristic for the graph isomorphism problem, have emerged as a powerful tool for machine learning with graphs and relational data. …
View article: OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs.
OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs. Open
Enabling effective and efficient machine learning (ML) over large-scale graph data (e.g., graphs with billions of edges) can have a huge impact on both industrial and scientific applications. However, community efforts to advance large-sca…
View article: The Power of the Weisfeiler-Leman Algorithm for Machine Learning with Graphs
The Power of the Weisfeiler-Leman Algorithm for Machine Learning with Graphs Open
In recent years, algorithms and neural architectures based on the Weisfeiler-Leman algorithm, a well-known heuristic for the graph isomorphism problem, emerged as a powerful tool for (supervised) machine learning with graphs and relational…
View article: GNNAutoScale: Scalable and Expressive Graph Neural Networks via\n Historical Embeddings
GNNAutoScale: Scalable and Expressive Graph Neural Networks via\n Historical Embeddings Open
We present GNNAutoScale (GAS), a framework for scaling arbitrary\nmessage-passing GNNs to large graphs. GAS prunes entire sub-trees of the\ncomputation graph by utilizing historical embeddings from prior training\niterations, leading to co…
View article: GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddings
GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddings Open
We present GNNAutoScale (GAS), a framework for scaling arbitrary message-passing GNNs to large graphs. GAS prunes entire sub-trees of the computation graph by utilizing historical embeddings from prior training iterations, leading to const…
View article: OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs
OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs Open
Enabling effective and efficient machine learning (ML) over large-scale graph data (e.g., graphs with billions of edges) can have a great impact on both industrial and scientific applications. However, existing efforts to advance large-sca…
View article: Hierarchical Inter-Message Passing for Learning on Molecular Graphs
Hierarchical Inter-Message Passing for Learning on Molecular Graphs Open
We present a hierarchical neural message passing architecture for learning on molecular graphs. Our model takes in two complementary graph representations: the raw molecular graph representation and its associated junction tree, where node…
View article: Open Graph Benchmark: Datasets for Machine Learning on Graphs
Open Graph Benchmark: Datasets for Machine Learning on Graphs Open
We present the Open Graph Benchmark (OGB), a diverse set of challenging and realistic benchmark datasets to facilitate scalable, robust, and reproducible graph machine learning (ML) research. OGB datasets are large-scale, encompass multipl…
View article: Adversarial Generation of Continuous Implicit Shape Representations
Adversarial Generation of Continuous Implicit Shape Representations Open
This work presents a generative adversarial architecture for generating three-dimensional shapes based on signed distance representations. While the deep generation of shapes has been mostly tackled by voxel and surface point cloud approac…
View article: Deep Graph Matching Consensus
Deep Graph Matching Consensus Open
This work presents a two-stage neural architecture for learning and refining structural correspondences between graphs. First, we use localized node embeddings computed by a graph neural network to obtain an initial ranking of soft corresp…
View article: Adversarial Generation of Continuous Implicit Shape Representations
Adversarial Generation of Continuous Implicit Shape Representations Open
This work presents a generative adversarial architecture for generating three-dimensional shapes based on signed distance representations. While the deep generation of shapes has been mostly tackled by voxel and surface point cloud approac…
View article: Weisfeiler and Leman Go Neural: Higher-Order Graph Neural Networks
Weisfeiler and Leman Go Neural: Higher-Order Graph Neural Networks Open
In recent years, graph neural networks (GNNs) have emerged as a powerful neural architecture to learn vector representations of nodes and graphs in a supervised, end-to-end fashion. Up to now, GNNs have only been evaluated empirically—show…
View article: Just Jump: Dynamic Neighborhood Aggregation in Graph Neural Networks
Just Jump: Dynamic Neighborhood Aggregation in Graph Neural Networks Open
We propose a dynamic neighborhood aggregation (DNA) procedure guided by (multi-head) attention for representation learning on graphs. In contrast to current graph neural networks which follow a simple neighborhood aggregation scheme, our D…
View article: Fast Graph Representation Learning with PyTorch Geometric
Fast Graph Representation Learning with PyTorch Geometric Open
We introduce PyTorch Geometric, a library for deep learning on irregularly structured input data such as graphs, point clouds and manifolds, built upon PyTorch. In addition to general graph data structures and processing methods, it contai…
View article: Group Equivariant Capsule Networks
Group Equivariant Capsule Networks Open
We present group equivariant capsule networks, a framework to introduce guaranteed equivariance and invariance properties to the capsule network idea. Our work can be divided into two contributions. First, we present a generic routing by a…
View article: Recognizing Cuneiform Signs Using Graph Based Methods
Recognizing Cuneiform Signs Using Graph Based Methods Open
The cuneiform script constitutes one of the earliest systems of writing and is realized by wedge-shaped marks on clay tablets. A tremendous number of cuneiform tablets have already been discovered and are incrementally digitalized and made…
View article: SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels
SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels Open
We present Spline-based Convolutional Neural Networks (SplineCNNs), a variant of deep neural networks for irregular structured and geometric input, e.g., graphs or meshes. Our main contribution is a novel convolution operator based on B-sp…