Fragment-based Pretraining and Finetuning on Molecular Graphs Article Swipe

MAKE IMPACT

Kha-Dinh Luong , Ambuj K. Singh · YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2310.03274

Property prediction on molecular graphs is an important application of Graph Neural Networks. Recently, unlabeled molecular data has become abundant, which facilitates the rapid development of self-supervised learning for GNNs in the chemical domain. In this work, we propose pretraining GNNs at the fragment level, a promising middle ground to overcome the limitations of node-level and graph-level pretraining. Borrowing techniques from recent work on principal subgraph mining, we obtain a compact vocabulary of prevalent fragments from a large pretraining dataset. From the extracted vocabulary, we introduce several fragment-based contrastive and predictive pretraining tasks. The contrastive learning task jointly pretrains two different GNNs: one on molecular graphs and the other on fragment graphs, which represents higher-order connectivity within molecules. By enforcing consistency between the fragment embedding and the aggregated embedding of the corresponding atoms from the molecular graphs, we ensure that the embeddings capture structural information at multiple resolutions. The structural information of fragment graphs is further exploited to extract auxiliary labels for graph-level predictive pretraining. We employ both the pretrained molecular-based and fragment-based GNNs for downstream prediction, thus utilizing the fragment information during finetuning. Our graph fragment-based pretraining (GraphFP) advances the performances on 5 out of 8 common molecular benchmarks and improves the performances on long-range biological benchmarks by at least 11.5%. Code is available at: https://github.com/lvkd84/GraphFP.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2310.03274

PDF

https://arxiv.org/pdf/2310.03274

OA Status

green

Cited By

3

Related Works

10

OpenAlex ID

https://openalex.org/W4387432253

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4387432253

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2310.03274

Digital Object Identifier

Title

Fragment-based Pretraining and Finetuning on Molecular Graphs

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2023

Year of publication

Publication date

2023-10-05

Full publication date if available

Authors

Kha-Dinh Luong, Ambuj K. Singh

List of authors in order

Landing page

https://arxiv.org/abs/2310.03274

Publisher landing page

PDF URL

https://arxiv.org/pdf/2310.03274

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2310.03274

Direct OA link when available

Concepts

Fragment (logic), Computer science, Embedding, Molecular graph, Graph, Theoretical computer science, Artificial intelligence, Algorithm

Top concepts (fields/topics) attached by OpenAlex

Cited by

3

Total citation count in OpenAlex

Citations by year (recent)

2025: 2, 2024: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload