Federated EndoViT: Pretraining Vision Transformers via Federated Learning on Endoscopic Image Collections Article Swipe

MAKE IMPACT

Max Kirchner , Alexander C. Jenke , Sebastian Bodenstedt , Fiona R. Kolbinger , Oliver Lester Saldanha , Jakob Nikolas Kather , Martin Wagner , Stefanie Speidel · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2504.16612

Purpose: In this study, we investigate the training of foundation models using federated learning to address data-sharing limitations and enable collaborative model training without data transfer for minimally invasive surgery. Methods: Inspired by the EndoViT study, we adapt the Masked Autoencoder for federated learning, enhancing it with adaptive Sharpness-Aware Minimization (FedSAM) and Stochastic Weight Averaging (SWA). Our model is pretrained on the Endo700k dataset collection and later fine-tuned and evaluated for tasks such as Semantic Segmentation, Action Triplet Recognition, and Surgical Phase Recognition. Results: Our findings demonstrate that integrating adaptive FedSAM into the federated MAE approach improves pretraining, leading to a reduction in reconstruction loss per patch. The application of FL-EndoViT in surgical downstream tasks results in performance comparable to CEN-EndoViT. Furthermore, FL-EndoViT exhibits advantages over CEN-EndoViT in surgical scene segmentation when data is limited and in action triplet recognition when large datasets are used. Conclusion: These findings highlight the potential of federated learning for privacy-preserving training of surgical foundation models, offering a robust and generalizable solution for surgical data science. Effective collaboration requires adapting federated learning methods, such as the integration of FedSAM, which can accommodate the inherent data heterogeneity across institutions. In future, exploring FL in video-based models may enhance these capabilities by incorporating spatiotemporal dynamics crucial for real-world surgical environments.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2504.16612

PDF

https://arxiv.org/pdf/2504.16612

OA Status

green

OpenAlex ID

https://openalex.org/W4415066388

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4415066388

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2504.16612

Digital Object Identifier

Title

Federated EndoViT: Pretraining Vision Transformers via Federated Learning on Endoscopic Image Collections

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-04-23

Full publication date if available

Authors

Max Kirchner, Alexander C. Jenke, Sebastian Bodenstedt, Fiona R. Kolbinger, Oliver Lester Saldanha, Jakob Nikolas Kather, Martin Wagner, Stefanie Speidel

List of authors in order

Landing page

https://arxiv.org/abs/2504.16612

Publisher landing page

PDF URL

https://arxiv.org/pdf/2504.16612

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2504.16612

Direct OA link when available

Cited by

0

Total citation count in OpenAlex

Full payload