Alignment of auditory artificial networks with massive individual fMRI brain data leads to generalisable improvements in brain encoding and downstream tasks Article Swipe

MAKE IMPACT

Maëlle Freteault , Maximilien Le Clei , Loïc Tetrel , Pierre Bellec , Nicolas Farrugia · YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.1162/imag_a_00525

Artificial neural networks trained in the field of artificial intelligence (AI) have emerged as key tools to model brain processes, sparking the idea of aligning network representations with brain dynamics to enhance performance on AI tasks. While this concept has gained support in the visual domain, we investigate here the feasibility of creating auditory artificial neural models directly aligned with individual brain activity. This objective raises major computational challenges, as models have to be trained directly with brain data, which is typically collected at a much smaller scale than data used to train AI models. We aimed to answer two key questions: (1) Can brain alignment of auditory models lead to improved brain encoding for novel, previously unseen stimuli? (2) Can brain alignment lead to generalisable representations of auditory signals that are useful for solving a variety of complex auditory tasks? To answer these questions, we relied on two massive datasets: a deep phenotyping dataset from the Courtois neuronal modelling project, where six subjects watched four seasons (36 h) of the Friends TV series in functional magnetic resonance imaging and the HEAR benchmark, a large battery of downstream auditory tasks. We fine-tuned SoundNet, a small pretrained convolutional neural network with ~2.5 M parameters. Aligning SoundNet with brain data from three seasons of Friends led to substantial improvement in brain encoding in the fourth season, extending beyond auditory and visual cortices. We also observed consistent performance gains on the HEAR benchmark, particularly for tasks with limited training data, where brain-aligned models performed comparably with the best-performing models regardless of size. We finally compared individual and group models, finding that individual models often matched or outperformed group models in both brain encoding and downstream task performance, highlighting the data efficiency of fine-tuning with individual brain data. Our results demonstrate the feasibility of aligning artificial neural network representations with individual brain activity during auditory processing, and suggest that this alignment is particularly beneficial for tasks with limited training data. Future research is needed to establish whether larger models can achieve even better performance and whether the observed gains extend to other tasks, particularly in the context of few-shot learning.

Type

article

Language

en

Landing Page

https://doi.org/10.1162/imag_a_00525

OA Status

diamond

Cited By

1

References

62

Related Works

10

OpenAlex ID

https://openalex.org/W4408560767

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4408560767

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.1162/imag_a_00525

Digital Object Identifier

Title

Alignment of auditory artificial networks with massive individual fMRI brain data leads to generalisable improvements in brain encoding and downstream tasks

Work title

Type

article

OpenAlex work type

Language

en

Primary language

Publication year

2025

Year of publication

Publication date

2025-01-01

Full publication date if available

Authors

Maëlle Freteault, Maximilien Le Clei, Loïc Tetrel, Pierre Bellec, Nicolas Farrugia

List of authors in order

Landing page

https://doi.org/10.1162/imag_a_00525

Publisher landing page

Open access

Yes

Whether a free full text is available

OA status

diamond

Open access status per OpenAlex

OA URL

https://doi.org/10.1162/imag_a_00525

Direct OA link when available

Concepts

Computer science, Encoding (memory), Brain activity and meditation, Benchmark (surveying), Functional magnetic resonance imaging, Artificial neural network, Artificial intelligence, Convolutional neural network, Psychology, Neuroscience, Electroencephalography, Geography, Geodesy

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2025: 1

Per-year citation counts (last 5 years)

References (count)

62

Number of works referenced by this work

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload