Training Barlow Twins with Small Batch Sizes by Using a Queue of Previous Outputs Article Swipe

PDF

Pedro de Carvalho Cayres Pinto , José Gabriel R. C. Gomes ·

YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.14209/sbrt.2023.1570915729

We present two methods based on Barlow Twins, a self-supervised method, to improve training with smaller batches.The first method randomly drops features from the output before computing the loss to reduce the variance.The second method introduces a queue of outputs from previous batches to improve the loss estimate during training.The first method, with a batch size of 64, achieves an accuracy of 64.1%, the second method, with a batch size of 64 and 192 queued outputs, achieves an accuracy of 65.4%, while the original method, with a batch size of 256, achieves an accuracy of 66.0%.

Related Topics

Computer Science

Artificial Intelligence

Physics

Meteorology

Concepts

Queue Training (meteorology) Computer science Artificial intelligence Computer network Physics Meteorology

Metadata

Type: article
Language: pt
Landing Page: https://doi.org/10.14209/sbrt.2023.1570915729
PDF: https://biblioteca.sbrt.org.br/articlefile/4439.pdf
OA Status: gold
References: 24
Related Works: 10
OpenAlex ID: https://openalex.org/W4387942922

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4387942922

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.14209/sbrt.2023.1570915729

Digital Object Identifier
Title: Training Barlow Twins with Small Batch Sizes by Using a Queue of Previous Outputs

Work title
Type: article

OpenAlex work type
Language: pt

Primary language
Publication year: 2023

Year of publication
Publication date: 2023-01-01

Full publication date if available
Authors: Pedro de Carvalho Cayres Pinto, José Gabriel R. C. Gomes

List of authors in order
Landing page: https://doi.org/10.14209/sbrt.2023.1570915729

Publisher landing page
PDF URL: https://biblioteca.sbrt.org.br/articlefile/4439.pdf

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: gold

Open access status per OpenAlex
OA URL: https://biblioteca.sbrt.org.br/articlefile/4439.pdf

Direct OA link when available
Concepts: Queue, Training (meteorology), Computer science, Artificial intelligence, Computer network, Physics, Meteorology

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
References (count): 24

Number of works referenced by this work
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4387942922
doi	https://doi.org/10.14209/sbrt.2023.1570915729
ids.doi	https://doi.org/10.14209/sbrt.2023.1570915729
ids.openalex	https://openalex.org/W4387942922
fwci	0.0
type	article
title	Training Barlow Twins with Small Batch Sizes by Using a Queue of Previous Outputs
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T12072
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.8815000057220459
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Machine Learning and Algorithms
topics[1].id	https://openalex.org/T12423
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.7549999952316284
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1712
topics[1].subfield.display_name	Software
topics[1].display_name	Software Reliability and Analysis Research
topics[2].id	https://openalex.org/T12288
topics[2].field.id	https://openalex.org/fields/17
topics[2].field.display_name	Computer Science
topics[2].score	0.7156999707221985
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/1705
topics[2].subfield.display_name	Computer Networks and Communications
topics[2].display_name	Optimization and Search Problems
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C160403385
concepts[0].level	2
concepts[0].score	0.7278175354003906
concepts[0].wikidata	https://www.wikidata.org/wiki/Q220543
concepts[0].display_name	Queue
concepts[1].id	https://openalex.org/C2777211547
concepts[1].level	2
concepts[1].score	0.6702533960342407
concepts[1].wikidata	https://www.wikidata.org/wiki/Q17141490
concepts[1].display_name	Training (meteorology)
concepts[2].id	https://openalex.org/C41008148
concepts[2].level	0
concepts[2].score	0.6592247486114502
concepts[2].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[2].display_name	Computer science
concepts[3].id	https://openalex.org/C154945302
concepts[3].level	1
concepts[3].score	0.36076855659484863
concepts[3].wikidata	https://www.wikidata.org/wiki/Q11660
concepts[3].display_name	Artificial intelligence
concepts[4].id	https://openalex.org/C31258907
concepts[4].level	1
concepts[4].score	0.20689436793327332
concepts[4].wikidata	https://www.wikidata.org/wiki/Q1301371
concepts[4].display_name	Computer network
concepts[5].id	https://openalex.org/C121332964
concepts[5].level	0
concepts[5].score	0.05433034896850586
concepts[5].wikidata	https://www.wikidata.org/wiki/Q413
concepts[5].display_name	Physics
concepts[6].id	https://openalex.org/C153294291
concepts[6].level	1
concepts[6].score	0.0
concepts[6].wikidata	https://www.wikidata.org/wiki/Q25261
concepts[6].display_name	Meteorology
keywords[0].id	https://openalex.org/keywords/queue
keywords[0].score	0.7278175354003906
keywords[0].display_name	Queue
keywords[1].id	https://openalex.org/keywords/training
keywords[1].score	0.6702533960342407
keywords[1].display_name	Training (meteorology)
keywords[2].id	https://openalex.org/keywords/computer-science
keywords[2].score	0.6592247486114502
keywords[2].display_name	Computer science
keywords[3].id	https://openalex.org/keywords/artificial-intelligence
keywords[3].score	0.36076855659484863
keywords[3].display_name	Artificial intelligence
keywords[4].id	https://openalex.org/keywords/computer-network
keywords[4].score	0.20689436793327332
keywords[4].display_name	Computer network
keywords[5].id	https://openalex.org/keywords/physics
keywords[5].score	0.05433034896850586
keywords[5].display_name	Physics
language	pt
locations[0].id	doi:10.14209/sbrt.2023.1570915729
locations[0].is_oa	True
locations[0].source
locations[0].license
locations[0].pdf_url	https://biblioteca.sbrt.org.br/articlefile/4439.pdf
locations[0].version	publishedVersion
locations[0].raw_type	proceedings-article
locations[0].license_id
locations[0].is_accepted	True
locations[0].is_published	True
locations[0].raw_source_name	Anais do XLI Simpósio Brasileiro de Telecomunicações e Processamento de Sinais
locations[0].landing_page_url	https://doi.org/10.14209/sbrt.2023.1570915729
indexed_in	crossref
authorships[0].author.id	https://openalex.org/A5042773783
authorships[0].author.orcid
authorships[0].author.display_name	Pedro de Carvalho Cayres Pinto
authorships[0].author_position	first
authorships[0].raw_author_name	Pedro de Carvalho Cayres Pinto
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5027382066
authorships[1].author.orcid	https://orcid.org/0000-0003-3242-023X
authorships[1].author.display_name	José Gabriel R. C. Gomes
authorships[1].author_position	last
authorships[1].raw_author_name	Jose Gabriel Gomes
authorships[1].is_corresponding	False
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://biblioteca.sbrt.org.br/articlefile/4439.pdf
open_access.oa_status	gold
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Training Barlow Twins with Small Batch Sizes by Using a Queue of Previous Outputs
has_fulltext	True
is_retracted	False
updated_date	2025-11-06T03:46:38.306776
primary_topic.id	https://openalex.org/T12072
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.8815000057220459
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Machine Learning and Algorithms
related_works	https://openalex.org/W2748952813, https://openalex.org/W230091440, https://openalex.org/W2390279801, https://openalex.org/W2233261550, https://openalex.org/W2358668433, https://openalex.org/W2810751659, https://openalex.org/W258997015, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W2997094352
cited_by_count	0
locations_count	1
best_oa_location.id	doi:10.14209/sbrt.2023.1570915729
best_oa_location.is_oa	True
best_oa_location.source
best_oa_location.license
best_oa_location.pdf_url	https://biblioteca.sbrt.org.br/articlefile/4439.pdf
best_oa_location.version	publishedVersion
best_oa_location.raw_type	proceedings-article
best_oa_location.license_id
best_oa_location.is_accepted	True
best_oa_location.is_published	True
best_oa_location.raw_source_name	Anais do XLI Simpósio Brasileiro de Telecomunicações e Processamento de Sinais
best_oa_location.landing_page_url	https://doi.org/10.14209/sbrt.2023.1570915729
primary_location.id	doi:10.14209/sbrt.2023.1570915729
primary_location.is_oa	True
primary_location.source
primary_location.license
primary_location.pdf_url	https://biblioteca.sbrt.org.br/articlefile/4439.pdf
primary_location.version	publishedVersion
primary_location.raw_type	proceedings-article
primary_location.license_id
primary_location.is_accepted	True
primary_location.is_published	True
primary_location.raw_source_name	Anais do XLI Simpósio Brasileiro de Telecomunicações e Processamento de Sinais
primary_location.landing_page_url	https://doi.org/10.14209/sbrt.2023.1570915729
publication_date	2023-01-01
publication_year	2023
referenced_works	https://openalex.org/W3134652006, https://openalex.org/W3005680577, https://openalex.org/W2987283559, https://openalex.org/W3009561768, https://openalex.org/W3036224891, https://openalex.org/W3035060554, https://openalex.org/W3163602117, https://openalex.org/W6676297131, https://openalex.org/W6687483927, https://openalex.org/W6735463952, https://openalex.org/W3094502228, https://openalex.org/W6803870738, https://openalex.org/W3159481202, https://openalex.org/W4366208220, https://openalex.org/W3118608800, https://openalex.org/W2905060314, https://openalex.org/W1836465849, https://openalex.org/W6744513255, https://openalex.org/W4313156423, https://openalex.org/W2757910899, https://openalex.org/W2194775991, https://openalex.org/W2108598243, https://openalex.org/W3035524453, https://openalex.org/W4286695273
referenced_works_count	24
abstract_inverted_index.a	8, 36, 53, 67, 86
abstract_inverted_index.64	71
abstract_inverted_index.We	0
abstract_inverted_index.an	59, 77, 92
abstract_inverted_index.of	38, 56, 61, 70, 79, 89, 94
abstract_inverted_index.on	5
abstract_inverted_index.to	11, 29, 43
abstract_inverted_index.192	73
abstract_inverted_index.64,	57
abstract_inverted_index.and	72
abstract_inverted_index.the	23, 27, 31, 45, 63, 82
abstract_inverted_index.two	2
abstract_inverted_index.256,	90
abstract_inverted_index.from	22, 40
abstract_inverted_index.loss	28, 46
abstract_inverted_index.size	55, 69, 88
abstract_inverted_index.with	14, 52, 66, 85
abstract_inverted_index.based	4
abstract_inverted_index.batch	54, 68, 87
abstract_inverted_index.drops	20
abstract_inverted_index.first	17, 50
abstract_inverted_index.queue	37
abstract_inverted_index.while	81
abstract_inverted_index.64.1%,	62
abstract_inverted_index.65.4%,	80
abstract_inverted_index.66.0%.	95
abstract_inverted_index.Barlow	6
abstract_inverted_index.Twins,	7
abstract_inverted_index.before	25
abstract_inverted_index.during	48
abstract_inverted_index.method	18, 34
abstract_inverted_index.output	24
abstract_inverted_index.queued	74
abstract_inverted_index.reduce	30
abstract_inverted_index.second	33, 64
abstract_inverted_index.batches	42
abstract_inverted_index.improve	12, 44
abstract_inverted_index.method,	10, 51, 65, 84
abstract_inverted_index.methods	3
abstract_inverted_index.outputs	39
abstract_inverted_index.present	1
abstract_inverted_index.smaller	15
abstract_inverted_index.accuracy	60, 78, 93
abstract_inverted_index.achieves	58, 76, 91
abstract_inverted_index.estimate	47
abstract_inverted_index.features	21
abstract_inverted_index.original	83
abstract_inverted_index.outputs,	75
abstract_inverted_index.previous	41
abstract_inverted_index.randomly	19
abstract_inverted_index.training	13
abstract_inverted_index.computing	26
abstract_inverted_index.introduces	35
abstract_inverted_index.batches.The	16
abstract_inverted_index.training.The	49
abstract_inverted_index.variance.The	32
abstract_inverted_index.self-supervised	9
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	2
citation_normalized_percentile.value	0.16461486
citation_normalized_percentile.is_in_top_1_percent	False
citation_normalized_percentile.is_in_top_10_percent	False