How Fast can BERT Learn Simple Natural Language Inference? Article Swipe

PDF

YOU? · · 2021 · Open Access · · DOI: https://doi.org/10.18653/v1/2021.eacl-main.51

This paper empirically studies whether BERT can really learn to conduct natural language inference (NLI) without utilizing hidden dataset bias; and how efficiently it can learn if it could. This is done via creating a simple entailment judgment case which involves only binary predicates in plain English. The results show that the learning process of BERT is very slow. However, the efficiency of learning can be greatly improved (data reduction by a factor of 1,500) if task-related features are added. This suggests that domain knowledge greatly helps when conducting NLI with neural networks.

Related Topics

Computer Science

Artificial Intelligence

Mathematical Analysis

Archaeology

Management

Economics

Concepts

Computer science Inference Artificial intelligence Task (project management) Simple (philosophy) Natural language processing Natural language Process (computing) Logical consequence Textual entailment Domain (mathematical analysis) Binary number Natural (archaeology) Artificial neural network Machine learning Programming language Arithmetic History Mathematics Epistemology Philosophy Mathematical analysis Archaeology Management Economics

Metadata

Type: article
Language: en
Landing Page: https://doi.org/10.18653/v1/2021.eacl-main.51
PDF: https://aclanthology.org/2021.eacl-main.51.pdf
OA Status: gold
Cited By: 8
References: 43
Related Works: 10
OpenAlex ID: https://openalex.org/W3154482395

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W3154482395

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.18653/v1/2021.eacl-main.51

Digital Object Identifier
Title: How Fast can BERT Learn Simple Natural Language Inference?

Work title
Type: article

OpenAlex work type
Language: en

Primary language
Publication year: 2021

Year of publication
Publication date: 2021-01-01

Full publication date if available
Authors: Yi‐Chung Lin, Keh‐Yih Su

List of authors in order
Landing page: https://doi.org/10.18653/v1/2021.eacl-main.51

Publisher landing page
PDF URL: https://aclanthology.org/2021.eacl-main.51.pdf

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: gold

Open access status per OpenAlex
OA URL: https://aclanthology.org/2021.eacl-main.51.pdf

Direct OA link when available
Concepts: Computer science, Inference, Artificial intelligence, Task (project management), Simple (philosophy), Natural language processing, Natural language, Process (computing), Logical consequence, Textual entailment, Domain (mathematical analysis), Binary number, Natural (archaeology), Artificial neural network, Machine learning, Programming language, Arithmetic, History, Mathematics, Epistemology, Philosophy, Mathematical analysis, Archaeology, Management, Economics

Top concepts (fields/topics) attached by OpenAlex
Cited by: 8

Total citation count in OpenAlex
Citations by year (recent): 2025: 2, 2024: 1, 2023: 4, 2021: 1

Per-year citation counts (last 5 years)
References (count): 43

Number of works referenced by this work
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W3154482395
doi	https://doi.org/10.18653/v1/2021.eacl-main.51
ids.doi	https://doi.org/10.18653/v1/2021.eacl-main.51
ids.mag	3154482395
ids.openalex	https://openalex.org/W3154482395
fwci	0.84662145
type	article
title	How Fast can BERT Learn Simple Natural Language Inference?
biblio.issue
biblio.volume
biblio.last_page	633
biblio.first_page	626
topics[0].id	https://openalex.org/T10028
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9998999834060669
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Topic Modeling
topics[1].id	https://openalex.org/T10181
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9998000264167786
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1702
topics[1].subfield.display_name	Artificial Intelligence
topics[1].display_name	Natural Language Processing Techniques
topics[2].id	https://openalex.org/T10201
topics[2].field.id	https://openalex.org/fields/17
topics[2].field.display_name	Computer Science
topics[2].score	0.9945999979972839
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/1702
topics[2].subfield.display_name	Artificial Intelligence
topics[2].display_name	Speech Recognition and Synthesis
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C41008148
concepts[0].level	0
concepts[0].score	0.8391774892807007
concepts[0].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[0].display_name	Computer science
concepts[1].id	https://openalex.org/C2776214188
concepts[1].level	2
concepts[1].score	0.6967824101448059
concepts[1].wikidata	https://www.wikidata.org/wiki/Q408386
concepts[1].display_name	Inference
concepts[2].id	https://openalex.org/C154945302
concepts[2].level	1
concepts[2].score	0.6704915761947632
concepts[2].wikidata	https://www.wikidata.org/wiki/Q11660
concepts[2].display_name	Artificial intelligence
concepts[3].id	https://openalex.org/C2780451532
concepts[3].level	2
concepts[3].score	0.6619967222213745
concepts[3].wikidata	https://www.wikidata.org/wiki/Q759676
concepts[3].display_name	Task (project management)
concepts[4].id	https://openalex.org/C2780586882
concepts[4].level	2
concepts[4].score	0.6189234852790833
concepts[4].wikidata	https://www.wikidata.org/wiki/Q7520643
concepts[4].display_name	Simple (philosophy)
concepts[5].id	https://openalex.org/C204321447
concepts[5].level	1
concepts[5].score	0.5962701439857483
concepts[5].wikidata	https://www.wikidata.org/wiki/Q30642
concepts[5].display_name	Natural language processing
concepts[6].id	https://openalex.org/C195324797
concepts[6].level	2
concepts[6].score	0.5503309965133667
concepts[6].wikidata	https://www.wikidata.org/wiki/Q33742
concepts[6].display_name	Natural language
concepts[7].id	https://openalex.org/C98045186
concepts[7].level	2
concepts[7].score	0.5496269464492798
concepts[7].wikidata	https://www.wikidata.org/wiki/Q205663
concepts[7].display_name	Process (computing)
concepts[8].id	https://openalex.org/C134752490
concepts[8].level	2
concepts[8].score	0.5294278264045715
concepts[8].wikidata	https://www.wikidata.org/wiki/Q374182
concepts[8].display_name	Logical consequence
concepts[9].id	https://openalex.org/C95318506
concepts[9].level	3
concepts[9].score	0.47736215591430664
concepts[9].wikidata	https://www.wikidata.org/wiki/Q6588467
concepts[9].display_name	Textual entailment
concepts[10].id	https://openalex.org/C36503486
concepts[10].level	2
concepts[10].score	0.47079482674598694
concepts[10].wikidata	https://www.wikidata.org/wiki/Q11235244
concepts[10].display_name	Domain (mathematical analysis)
concepts[11].id	https://openalex.org/C48372109
concepts[11].level	2
concepts[11].score	0.4614824652671814
concepts[11].wikidata	https://www.wikidata.org/wiki/Q3913
concepts[11].display_name	Binary number
concepts[12].id	https://openalex.org/C2776608160
concepts[12].level	2
concepts[12].score	0.4315921664237976
concepts[12].wikidata	https://www.wikidata.org/wiki/Q4785462
concepts[12].display_name	Natural (archaeology)
concepts[13].id	https://openalex.org/C50644808
concepts[13].level	2
concepts[13].score	0.421994686126709
concepts[13].wikidata	https://www.wikidata.org/wiki/Q192776
concepts[13].display_name	Artificial neural network
concepts[14].id	https://openalex.org/C119857082
concepts[14].level	1
concepts[14].score	0.41497549414634705
concepts[14].wikidata	https://www.wikidata.org/wiki/Q2539
concepts[14].display_name	Machine learning
concepts[15].id	https://openalex.org/C199360897
concepts[15].level	1
concepts[15].score	0.16053807735443115
concepts[15].wikidata	https://www.wikidata.org/wiki/Q9143
concepts[15].display_name	Programming language
concepts[16].id	https://openalex.org/C94375191
concepts[16].level	1
concepts[16].score	0.08049571514129639
concepts[16].wikidata	https://www.wikidata.org/wiki/Q11205
concepts[16].display_name	Arithmetic
concepts[17].id	https://openalex.org/C95457728
concepts[17].level	0
concepts[17].score	0.0
concepts[17].wikidata	https://www.wikidata.org/wiki/Q309
concepts[17].display_name	History
concepts[18].id	https://openalex.org/C33923547
concepts[18].level	0
concepts[18].score	0.0
concepts[18].wikidata	https://www.wikidata.org/wiki/Q395
concepts[18].display_name	Mathematics
concepts[19].id	https://openalex.org/C111472728
concepts[19].level	1
concepts[19].score	0.0
concepts[19].wikidata	https://www.wikidata.org/wiki/Q9471
concepts[19].display_name	Epistemology
concepts[20].id	https://openalex.org/C138885662
concepts[20].level	0
concepts[20].score	0.0
concepts[20].wikidata	https://www.wikidata.org/wiki/Q5891
concepts[20].display_name	Philosophy
concepts[21].id	https://openalex.org/C134306372
concepts[21].level	1
concepts[21].score	0.0
concepts[21].wikidata	https://www.wikidata.org/wiki/Q7754
concepts[21].display_name	Mathematical analysis
concepts[22].id	https://openalex.org/C166957645
concepts[22].level	1
concepts[22].score	0.0
concepts[22].wikidata	https://www.wikidata.org/wiki/Q23498
concepts[22].display_name	Archaeology
concepts[23].id	https://openalex.org/C187736073
concepts[23].level	1
concepts[23].score	0.0
concepts[23].wikidata	https://www.wikidata.org/wiki/Q2920921
concepts[23].display_name	Management
concepts[24].id	https://openalex.org/C162324750
concepts[24].level	0
concepts[24].score	0.0
concepts[24].wikidata	https://www.wikidata.org/wiki/Q8134
concepts[24].display_name	Economics
keywords[0].id	https://openalex.org/keywords/computer-science
keywords[0].score	0.8391774892807007
keywords[0].display_name	Computer science
keywords[1].id	https://openalex.org/keywords/inference
keywords[1].score	0.6967824101448059
keywords[1].display_name	Inference
keywords[2].id	https://openalex.org/keywords/artificial-intelligence
keywords[2].score	0.6704915761947632
keywords[2].display_name	Artificial intelligence
keywords[3].id	https://openalex.org/keywords/task
keywords[3].score	0.6619967222213745
keywords[3].display_name	Task (project management)
keywords[4].id	https://openalex.org/keywords/simple
keywords[4].score	0.6189234852790833
keywords[4].display_name	Simple (philosophy)
keywords[5].id	https://openalex.org/keywords/natural-language-processing
keywords[5].score	0.5962701439857483
keywords[5].display_name	Natural language processing
keywords[6].id	https://openalex.org/keywords/natural-language
keywords[6].score	0.5503309965133667
keywords[6].display_name	Natural language
keywords[7].id	https://openalex.org/keywords/process
keywords[7].score	0.5496269464492798
keywords[7].display_name	Process (computing)
keywords[8].id	https://openalex.org/keywords/logical-consequence
keywords[8].score	0.5294278264045715
keywords[8].display_name	Logical consequence
keywords[9].id	https://openalex.org/keywords/textual-entailment
keywords[9].score	0.47736215591430664
keywords[9].display_name	Textual entailment
keywords[10].id	https://openalex.org/keywords/domain
keywords[10].score	0.47079482674598694
keywords[10].display_name	Domain (mathematical analysis)
keywords[11].id	https://openalex.org/keywords/binary-number
keywords[11].score	0.4614824652671814
keywords[11].display_name	Binary number
keywords[12].id	https://openalex.org/keywords/natural
keywords[12].score	0.4315921664237976
keywords[12].display_name	Natural (archaeology)
keywords[13].id	https://openalex.org/keywords/artificial-neural-network
keywords[13].score	0.421994686126709
keywords[13].display_name	Artificial neural network
keywords[14].id	https://openalex.org/keywords/machine-learning
keywords[14].score	0.41497549414634705
keywords[14].display_name	Machine learning
keywords[15].id	https://openalex.org/keywords/programming-language
keywords[15].score	0.16053807735443115
keywords[15].display_name	Programming language
keywords[16].id	https://openalex.org/keywords/arithmetic
keywords[16].score	0.08049571514129639
keywords[16].display_name	Arithmetic
language	en
locations[0].id	doi:10.18653/v1/2021.eacl-main.51
locations[0].is_oa	True
locations[0].source
locations[0].license	cc-by
locations[0].pdf_url	https://aclanthology.org/2021.eacl-main.51.pdf
locations[0].version	publishedVersion
locations[0].raw_type	proceedings-article
locations[0].license_id	https://openalex.org/licenses/cc-by
locations[0].is_accepted	True
locations[0].is_published	True
locations[0].raw_source_name	Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
locations[0].landing_page_url	https://doi.org/10.18653/v1/2021.eacl-main.51
indexed_in	crossref
authorships[0].author.id	https://openalex.org/A5058441595
authorships[0].author.orcid	https://orcid.org/0000-0002-0352-2694
authorships[0].author.display_name	Yi‐Chung Lin
authorships[0].countries	TW
authorships[0].affiliations[0].institution_ids	https://openalex.org/I4210098366
authorships[0].affiliations[0].raw_affiliation_string	Institute of Information Science Academia Sinica, Taiwan
authorships[0].institutions[0].id	https://openalex.org/I4210098366
authorships[0].institutions[0].ror	https://ror.org/00z83z196
authorships[0].institutions[0].type	facility
authorships[0].institutions[0].lineage	https://openalex.org/I4210098366, https://openalex.org/I84653119
authorships[0].institutions[0].country_code	TW
authorships[0].institutions[0].display_name	Institute of Information Science, Academia Sinica
authorships[0].author_position	first
authorships[0].raw_author_name	Yi-Chung Lin
authorships[0].is_corresponding	False
authorships[0].raw_affiliation_strings	Institute of Information Science Academia Sinica, Taiwan
authorships[1].author.id	https://openalex.org/A5070399630
authorships[1].author.orcid	https://orcid.org/0000-0003-2838-1362
authorships[1].author.display_name	Keh‐Yih Su
authorships[1].countries	TW
authorships[1].affiliations[0].institution_ids	https://openalex.org/I4210098366
authorships[1].affiliations[0].raw_affiliation_string	Institute of Information Science Academia Sinica, Taiwan
authorships[1].institutions[0].id	https://openalex.org/I4210098366
authorships[1].institutions[0].ror	https://ror.org/00z83z196
authorships[1].institutions[0].type	facility
authorships[1].institutions[0].lineage	https://openalex.org/I4210098366, https://openalex.org/I84653119
authorships[1].institutions[0].country_code	TW
authorships[1].institutions[0].display_name	Institute of Information Science, Academia Sinica
authorships[1].author_position	last
authorships[1].raw_author_name	Keh-Yih Su
authorships[1].is_corresponding	True
authorships[1].raw_affiliation_strings	Institute of Information Science Academia Sinica, Taiwan
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://aclanthology.org/2021.eacl-main.51.pdf
open_access.oa_status	gold
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	How Fast can BERT Learn Simple Natural Language Inference?
has_fulltext	True
is_retracted	False
updated_date	2025-11-06T03:46:38.306776
primary_topic.id	https://openalex.org/T10028
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9998999834060669
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Topic Modeling
related_works	https://openalex.org/W2169644218, https://openalex.org/W12963412, https://openalex.org/W2250460949, https://openalex.org/W3158371345, https://openalex.org/W3141423438, https://openalex.org/W2071098659, https://openalex.org/W2627035043, https://openalex.org/W4385571113, https://openalex.org/W2937401546, https://openalex.org/W2784994561
cited_by_count	8
counts_by_year[0].year	2025
counts_by_year[0].cited_by_count	2
counts_by_year[1].year	2024
counts_by_year[1].cited_by_count	1
counts_by_year[2].year	2023
counts_by_year[2].cited_by_count	4
counts_by_year[3].year	2021
counts_by_year[3].cited_by_count	1
locations_count	1
best_oa_location.id	doi:10.18653/v1/2021.eacl-main.51
best_oa_location.is_oa	True
best_oa_location.source
best_oa_location.license	cc-by
best_oa_location.pdf_url	https://aclanthology.org/2021.eacl-main.51.pdf
best_oa_location.version	publishedVersion
best_oa_location.raw_type	proceedings-article
best_oa_location.license_id	https://openalex.org/licenses/cc-by
best_oa_location.is_accepted	True
best_oa_location.is_published	True
best_oa_location.raw_source_name	Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
best_oa_location.landing_page_url	https://doi.org/10.18653/v1/2021.eacl-main.51
primary_location.id	doi:10.18653/v1/2021.eacl-main.51
primary_location.is_oa	True
primary_location.source
primary_location.license	cc-by
primary_location.pdf_url	https://aclanthology.org/2021.eacl-main.51.pdf
primary_location.version	publishedVersion
primary_location.raw_type	proceedings-article
primary_location.license_id	https://openalex.org/licenses/cc-by
primary_location.is_accepted	True
primary_location.is_published	True
primary_location.raw_source_name	Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
primary_location.landing_page_url	https://doi.org/10.18653/v1/2021.eacl-main.51
publication_date	2021-01-01
publication_year	2021
referenced_works	https://openalex.org/W2955315229, https://openalex.org/W2891012317, https://openalex.org/W2970438301, https://openalex.org/W2962736243, https://openalex.org/W2788496822, https://openalex.org/W2891555348, https://openalex.org/W4288548690, https://openalex.org/W3035507081, https://openalex.org/W2614311475, https://openalex.org/W2251869843, https://openalex.org/W2756386045, https://openalex.org/W1814328102, https://openalex.org/W2963846996, https://openalex.org/W2950700477, https://openalex.org/W2970597249, https://openalex.org/W2889787757, https://openalex.org/W2987553933, https://openalex.org/W2962985038, https://openalex.org/W2963341956, https://openalex.org/W2963719234, https://openalex.org/W2951286828, https://openalex.org/W2396767181, https://openalex.org/W2962843521, https://openalex.org/W2970727289, https://openalex.org/W4288347855, https://openalex.org/W2548036585, https://openalex.org/W2963995027, https://openalex.org/W2963394326, https://openalex.org/W2965373594, https://openalex.org/W2896457183, https://openalex.org/W2946417913, https://openalex.org/W2130158090, https://openalex.org/W2946609015, https://openalex.org/W2963120843, https://openalex.org/W2951936329, https://openalex.org/W2948771346, https://openalex.org/W3102568136, https://openalex.org/W2972324944, https://openalex.org/W2919420119, https://openalex.org/W2963351832, https://openalex.org/W1840435438, https://openalex.org/W2250790822, https://openalex.org/W2798665661
referenced_works_count	43
abstract_inverted_index.a	34, 71
abstract_inverted_index.be	65
abstract_inverted_index.by	70
abstract_inverted_index.if	26, 75
abstract_inverted_index.in	44
abstract_inverted_index.is	30, 56
abstract_inverted_index.it	23, 27
abstract_inverted_index.of	54, 62, 73
abstract_inverted_index.to	9
abstract_inverted_index.NLI	89
abstract_inverted_index.The	47
abstract_inverted_index.and	20
abstract_inverted_index.are	78
abstract_inverted_index.can	6, 24, 64
abstract_inverted_index.how	21
abstract_inverted_index.the	51, 60
abstract_inverted_index.via	32
abstract_inverted_index.BERT	5, 55
abstract_inverted_index.This	0, 29, 80
abstract_inverted_index.case	38
abstract_inverted_index.done	31
abstract_inverted_index.only	41
abstract_inverted_index.show	49
abstract_inverted_index.that	50, 82
abstract_inverted_index.very	57
abstract_inverted_index.when	87
abstract_inverted_index.with	90
abstract_inverted_index.(NLI)	14
abstract_inverted_index.(data	68
abstract_inverted_index.bias;	19
abstract_inverted_index.helps	86
abstract_inverted_index.learn	8, 25
abstract_inverted_index.paper	1
abstract_inverted_index.plain	45
abstract_inverted_index.slow.	58
abstract_inverted_index.which	39
abstract_inverted_index.1,500)	74
abstract_inverted_index.added.	79
abstract_inverted_index.binary	42
abstract_inverted_index.could.	28
abstract_inverted_index.domain	83
abstract_inverted_index.factor	72
abstract_inverted_index.hidden	17
abstract_inverted_index.neural	91
abstract_inverted_index.really	7
abstract_inverted_index.simple	35
abstract_inverted_index.conduct	10
abstract_inverted_index.dataset	18
abstract_inverted_index.greatly	66, 85
abstract_inverted_index.natural	11
abstract_inverted_index.process	53
abstract_inverted_index.results	48
abstract_inverted_index.studies	3
abstract_inverted_index.whether	4
abstract_inverted_index.without	15
abstract_inverted_index.English.	46
abstract_inverted_index.However,	59
abstract_inverted_index.creating	33
abstract_inverted_index.features	77
abstract_inverted_index.improved	67
abstract_inverted_index.involves	40
abstract_inverted_index.judgment	37
abstract_inverted_index.language	12
abstract_inverted_index.learning	52, 63
abstract_inverted_index.suggests	81
abstract_inverted_index.inference	13
abstract_inverted_index.knowledge	84
abstract_inverted_index.networks.	92
abstract_inverted_index.reduction	69
abstract_inverted_index.utilizing	16
abstract_inverted_index.conducting	88
abstract_inverted_index.efficiency	61
abstract_inverted_index.entailment	36
abstract_inverted_index.predicates	43
abstract_inverted_index.efficiently	22
abstract_inverted_index.empirically	2
abstract_inverted_index.task-related	76
cited_by_percentile_year.max	97
cited_by_percentile_year.min	89
corresponding_author_ids	https://openalex.org/A5070399630
countries_distinct_count	1
institutions_distinct_count	2
corresponding_institution_ids	https://openalex.org/I4210098366
sustainable_development_goals[0].id	https://metadata.un.org/sdg/4
sustainable_development_goals[0].score	0.8100000023841858
sustainable_development_goals[0].display_name	Quality Education
citation_normalized_percentile.value	0.78251369
citation_normalized_percentile.is_in_top_1_percent	False
citation_normalized_percentile.is_in_top_10_percent	False