Preserving Generalization of Language models in Few-shot Continual Relation Extraction Article Swipe
Quyen Tran
,
Nguyễn Xuân Thành
,
Nguyễn Hoàng Ánh
,
Nam Le Hai
,
Trung Le
,
Linh Ngo Van
,
Thien Huu Nguyen
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2410.00334
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2410.00334
Few-shot Continual Relations Extraction (FCRE) is an emerging and dynamic area of study where models can sequentially integrate knowledge from new relations with limited labeled data while circumventing catastrophic forgetting and preserving prior knowledge from pre-trained backbones. In this work, we introduce a novel method that leverages often-discarded language model heads. By employing these components via a mutual information maximization strategy, our approach helps maintain prior knowledge from the pre-trained backbone and strategically aligns the primary classification head, thereby enhancing model performance. Furthermore, we explore the potential of Large Language Models (LLMs), renowned for their wealth of knowledge, in addressing FCRE challenges. Our comprehensive experimental results underscore the efficacy of the proposed method and offer valuable insights for future work.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2410.00334
- https://arxiv.org/pdf/2410.00334
- OA Status
- green
- Cited By
- 1
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4403818994
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4403818994Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2410.00334Digital Object Identifier
- Title
-
Preserving Generalization of Language models in Few-shot Continual Relation ExtractionWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-10-01Full publication date if available
- Authors
-
Quyen Tran, Nguyễn Xuân Thành, Nguyễn Hoàng Ánh, Nam Le Hai, Trung Le, Linh Ngo Van, Thien Huu NguyenList of authors in order
- Landing page
-
https://arxiv.org/abs/2410.00334Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2410.00334Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2410.00334Direct OA link when available
- Concepts
-
Generalization, Shot (pellet), Relation (database), Natural language processing, Extraction (chemistry), Relationship extraction, Computer science, Artificial intelligence, Mathematics, Data mining, Chromatography, Chemistry, Mathematical analysis, Organic chemistryTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
1Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4403818994 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2410.00334 |
| ids.doi | https://doi.org/10.48550/arxiv.2410.00334 |
| ids.openalex | https://openalex.org/W4403818994 |
| fwci | |
| type | preprint |
| title | Preserving Generalization of Language models in Few-shot Continual Relation Extraction |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10181 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9994999766349792 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Natural Language Processing Techniques |
| topics[1].id | https://openalex.org/T10028 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9973999857902527 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Topic Modeling |
| topics[2].id | https://openalex.org/T13629 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9370999932289124 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Text Readability and Simplification |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C177148314 |
| concepts[0].level | 2 |
| concepts[0].score | 0.8233475685119629 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q170084 |
| concepts[0].display_name | Generalization |
| concepts[1].id | https://openalex.org/C2778344882 |
| concepts[1].level | 2 |
| concepts[1].score | 0.7553704977035522 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q278938 |
| concepts[1].display_name | Shot (pellet) |
| concepts[2].id | https://openalex.org/C25343380 |
| concepts[2].level | 2 |
| concepts[2].score | 0.7046020030975342 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q277521 |
| concepts[2].display_name | Relation (database) |
| concepts[3].id | https://openalex.org/C204321447 |
| concepts[3].level | 1 |
| concepts[3].score | 0.5562378764152527 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[3].display_name | Natural language processing |
| concepts[4].id | https://openalex.org/C4725764 |
| concepts[4].level | 2 |
| concepts[4].score | 0.5542798638343811 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q844704 |
| concepts[4].display_name | Extraction (chemistry) |
| concepts[5].id | https://openalex.org/C153604712 |
| concepts[5].level | 3 |
| concepts[5].score | 0.5507229566574097 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q7310755 |
| concepts[5].display_name | Relationship extraction |
| concepts[6].id | https://openalex.org/C41008148 |
| concepts[6].level | 0 |
| concepts[6].score | 0.5305906534194946 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[6].display_name | Computer science |
| concepts[7].id | https://openalex.org/C154945302 |
| concepts[7].level | 1 |
| concepts[7].score | 0.4665757119655609 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[7].display_name | Artificial intelligence |
| concepts[8].id | https://openalex.org/C33923547 |
| concepts[8].level | 0 |
| concepts[8].score | 0.28090140223503113 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[8].display_name | Mathematics |
| concepts[9].id | https://openalex.org/C124101348 |
| concepts[9].level | 1 |
| concepts[9].score | 0.1413554847240448 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q172491 |
| concepts[9].display_name | Data mining |
| concepts[10].id | https://openalex.org/C43617362 |
| concepts[10].level | 1 |
| concepts[10].score | 0.07969498634338379 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q170050 |
| concepts[10].display_name | Chromatography |
| concepts[11].id | https://openalex.org/C185592680 |
| concepts[11].level | 0 |
| concepts[11].score | 0.07597476243972778 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q2329 |
| concepts[11].display_name | Chemistry |
| concepts[12].id | https://openalex.org/C134306372 |
| concepts[12].level | 1 |
| concepts[12].score | 0.06494113802909851 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q7754 |
| concepts[12].display_name | Mathematical analysis |
| concepts[13].id | https://openalex.org/C178790620 |
| concepts[13].level | 1 |
| concepts[13].score | 0.0 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q11351 |
| concepts[13].display_name | Organic chemistry |
| keywords[0].id | https://openalex.org/keywords/generalization |
| keywords[0].score | 0.8233475685119629 |
| keywords[0].display_name | Generalization |
| keywords[1].id | https://openalex.org/keywords/shot |
| keywords[1].score | 0.7553704977035522 |
| keywords[1].display_name | Shot (pellet) |
| keywords[2].id | https://openalex.org/keywords/relation |
| keywords[2].score | 0.7046020030975342 |
| keywords[2].display_name | Relation (database) |
| keywords[3].id | https://openalex.org/keywords/natural-language-processing |
| keywords[3].score | 0.5562378764152527 |
| keywords[3].display_name | Natural language processing |
| keywords[4].id | https://openalex.org/keywords/extraction |
| keywords[4].score | 0.5542798638343811 |
| keywords[4].display_name | Extraction (chemistry) |
| keywords[5].id | https://openalex.org/keywords/relationship-extraction |
| keywords[5].score | 0.5507229566574097 |
| keywords[5].display_name | Relationship extraction |
| keywords[6].id | https://openalex.org/keywords/computer-science |
| keywords[6].score | 0.5305906534194946 |
| keywords[6].display_name | Computer science |
| keywords[7].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[7].score | 0.4665757119655609 |
| keywords[7].display_name | Artificial intelligence |
| keywords[8].id | https://openalex.org/keywords/mathematics |
| keywords[8].score | 0.28090140223503113 |
| keywords[8].display_name | Mathematics |
| keywords[9].id | https://openalex.org/keywords/data-mining |
| keywords[9].score | 0.1413554847240448 |
| keywords[9].display_name | Data mining |
| keywords[10].id | https://openalex.org/keywords/chromatography |
| keywords[10].score | 0.07969498634338379 |
| keywords[10].display_name | Chromatography |
| keywords[11].id | https://openalex.org/keywords/chemistry |
| keywords[11].score | 0.07597476243972778 |
| keywords[11].display_name | Chemistry |
| keywords[12].id | https://openalex.org/keywords/mathematical-analysis |
| keywords[12].score | 0.06494113802909851 |
| keywords[12].display_name | Mathematical analysis |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2410.00334 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | cc-by |
| locations[0].pdf_url | https://arxiv.org/pdf/2410.00334 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | https://openalex.org/licenses/cc-by |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2410.00334 |
| locations[1].id | doi:10.48550/arxiv.2410.00334 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2410.00334 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5064602068 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-2029-2584 |
| authorships[0].author.display_name | Quyen Tran |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Tran, Quyen |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5045213092 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-3984-7877 |
| authorships[1].author.display_name | Nguyễn Xuân Thành |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Thanh, Nguyen Xuan |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5068372747 |
| authorships[2].author.orcid | |
| authorships[2].author.display_name | Nguyễn Hoàng Ánh |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Anh, Nguyen Hoang |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5041849952 |
| authorships[3].author.orcid | https://orcid.org/0009-0005-8895-6051 |
| authorships[3].author.display_name | Nam Le Hai |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Hai, Nam Le |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5103134401 |
| authorships[4].author.orcid | https://orcid.org/0000-0003-2271-7486 |
| authorships[4].author.display_name | Trung Le |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Le, Trung |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5022485068 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-0011-5137 |
| authorships[5].author.display_name | Linh Ngo Van |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Van Ngo, Linh |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5026113034 |
| authorships[6].author.orcid | https://orcid.org/0000-0003-3768-4736 |
| authorships[6].author.display_name | Thien Huu Nguyen |
| authorships[6].author_position | last |
| authorships[6].raw_author_name | Nguyen, Thien Huu |
| authorships[6].is_corresponding | False |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2410.00334 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-10-28T00:00:00 |
| display_name | Preserving Generalization of Language models in Few-shot Continual Relation Extraction |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10181 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9994999766349792 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Natural Language Processing Techniques |
| related_works | https://openalex.org/W2976808399, https://openalex.org/W2609844752, https://openalex.org/W4285246823, https://openalex.org/W4226278302, https://openalex.org/W4221160509, https://openalex.org/W2547211086, https://openalex.org/W2538200646, https://openalex.org/W1968988659, https://openalex.org/W2888033806, https://openalex.org/W4385734297 |
| cited_by_count | 1 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2410.00334 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | cc-by |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2410.00334 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | https://openalex.org/licenses/cc-by |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2410.00334 |
| primary_location.id | pmh:oai:arXiv.org:2410.00334 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | cc-by |
| primary_location.pdf_url | https://arxiv.org/pdf/2410.00334 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | https://openalex.org/licenses/cc-by |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2410.00334 |
| publication_date | 2024-10-01 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 42, 56 |
| abstract_inverted_index.By | 51 |
| abstract_inverted_index.In | 37 |
| abstract_inverted_index.an | 6 |
| abstract_inverted_index.in | 98 |
| abstract_inverted_index.is | 5 |
| abstract_inverted_index.of | 11, 87, 96, 109 |
| abstract_inverted_index.we | 40, 83 |
| abstract_inverted_index.Our | 102 |
| abstract_inverted_index.and | 8, 30, 71, 113 |
| abstract_inverted_index.can | 15 |
| abstract_inverted_index.for | 93, 117 |
| abstract_inverted_index.new | 20 |
| abstract_inverted_index.our | 61 |
| abstract_inverted_index.the | 68, 74, 85, 107, 110 |
| abstract_inverted_index.via | 55 |
| abstract_inverted_index.FCRE | 100 |
| abstract_inverted_index.area | 10 |
| abstract_inverted_index.data | 25 |
| abstract_inverted_index.from | 19, 34, 67 |
| abstract_inverted_index.that | 45 |
| abstract_inverted_index.this | 38 |
| abstract_inverted_index.with | 22 |
| abstract_inverted_index.Large | 88 |
| abstract_inverted_index.head, | 77 |
| abstract_inverted_index.helps | 63 |
| abstract_inverted_index.model | 49, 80 |
| abstract_inverted_index.novel | 43 |
| abstract_inverted_index.offer | 114 |
| abstract_inverted_index.prior | 32, 65 |
| abstract_inverted_index.study | 12 |
| abstract_inverted_index.their | 94 |
| abstract_inverted_index.these | 53 |
| abstract_inverted_index.where | 13 |
| abstract_inverted_index.while | 26 |
| abstract_inverted_index.work, | 39 |
| abstract_inverted_index.work. | 119 |
| abstract_inverted_index.(FCRE) | 4 |
| abstract_inverted_index.Models | 90 |
| abstract_inverted_index.aligns | 73 |
| abstract_inverted_index.future | 118 |
| abstract_inverted_index.heads. | 50 |
| abstract_inverted_index.method | 44, 112 |
| abstract_inverted_index.models | 14 |
| abstract_inverted_index.mutual | 57 |
| abstract_inverted_index.wealth | 95 |
| abstract_inverted_index.(LLMs), | 91 |
| abstract_inverted_index.dynamic | 9 |
| abstract_inverted_index.explore | 84 |
| abstract_inverted_index.labeled | 24 |
| abstract_inverted_index.limited | 23 |
| abstract_inverted_index.primary | 75 |
| abstract_inverted_index.results | 105 |
| abstract_inverted_index.thereby | 78 |
| abstract_inverted_index.Few-shot | 0 |
| abstract_inverted_index.Language | 89 |
| abstract_inverted_index.approach | 62 |
| abstract_inverted_index.backbone | 70 |
| abstract_inverted_index.efficacy | 108 |
| abstract_inverted_index.emerging | 7 |
| abstract_inverted_index.insights | 116 |
| abstract_inverted_index.language | 48 |
| abstract_inverted_index.maintain | 64 |
| abstract_inverted_index.proposed | 111 |
| abstract_inverted_index.renowned | 92 |
| abstract_inverted_index.valuable | 115 |
| abstract_inverted_index.Continual | 1 |
| abstract_inverted_index.Relations | 2 |
| abstract_inverted_index.employing | 52 |
| abstract_inverted_index.enhancing | 79 |
| abstract_inverted_index.integrate | 17 |
| abstract_inverted_index.introduce | 41 |
| abstract_inverted_index.knowledge | 18, 33, 66 |
| abstract_inverted_index.leverages | 46 |
| abstract_inverted_index.potential | 86 |
| abstract_inverted_index.relations | 21 |
| abstract_inverted_index.strategy, | 60 |
| abstract_inverted_index.Extraction | 3 |
| abstract_inverted_index.addressing | 99 |
| abstract_inverted_index.backbones. | 36 |
| abstract_inverted_index.components | 54 |
| abstract_inverted_index.forgetting | 29 |
| abstract_inverted_index.knowledge, | 97 |
| abstract_inverted_index.preserving | 31 |
| abstract_inverted_index.underscore | 106 |
| abstract_inverted_index.challenges. | 101 |
| abstract_inverted_index.information | 58 |
| abstract_inverted_index.pre-trained | 35, 69 |
| abstract_inverted_index.Furthermore, | 82 |
| abstract_inverted_index.catastrophic | 28 |
| abstract_inverted_index.experimental | 104 |
| abstract_inverted_index.maximization | 59 |
| abstract_inverted_index.performance. | 81 |
| abstract_inverted_index.sequentially | 16 |
| abstract_inverted_index.circumventing | 27 |
| abstract_inverted_index.comprehensive | 103 |
| abstract_inverted_index.strategically | 72 |
| abstract_inverted_index.classification | 76 |
| abstract_inverted_index.often-discarded | 47 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 7 |
| citation_normalized_percentile |