A Combinatorial Perspective on Transfer Learning Article Swipe
Jianan Wang
,
Eren Sezener
,
David Budden
,
Marcus Hütter
,
Joel Veness
·
YOU?
·
· 2020
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2010.12268
YOU?
·
· 2020
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2010.12268
Human intelligence is characterized not only by the capacity to learn complex skills, but the ability to rapidly adapt and acquire new skills within an ever-changing environment. In this work we study how the learning of modular solutions can allow for effective generalization to both unseen and potentially differently distributed data. Our main postulate is that the combination of task segmentation, modular learning and memory-based ensembling can give rise to generalization on an exponentially growing number of unseen tasks. We provide a concrete instantiation of this idea using a combination of: (1) the Forget-Me-Not Process, for task segmentation and memory based ensembling; and (2) Gated Linear Networks, which in contrast to contemporary deep learning techniques use a modular and local learning mechanism. We demonstrate that this system exhibits a number of desirable continual learning properties: robustness to catastrophic forgetting, no negative transfer and increasing levels of positive transfer as more tasks are seen. We show competitive performance against both offline and online methods on standard continual learning benchmarks.
Related Topics
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2010.12268
- https://arxiv.org/pdf/2010.12268
- OA Status
- green
- Cited By
- 4
- References
- 35
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W3094087933
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W3094087933Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2010.12268Digital Object Identifier
- Title
-
A Combinatorial Perspective on Transfer LearningWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2020Year of publication
- Publication date
-
2020-10-23Full publication date if available
- Authors
-
Jianan Wang, Eren Sezener, David Budden, Marcus Hütter, Joel VenessList of authors in order
- Landing page
-
https://arxiv.org/abs/2010.12268Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2010.12268Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2010.12268Direct OA link when available
- Concepts
-
Perspective (graphical), Transfer of learning, Computer science, Artificial intelligenceTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
4Total citation count in OpenAlex
- Citations by year (recent)
-
2023: 1, 2022: 1, 2021: 2Per-year citation counts (last 5 years)
- References (count)
-
35Number of works referenced by this work
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W3094087933 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2010.12268 |
| ids.doi | https://doi.org/10.48550/arxiv.2010.12268 |
| ids.mag | 3094087933 |
| ids.openalex | https://openalex.org/W3094087933 |
| fwci | |
| type | preprint |
| title | A Combinatorial Perspective on Transfer Learning |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T11307 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9998000264167786 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Domain Adaptation and Few-Shot Learning |
| topics[1].id | https://openalex.org/T12072 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9994000196456909 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Machine Learning and Algorithms |
| topics[2].id | https://openalex.org/T11550 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9955999851226807 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Text and Document Classification Technologies |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C12713177 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7249670624732971 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q1900281 |
| concepts[0].display_name | Perspective (graphical) |
| concepts[1].id | https://openalex.org/C150899416 |
| concepts[1].level | 2 |
| concepts[1].score | 0.6469800472259521 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q1820378 |
| concepts[1].display_name | Transfer of learning |
| concepts[2].id | https://openalex.org/C41008148 |
| concepts[2].level | 0 |
| concepts[2].score | 0.41123902797698975 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[2].display_name | Computer science |
| concepts[3].id | https://openalex.org/C154945302 |
| concepts[3].level | 1 |
| concepts[3].score | 0.28038638830184937 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[3].display_name | Artificial intelligence |
| keywords[0].id | https://openalex.org/keywords/perspective |
| keywords[0].score | 0.7249670624732971 |
| keywords[0].display_name | Perspective (graphical) |
| keywords[1].id | https://openalex.org/keywords/transfer-of-learning |
| keywords[1].score | 0.6469800472259521 |
| keywords[1].display_name | Transfer of learning |
| keywords[2].id | https://openalex.org/keywords/computer-science |
| keywords[2].score | 0.41123902797698975 |
| keywords[2].display_name | Computer science |
| keywords[3].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[3].score | 0.28038638830184937 |
| keywords[3].display_name | Artificial intelligence |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2010.12268 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2010.12268 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2010.12268 |
| locations[1].id | doi:10.48550/arxiv.2010.12268 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2010.12268 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5100458008 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-4161-1687 |
| authorships[0].author.display_name | Jianan Wang |
| authorships[0].countries | US |
| authorships[0].affiliations[0].institution_ids | https://openalex.org/I1291425158 |
| authorships[0].affiliations[0].raw_affiliation_string | Google,,,,, |
| authorships[0].institutions[0].id | https://openalex.org/I1291425158 |
| authorships[0].institutions[0].ror | https://ror.org/00njsd438 |
| authorships[0].institutions[0].type | company |
| authorships[0].institutions[0].lineage | https://openalex.org/I1291425158, https://openalex.org/I4210128969 |
| authorships[0].institutions[0].country_code | US |
| authorships[0].institutions[0].display_name | Google (United States) |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Jianan Wang |
| authorships[0].is_corresponding | False |
| authorships[0].raw_affiliation_strings | Google,,,,, |
| authorships[1].author.id | https://openalex.org/A5088896420 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Eren Sezener |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Eren Sezener |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5003238807 |
| authorships[2].author.orcid | https://orcid.org/0000-0003-4372-8985 |
| authorships[2].author.display_name | David Budden |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | David Budden |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5073944062 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-3263-4097 |
| authorships[3].author.display_name | Marcus Hütter |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Marcus Hutter |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5060709021 |
| authorships[4].author.orcid | |
| authorships[4].author.display_name | Joel Veness |
| authorships[4].author_position | last |
| authorships[4].raw_author_name | Joel Veness |
| authorships[4].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2010.12268 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | A Combinatorial Perspective on Transfer Learning |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T11307 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9998000264167786 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Domain Adaptation and Few-Shot Learning |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W2358668433, https://openalex.org/W2149537132, https://openalex.org/W2376932109, https://openalex.org/W2018871932, https://openalex.org/W2001405890, https://openalex.org/W641279757, https://openalex.org/W370975646 |
| cited_by_count | 4 |
| counts_by_year[0].year | 2023 |
| counts_by_year[0].cited_by_count | 1 |
| counts_by_year[1].year | 2022 |
| counts_by_year[1].cited_by_count | 1 |
| counts_by_year[2].year | 2021 |
| counts_by_year[2].cited_by_count | 2 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2010.12268 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2010.12268 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2010.12268 |
| primary_location.id | pmh:oai:arXiv.org:2010.12268 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2010.12268 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2010.12268 |
| publication_date | 2020-10-23 |
| publication_year | 2020 |
| referenced_works | https://openalex.org/W2155541015, https://openalex.org/W3098842019, https://openalex.org/W2102605133, https://openalex.org/W2473930607, https://openalex.org/W2771657877, https://openalex.org/W2171809276, https://openalex.org/W2971176100, https://openalex.org/W2771116268, https://openalex.org/W3103187199, https://openalex.org/W2792568620, https://openalex.org/W2116522068, https://openalex.org/W2914607694, https://openalex.org/W2963559848, https://openalex.org/W2963588172, https://openalex.org/W2560647685, https://openalex.org/W2972335934, https://openalex.org/W2926477959, https://openalex.org/W3006992011, https://openalex.org/W2788388592, https://openalex.org/W2565989828, https://openalex.org/W2899063268, https://openalex.org/W2148029210, https://openalex.org/W3174224867, https://openalex.org/W2121391028, https://openalex.org/W2737492962, https://openalex.org/W2964189064, https://openalex.org/W2106304233, https://openalex.org/W2963850662, https://openalex.org/W1682403713, https://openalex.org/W2113839990, https://openalex.org/W2962724315, https://openalex.org/W2553665199, https://openalex.org/W2148825261, https://openalex.org/W2125055259, https://openalex.org/W2894094671 |
| referenced_works_count | 35 |
| abstract_inverted_index.a | 81, 88, 116, 128 |
| abstract_inverted_index.In | 27 |
| abstract_inverted_index.We | 79, 122, 153 |
| abstract_inverted_index.an | 24, 72 |
| abstract_inverted_index.as | 148 |
| abstract_inverted_index.by | 6 |
| abstract_inverted_index.in | 108 |
| abstract_inverted_index.is | 2, 54 |
| abstract_inverted_index.no | 139 |
| abstract_inverted_index.of | 35, 58, 76, 84, 130, 145 |
| abstract_inverted_index.on | 71, 163 |
| abstract_inverted_index.to | 9, 16, 43, 69, 110, 136 |
| abstract_inverted_index.we | 30 |
| abstract_inverted_index.(1) | 91 |
| abstract_inverted_index.(2) | 103 |
| abstract_inverted_index.Our | 51 |
| abstract_inverted_index.and | 19, 46, 63, 98, 102, 118, 142, 160 |
| abstract_inverted_index.are | 151 |
| abstract_inverted_index.but | 13 |
| abstract_inverted_index.can | 38, 66 |
| abstract_inverted_index.for | 40, 95 |
| abstract_inverted_index.how | 32 |
| abstract_inverted_index.new | 21 |
| abstract_inverted_index.not | 4 |
| abstract_inverted_index.of: | 90 |
| abstract_inverted_index.the | 7, 14, 33, 56, 92 |
| abstract_inverted_index.use | 115 |
| abstract_inverted_index.both | 44, 158 |
| abstract_inverted_index.deep | 112 |
| abstract_inverted_index.give | 67 |
| abstract_inverted_index.idea | 86 |
| abstract_inverted_index.main | 52 |
| abstract_inverted_index.more | 149 |
| abstract_inverted_index.only | 5 |
| abstract_inverted_index.rise | 68 |
| abstract_inverted_index.show | 154 |
| abstract_inverted_index.task | 59, 96 |
| abstract_inverted_index.that | 55, 124 |
| abstract_inverted_index.this | 28, 85, 125 |
| abstract_inverted_index.work | 29 |
| abstract_inverted_index.Gated | 104 |
| abstract_inverted_index.Human | 0 |
| abstract_inverted_index.adapt | 18 |
| abstract_inverted_index.allow | 39 |
| abstract_inverted_index.based | 100 |
| abstract_inverted_index.data. | 50 |
| abstract_inverted_index.learn | 10 |
| abstract_inverted_index.local | 119 |
| abstract_inverted_index.seen. | 152 |
| abstract_inverted_index.study | 31 |
| abstract_inverted_index.tasks | 150 |
| abstract_inverted_index.using | 87 |
| abstract_inverted_index.which | 107 |
| abstract_inverted_index.Linear | 105 |
| abstract_inverted_index.levels | 144 |
| abstract_inverted_index.memory | 99 |
| abstract_inverted_index.number | 75, 129 |
| abstract_inverted_index.online | 161 |
| abstract_inverted_index.skills | 22 |
| abstract_inverted_index.system | 126 |
| abstract_inverted_index.tasks. | 78 |
| abstract_inverted_index.unseen | 45, 77 |
| abstract_inverted_index.within | 23 |
| abstract_inverted_index.ability | 15 |
| abstract_inverted_index.acquire | 20 |
| abstract_inverted_index.against | 157 |
| abstract_inverted_index.complex | 11 |
| abstract_inverted_index.growing | 74 |
| abstract_inverted_index.methods | 162 |
| abstract_inverted_index.modular | 36, 61, 117 |
| abstract_inverted_index.offline | 159 |
| abstract_inverted_index.provide | 80 |
| abstract_inverted_index.rapidly | 17 |
| abstract_inverted_index.skills, | 12 |
| abstract_inverted_index.Process, | 94 |
| abstract_inverted_index.capacity | 8 |
| abstract_inverted_index.concrete | 82 |
| abstract_inverted_index.contrast | 109 |
| abstract_inverted_index.exhibits | 127 |
| abstract_inverted_index.learning | 34, 62, 113, 120, 133, 166 |
| abstract_inverted_index.negative | 140 |
| abstract_inverted_index.positive | 146 |
| abstract_inverted_index.standard | 164 |
| abstract_inverted_index.transfer | 141, 147 |
| abstract_inverted_index.Networks, | 106 |
| abstract_inverted_index.continual | 132, 165 |
| abstract_inverted_index.desirable | 131 |
| abstract_inverted_index.effective | 41 |
| abstract_inverted_index.postulate | 53 |
| abstract_inverted_index.solutions | 37 |
| abstract_inverted_index.ensembling | 65 |
| abstract_inverted_index.increasing | 143 |
| abstract_inverted_index.mechanism. | 121 |
| abstract_inverted_index.robustness | 135 |
| abstract_inverted_index.techniques | 114 |
| abstract_inverted_index.benchmarks. | 167 |
| abstract_inverted_index.combination | 57, 89 |
| abstract_inverted_index.competitive | 155 |
| abstract_inverted_index.demonstrate | 123 |
| abstract_inverted_index.differently | 48 |
| abstract_inverted_index.distributed | 49 |
| abstract_inverted_index.ensembling; | 101 |
| abstract_inverted_index.forgetting, | 138 |
| abstract_inverted_index.performance | 156 |
| abstract_inverted_index.potentially | 47 |
| abstract_inverted_index.properties: | 134 |
| abstract_inverted_index.catastrophic | 137 |
| abstract_inverted_index.contemporary | 111 |
| abstract_inverted_index.environment. | 26 |
| abstract_inverted_index.intelligence | 1 |
| abstract_inverted_index.memory-based | 64 |
| abstract_inverted_index.segmentation | 97 |
| abstract_inverted_index.Forget-Me-Not | 93 |
| abstract_inverted_index.characterized | 3 |
| abstract_inverted_index.ever-changing | 25 |
| abstract_inverted_index.exponentially | 73 |
| abstract_inverted_index.instantiation | 83 |
| abstract_inverted_index.segmentation, | 60 |
| abstract_inverted_index.generalization | 42, 70 |
| cited_by_percentile_year | |
| countries_distinct_count | 1 |
| institutions_distinct_count | 5 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/4 |
| sustainable_development_goals[0].score | 0.4699999988079071 |
| sustainable_development_goals[0].display_name | Quality Education |
| citation_normalized_percentile |