Model and Feature Diversity for Bayesian Neural Networks in Mutual Learning Article Swipe
Cuong Pham
,
Cuong C. Nguyen
,
Trung Le
,
Dinh Phung
,
Gustavo Carneiro
,
Thanh-Toan Do
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2407.02721
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2407.02721
Bayesian Neural Networks (BNNs) offer probability distributions for model parameters, enabling uncertainty quantification in predictions. However, they often underperform compared to deterministic neural networks. Utilizing mutual learning can effectively enhance the performance of peer BNNs. In this paper, we propose a novel approach to improve BNNs performance through deep mutual learning. The proposed approaches aim to increase diversity in both network parameter distributions and feature distributions, promoting peer networks to acquire distinct features that capture different characteristics of the input, which enhances the effectiveness of mutual learning. Experimental results demonstrate significant improvements in the classification accuracy, negative log-likelihood, and expected calibration error when compared to traditional mutual learning for BNNs.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2407.02721
- https://arxiv.org/pdf/2407.02721
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4400375752
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4400375752Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2407.02721Digital Object Identifier
- Title
-
Model and Feature Diversity for Bayesian Neural Networks in Mutual LearningWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-07-03Full publication date if available
- Authors
-
Cuong Pham, Cuong C. Nguyen, Trung Le, Dinh Phung, Gustavo Carneiro, Thanh-Toan DoList of authors in order
- Landing page
-
https://arxiv.org/abs/2407.02721Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2407.02721Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2407.02721Direct OA link when available
- Concepts
-
Diversity (politics), Feature (linguistics), Artificial intelligence, Artificial neural network, Bayesian probability, Computer science, Machine learning, Sociology, Anthropology, Linguistics, PhilosophyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4400375752 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2407.02721 |
| ids.doi | https://doi.org/10.48550/arxiv.2407.02721 |
| ids.openalex | https://openalex.org/W4400375752 |
| fwci | |
| type | preprint |
| title | Model and Feature Diversity for Bayesian Neural Networks in Mutual Learning |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10320 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.963699996471405 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Neural Networks and Applications |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C2781316041 |
| concepts[0].level | 2 |
| concepts[0].score | 0.6652134656906128 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q1230584 |
| concepts[0].display_name | Diversity (politics) |
| concepts[1].id | https://openalex.org/C2776401178 |
| concepts[1].level | 2 |
| concepts[1].score | 0.6352445483207703 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q12050496 |
| concepts[1].display_name | Feature (linguistics) |
| concepts[2].id | https://openalex.org/C154945302 |
| concepts[2].level | 1 |
| concepts[2].score | 0.5802096724510193 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[2].display_name | Artificial intelligence |
| concepts[3].id | https://openalex.org/C50644808 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5545762181282043 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q192776 |
| concepts[3].display_name | Artificial neural network |
| concepts[4].id | https://openalex.org/C107673813 |
| concepts[4].level | 2 |
| concepts[4].score | 0.5109562277793884 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q812534 |
| concepts[4].display_name | Bayesian probability |
| concepts[5].id | https://openalex.org/C41008148 |
| concepts[5].level | 0 |
| concepts[5].score | 0.45866456627845764 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[5].display_name | Computer science |
| concepts[6].id | https://openalex.org/C119857082 |
| concepts[6].level | 1 |
| concepts[6].score | 0.4016662836074829 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[6].display_name | Machine learning |
| concepts[7].id | https://openalex.org/C144024400 |
| concepts[7].level | 0 |
| concepts[7].score | 0.11373230814933777 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q21201 |
| concepts[7].display_name | Sociology |
| concepts[8].id | https://openalex.org/C19165224 |
| concepts[8].level | 1 |
| concepts[8].score | 0.0 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q23404 |
| concepts[8].display_name | Anthropology |
| concepts[9].id | https://openalex.org/C41895202 |
| concepts[9].level | 1 |
| concepts[9].score | 0.0 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q8162 |
| concepts[9].display_name | Linguistics |
| concepts[10].id | https://openalex.org/C138885662 |
| concepts[10].level | 0 |
| concepts[10].score | 0.0 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q5891 |
| concepts[10].display_name | Philosophy |
| keywords[0].id | https://openalex.org/keywords/diversity |
| keywords[0].score | 0.6652134656906128 |
| keywords[0].display_name | Diversity (politics) |
| keywords[1].id | https://openalex.org/keywords/feature |
| keywords[1].score | 0.6352445483207703 |
| keywords[1].display_name | Feature (linguistics) |
| keywords[2].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[2].score | 0.5802096724510193 |
| keywords[2].display_name | Artificial intelligence |
| keywords[3].id | https://openalex.org/keywords/artificial-neural-network |
| keywords[3].score | 0.5545762181282043 |
| keywords[3].display_name | Artificial neural network |
| keywords[4].id | https://openalex.org/keywords/bayesian-probability |
| keywords[4].score | 0.5109562277793884 |
| keywords[4].display_name | Bayesian probability |
| keywords[5].id | https://openalex.org/keywords/computer-science |
| keywords[5].score | 0.45866456627845764 |
| keywords[5].display_name | Computer science |
| keywords[6].id | https://openalex.org/keywords/machine-learning |
| keywords[6].score | 0.4016662836074829 |
| keywords[6].display_name | Machine learning |
| keywords[7].id | https://openalex.org/keywords/sociology |
| keywords[7].score | 0.11373230814933777 |
| keywords[7].display_name | Sociology |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2407.02721 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2407.02721 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2407.02721 |
| locations[1].id | doi:10.48550/arxiv.2407.02721 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2407.02721 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5062890024 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-0973-0889 |
| authorships[0].author.display_name | Cuong Pham |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Pham, Cuong |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5021933953 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-2672-6291 |
| authorships[1].author.display_name | Cuong C. Nguyen |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Nguyen, Cuong C. |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5103082579 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-4328-2138 |
| authorships[2].author.display_name | Trung Le |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Le, Trung |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5036447132 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-9977-8247 |
| authorships[3].author.display_name | Dinh Phung |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Phung, Dinh |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5029215323 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-5571-6220 |
| authorships[4].author.display_name | Gustavo Carneiro |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Carneiro, Gustavo |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5025723803 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-6249-0848 |
| authorships[5].author.display_name | Thanh-Toan Do |
| authorships[5].author_position | last |
| authorships[5].raw_author_name | Do, Thanh-Toan |
| authorships[5].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2407.02721 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-07-06T00:00:00 |
| display_name | Model and Feature Diversity for Bayesian Neural Networks in Mutual Learning |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10320 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.963699996471405 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Neural Networks and Applications |
| related_works | https://openalex.org/W4236583520, https://openalex.org/W2322261865, https://openalex.org/W4398861705, https://openalex.org/W2961085424, https://openalex.org/W2391251536, https://openalex.org/W3147584709, https://openalex.org/W2734736160, https://openalex.org/W2409468626, https://openalex.org/W4206863193, https://openalex.org/W4387158780 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2407.02721 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2407.02721 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2407.02721 |
| primary_location.id | pmh:oai:arXiv.org:2407.02721 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2407.02721 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2407.02721 |
| publication_date | 2024-07-03 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 40 |
| abstract_inverted_index.In | 35 |
| abstract_inverted_index.in | 13, 58, 92 |
| abstract_inverted_index.of | 32, 77, 84 |
| abstract_inverted_index.to | 20, 43, 55, 69, 104 |
| abstract_inverted_index.we | 38 |
| abstract_inverted_index.The | 51 |
| abstract_inverted_index.aim | 54 |
| abstract_inverted_index.and | 63, 98 |
| abstract_inverted_index.can | 27 |
| abstract_inverted_index.for | 7, 108 |
| abstract_inverted_index.the | 30, 78, 82, 93 |
| abstract_inverted_index.BNNs | 45 |
| abstract_inverted_index.both | 59 |
| abstract_inverted_index.deep | 48 |
| abstract_inverted_index.peer | 33, 67 |
| abstract_inverted_index.that | 73 |
| abstract_inverted_index.they | 16 |
| abstract_inverted_index.this | 36 |
| abstract_inverted_index.when | 102 |
| abstract_inverted_index.BNNs. | 34, 109 |
| abstract_inverted_index.error | 101 |
| abstract_inverted_index.model | 8 |
| abstract_inverted_index.novel | 41 |
| abstract_inverted_index.offer | 4 |
| abstract_inverted_index.often | 17 |
| abstract_inverted_index.which | 80 |
| abstract_inverted_index.(BNNs) | 3 |
| abstract_inverted_index.Neural | 1 |
| abstract_inverted_index.input, | 79 |
| abstract_inverted_index.mutual | 25, 49, 85, 106 |
| abstract_inverted_index.neural | 22 |
| abstract_inverted_index.paper, | 37 |
| abstract_inverted_index.acquire | 70 |
| abstract_inverted_index.capture | 74 |
| abstract_inverted_index.enhance | 29 |
| abstract_inverted_index.feature | 64 |
| abstract_inverted_index.improve | 44 |
| abstract_inverted_index.network | 60 |
| abstract_inverted_index.propose | 39 |
| abstract_inverted_index.results | 88 |
| abstract_inverted_index.through | 47 |
| abstract_inverted_index.Bayesian | 0 |
| abstract_inverted_index.However, | 15 |
| abstract_inverted_index.Networks | 2 |
| abstract_inverted_index.approach | 42 |
| abstract_inverted_index.compared | 19, 103 |
| abstract_inverted_index.distinct | 71 |
| abstract_inverted_index.enabling | 10 |
| abstract_inverted_index.enhances | 81 |
| abstract_inverted_index.expected | 99 |
| abstract_inverted_index.features | 72 |
| abstract_inverted_index.increase | 56 |
| abstract_inverted_index.learning | 26, 107 |
| abstract_inverted_index.negative | 96 |
| abstract_inverted_index.networks | 68 |
| abstract_inverted_index.proposed | 52 |
| abstract_inverted_index.Utilizing | 24 |
| abstract_inverted_index.accuracy, | 95 |
| abstract_inverted_index.different | 75 |
| abstract_inverted_index.diversity | 57 |
| abstract_inverted_index.learning. | 50, 86 |
| abstract_inverted_index.networks. | 23 |
| abstract_inverted_index.parameter | 61 |
| abstract_inverted_index.promoting | 66 |
| abstract_inverted_index.approaches | 53 |
| abstract_inverted_index.calibration | 100 |
| abstract_inverted_index.demonstrate | 89 |
| abstract_inverted_index.effectively | 28 |
| abstract_inverted_index.parameters, | 9 |
| abstract_inverted_index.performance | 31, 46 |
| abstract_inverted_index.probability | 5 |
| abstract_inverted_index.significant | 90 |
| abstract_inverted_index.traditional | 105 |
| abstract_inverted_index.uncertainty | 11 |
| abstract_inverted_index.Experimental | 87 |
| abstract_inverted_index.improvements | 91 |
| abstract_inverted_index.predictions. | 14 |
| abstract_inverted_index.underperform | 18 |
| abstract_inverted_index.deterministic | 21 |
| abstract_inverted_index.distributions | 6, 62 |
| abstract_inverted_index.effectiveness | 83 |
| abstract_inverted_index.classification | 94 |
| abstract_inverted_index.distributions, | 65 |
| abstract_inverted_index.quantification | 12 |
| abstract_inverted_index.characteristics | 76 |
| abstract_inverted_index.log-likelihood, | 97 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 6 |
| citation_normalized_percentile |