Mitigating Vanishing Activations in Deep CapsNets Using Channel Pruning Article Swipe
Sangeeta Sahu
,
Abdulrahman Altahhan
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2410.16908
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2410.16908
Capsule Networks outperform Convolutional Neural Networks in learning the part-whole relationships with viewpoint invariance, and the credit goes to their multidimensional capsules. It was assumed that increasing the number of capsule layers in the capsule networks would enhance the model performance. However, recent studies found that Capsule Networks lack scalability due to vanishing activations in the capsules of deeper layers. This paper thoroughly investigates the vanishing activation problem in deep Capsule Networks. To analyze this issue and understand how increasing capsule dimensions can facilitate deeper networks, various Capsule Network models are constructed and evaluated with different numbers of capsules, capsule dimensions, and intermediate layers for this paper. Unlike traditional model pruning, which reduces the number of model parameters and expedites model training, this study uses pruning to mitigate the vanishing activations in the deeper capsule layers. In addition, the backbone network and capsule layers are pruned with different pruning ratios to reduce the number of inactive capsules and achieve better model accuracy than the unpruned models.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2410.16908
- https://arxiv.org/pdf/2410.16908
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4404261334
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4404261334Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2410.16908Digital Object Identifier
- Title
-
Mitigating Vanishing Activations in Deep CapsNets Using Channel PruningWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-10-22Full publication date if available
- Authors
-
Sangeeta Sahu, Abdulrahman AltahhanList of authors in order
- Landing page
-
https://arxiv.org/abs/2410.16908Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2410.16908Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2410.16908Direct OA link when available
- Concepts
-
Pruning, Channel (broadcasting), Computer science, Artificial intelligence, Geology, Biology, Telecommunications, HorticultureTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4404261334 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2410.16908 |
| ids.doi | https://doi.org/10.48550/arxiv.2410.16908 |
| ids.openalex | https://openalex.org/W4404261334 |
| fwci | |
| type | preprint |
| title | Mitigating Vanishing Activations in Deep CapsNets Using Channel Pruning |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T11512 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.8263999819755554 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Anomaly Detection Techniques and Applications |
| topics[1].id | https://openalex.org/T11689 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.7993999719619751 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Adversarial Robustness in Machine Learning |
| topics[2].id | https://openalex.org/T12611 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.7785999774932861 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Neural Networks and Reservoir Computing |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C108010975 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7200113534927368 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q500094 |
| concepts[0].display_name | Pruning |
| concepts[1].id | https://openalex.org/C127162648 |
| concepts[1].level | 2 |
| concepts[1].score | 0.5860208868980408 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q16858953 |
| concepts[1].display_name | Channel (broadcasting) |
| concepts[2].id | https://openalex.org/C41008148 |
| concepts[2].level | 0 |
| concepts[2].score | 0.43667343258857727 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[2].display_name | Computer science |
| concepts[3].id | https://openalex.org/C154945302 |
| concepts[3].level | 1 |
| concepts[3].score | 0.41127297282218933 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[3].display_name | Artificial intelligence |
| concepts[4].id | https://openalex.org/C127313418 |
| concepts[4].level | 0 |
| concepts[4].score | 0.32018929719924927 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q1069 |
| concepts[4].display_name | Geology |
| concepts[5].id | https://openalex.org/C86803240 |
| concepts[5].level | 0 |
| concepts[5].score | 0.12640899419784546 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q420 |
| concepts[5].display_name | Biology |
| concepts[6].id | https://openalex.org/C76155785 |
| concepts[6].level | 1 |
| concepts[6].score | 0.1125015914440155 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q418 |
| concepts[6].display_name | Telecommunications |
| concepts[7].id | https://openalex.org/C144027150 |
| concepts[7].level | 1 |
| concepts[7].score | 0.0931117832660675 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q48803 |
| concepts[7].display_name | Horticulture |
| keywords[0].id | https://openalex.org/keywords/pruning |
| keywords[0].score | 0.7200113534927368 |
| keywords[0].display_name | Pruning |
| keywords[1].id | https://openalex.org/keywords/channel |
| keywords[1].score | 0.5860208868980408 |
| keywords[1].display_name | Channel (broadcasting) |
| keywords[2].id | https://openalex.org/keywords/computer-science |
| keywords[2].score | 0.43667343258857727 |
| keywords[2].display_name | Computer science |
| keywords[3].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[3].score | 0.41127297282218933 |
| keywords[3].display_name | Artificial intelligence |
| keywords[4].id | https://openalex.org/keywords/geology |
| keywords[4].score | 0.32018929719924927 |
| keywords[4].display_name | Geology |
| keywords[5].id | https://openalex.org/keywords/biology |
| keywords[5].score | 0.12640899419784546 |
| keywords[5].display_name | Biology |
| keywords[6].id | https://openalex.org/keywords/telecommunications |
| keywords[6].score | 0.1125015914440155 |
| keywords[6].display_name | Telecommunications |
| keywords[7].id | https://openalex.org/keywords/horticulture |
| keywords[7].score | 0.0931117832660675 |
| keywords[7].display_name | Horticulture |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2410.16908 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2410.16908 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2410.16908 |
| locations[1].id | doi:10.48550/arxiv.2410.16908 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2410.16908 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5072062135 |
| authorships[0].author.orcid | https://orcid.org/0000-0001-6149-0398 |
| authorships[0].author.display_name | Sangeeta Sahu |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Sahu, Siddharth |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5054979460 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-1133-7744 |
| authorships[1].author.display_name | Abdulrahman Altahhan |
| authorships[1].author_position | last |
| authorships[1].raw_author_name | Altahhan, Abdulrahman |
| authorships[1].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2410.16908 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Mitigating Vanishing Activations in Deep CapsNets Using Channel Pruning |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T11512 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.8263999819755554 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Anomaly Detection Techniques and Applications |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2324615561, https://openalex.org/W2086120259, https://openalex.org/W2390279801, https://openalex.org/W2245170124, https://openalex.org/W2076393078, https://openalex.org/W4391913857, https://openalex.org/W2358668433 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2410.16908 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2410.16908 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2410.16908 |
| primary_location.id | pmh:oai:arXiv.org:2410.16908 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2410.16908 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2410.16908 |
| publication_date | 2024-10-22 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.In | 136 |
| abstract_inverted_index.It | 22 |
| abstract_inverted_index.To | 72 |
| abstract_inverted_index.in | 6, 32, 54, 68, 131 |
| abstract_inverted_index.of | 29, 57, 97, 115, 154 |
| abstract_inverted_index.to | 18, 51, 126, 150 |
| abstract_inverted_index.and | 14, 76, 92, 101, 118, 141, 157 |
| abstract_inverted_index.are | 90, 144 |
| abstract_inverted_index.can | 82 |
| abstract_inverted_index.due | 50 |
| abstract_inverted_index.for | 104 |
| abstract_inverted_index.how | 78 |
| abstract_inverted_index.the | 8, 15, 27, 33, 38, 55, 64, 113, 128, 132, 138, 152, 163 |
| abstract_inverted_index.was | 23 |
| abstract_inverted_index.This | 60 |
| abstract_inverted_index.deep | 69 |
| abstract_inverted_index.goes | 17 |
| abstract_inverted_index.lack | 48 |
| abstract_inverted_index.than | 162 |
| abstract_inverted_index.that | 25, 45 |
| abstract_inverted_index.this | 74, 105, 122 |
| abstract_inverted_index.uses | 124 |
| abstract_inverted_index.with | 11, 94, 146 |
| abstract_inverted_index.found | 44 |
| abstract_inverted_index.issue | 75 |
| abstract_inverted_index.model | 39, 109, 116, 120, 160 |
| abstract_inverted_index.paper | 61 |
| abstract_inverted_index.study | 123 |
| abstract_inverted_index.their | 19 |
| abstract_inverted_index.which | 111 |
| abstract_inverted_index.would | 36 |
| abstract_inverted_index.Neural | 4 |
| abstract_inverted_index.Unlike | 107 |
| abstract_inverted_index.better | 159 |
| abstract_inverted_index.credit | 16 |
| abstract_inverted_index.deeper | 58, 84, 133 |
| abstract_inverted_index.layers | 31, 103, 143 |
| abstract_inverted_index.models | 89 |
| abstract_inverted_index.number | 28, 114, 153 |
| abstract_inverted_index.paper. | 106 |
| abstract_inverted_index.pruned | 145 |
| abstract_inverted_index.ratios | 149 |
| abstract_inverted_index.recent | 42 |
| abstract_inverted_index.reduce | 151 |
| abstract_inverted_index.Capsule | 0, 46, 70, 87 |
| abstract_inverted_index.Network | 88 |
| abstract_inverted_index.achieve | 158 |
| abstract_inverted_index.analyze | 73 |
| abstract_inverted_index.assumed | 24 |
| abstract_inverted_index.capsule | 30, 34, 80, 99, 134, 142 |
| abstract_inverted_index.enhance | 37 |
| abstract_inverted_index.layers. | 59, 135 |
| abstract_inverted_index.models. | 165 |
| abstract_inverted_index.network | 140 |
| abstract_inverted_index.numbers | 96 |
| abstract_inverted_index.problem | 67 |
| abstract_inverted_index.pruning | 125, 148 |
| abstract_inverted_index.reduces | 112 |
| abstract_inverted_index.studies | 43 |
| abstract_inverted_index.various | 86 |
| abstract_inverted_index.However, | 41 |
| abstract_inverted_index.Networks | 1, 5, 47 |
| abstract_inverted_index.accuracy | 161 |
| abstract_inverted_index.backbone | 139 |
| abstract_inverted_index.capsules | 56, 156 |
| abstract_inverted_index.inactive | 155 |
| abstract_inverted_index.learning | 7 |
| abstract_inverted_index.mitigate | 127 |
| abstract_inverted_index.networks | 35 |
| abstract_inverted_index.pruning, | 110 |
| abstract_inverted_index.unpruned | 164 |
| abstract_inverted_index.Networks. | 71 |
| abstract_inverted_index.addition, | 137 |
| abstract_inverted_index.capsules, | 98 |
| abstract_inverted_index.capsules. | 21 |
| abstract_inverted_index.different | 95, 147 |
| abstract_inverted_index.evaluated | 93 |
| abstract_inverted_index.expedites | 119 |
| abstract_inverted_index.networks, | 85 |
| abstract_inverted_index.training, | 121 |
| abstract_inverted_index.vanishing | 52, 65, 129 |
| abstract_inverted_index.viewpoint | 12 |
| abstract_inverted_index.activation | 66 |
| abstract_inverted_index.dimensions | 81 |
| abstract_inverted_index.facilitate | 83 |
| abstract_inverted_index.increasing | 26, 79 |
| abstract_inverted_index.outperform | 2 |
| abstract_inverted_index.parameters | 117 |
| abstract_inverted_index.part-whole | 9 |
| abstract_inverted_index.thoroughly | 62 |
| abstract_inverted_index.understand | 77 |
| abstract_inverted_index.activations | 53, 130 |
| abstract_inverted_index.constructed | 91 |
| abstract_inverted_index.dimensions, | 100 |
| abstract_inverted_index.invariance, | 13 |
| abstract_inverted_index.scalability | 49 |
| abstract_inverted_index.traditional | 108 |
| abstract_inverted_index.intermediate | 102 |
| abstract_inverted_index.investigates | 63 |
| abstract_inverted_index.performance. | 40 |
| abstract_inverted_index.Convolutional | 3 |
| abstract_inverted_index.relationships | 10 |
| abstract_inverted_index.multidimensional | 20 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 2 |
| citation_normalized_percentile |