Adaptive Multi-Modal Fusion Instance Segmentation for CAEVs in Complex Conditions: Dataset, Framework and Verifications Article Swipe
Pai Peng
,
Keke Geng
,
Guodong Yin
,
Yanbo Lu
,
Weichao Zhuang
,
Shuaipeng Liu
·
YOU?
·
· 2020
· Open Access
·
· DOI: https://doi.org/10.21203/rs.3.rs-44968/v1
YOU?
·
· 2020
· Open Access
·
· DOI: https://doi.org/10.21203/rs.3.rs-44968/v1
This paper aims to develop an end-to-end sharpening mixture of experts (SMoE) fusion framework to improve the robustness and accuracy of the perception systems for CAEVs in complex illumination and weather conditions. Three original contributions make our work distinctive from the existing relevant literature. First, we introduce the Complex KITTI dataset which consists of 7481 pairs of modified KITTI RGB images and the generated LiDAR dense depth maps, this dataset is fine annotated in instance-level with our proposed semi-automatic annotation method. Second, the SMoE fusion approach is devised to adaptively learn the robust kernels from complementary modalities. Finally, we implement comprehensive comparative experiments, the results show that our proposed SMoE framework yield significant improvements over the other fusion techniques in adverse environmental conditions.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- https://doi.org/10.21203/rs.3.rs-44968/v1
- https://www.researchsquare.com/article/rs-44968/v1.pdf
- OA Status
- gold
- References
- 74
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4238237943
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4238237943Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.21203/rs.3.rs-44968/v1Digital Object Identifier
- Title
-
Adaptive Multi-Modal Fusion Instance Segmentation for CAEVs in Complex Conditions: Dataset, Framework and VerificationsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2020Year of publication
- Publication date
-
2020-07-23Full publication date if available
- Authors
-
Pai Peng, Keke Geng, Guodong Yin, Yanbo Lu, Weichao Zhuang, Shuaipeng LiuList of authors in order
- Landing page
-
https://doi.org/10.21203/rs.3.rs-44968/v1Publisher landing page
- PDF URL
-
https://www.researchsquare.com/article/rs-44968/v1.pdfDirect link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
goldOpen access status per OpenAlex
- OA URL
-
https://www.researchsquare.com/article/rs-44968/v1.pdfDirect OA link when available
- Concepts
-
Modal, Segmentation, Fusion, Computer science, Artificial intelligence, Pattern recognition (psychology), Philosophy, Linguistics, Polymer chemistry, ChemistryTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- References (count)
-
74Number of works referenced by this work
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4238237943 |
|---|---|
| doi | https://doi.org/10.21203/rs.3.rs-44968/v1 |
| ids.doi | https://doi.org/10.21203/rs.3.rs-44968/v1 |
| ids.openalex | https://openalex.org/W4238237943 |
| fwci | 0.0 |
| type | preprint |
| title | Adaptive Multi-Modal Fusion Instance Segmentation for CAEVs in Complex Conditions: Dataset, Framework and Verifications |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T11099 |
| topics[0].field.id | https://openalex.org/fields/22 |
| topics[0].field.display_name | Engineering |
| topics[0].score | 0.9865000247955322 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/2203 |
| topics[0].subfield.display_name | Automotive Engineering |
| topics[0].display_name | Autonomous Vehicle Technology and Safety |
| topics[1].id | https://openalex.org/T10036 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9837999939918518 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1707 |
| topics[1].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[1].display_name | Advanced Neural Network Applications |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C71139939 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7057366967201233 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q910194 |
| concepts[0].display_name | Modal |
| concepts[1].id | https://openalex.org/C89600930 |
| concepts[1].level | 2 |
| concepts[1].score | 0.7048293948173523 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q1423946 |
| concepts[1].display_name | Segmentation |
| concepts[2].id | https://openalex.org/C158525013 |
| concepts[2].level | 2 |
| concepts[2].score | 0.612366259098053 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q2593739 |
| concepts[2].display_name | Fusion |
| concepts[3].id | https://openalex.org/C41008148 |
| concepts[3].level | 0 |
| concepts[3].score | 0.5629415512084961 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[3].display_name | Computer science |
| concepts[4].id | https://openalex.org/C154945302 |
| concepts[4].level | 1 |
| concepts[4].score | 0.5100796818733215 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[4].display_name | Artificial intelligence |
| concepts[5].id | https://openalex.org/C153180895 |
| concepts[5].level | 2 |
| concepts[5].score | 0.3685615062713623 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q7148389 |
| concepts[5].display_name | Pattern recognition (psychology) |
| concepts[6].id | https://openalex.org/C138885662 |
| concepts[6].level | 0 |
| concepts[6].score | 0.06713330745697021 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q5891 |
| concepts[6].display_name | Philosophy |
| concepts[7].id | https://openalex.org/C41895202 |
| concepts[7].level | 1 |
| concepts[7].score | 0.04793384671211243 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q8162 |
| concepts[7].display_name | Linguistics |
| concepts[8].id | https://openalex.org/C188027245 |
| concepts[8].level | 1 |
| concepts[8].score | 0.0 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q750446 |
| concepts[8].display_name | Polymer chemistry |
| concepts[9].id | https://openalex.org/C185592680 |
| concepts[9].level | 0 |
| concepts[9].score | 0.0 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q2329 |
| concepts[9].display_name | Chemistry |
| keywords[0].id | https://openalex.org/keywords/modal |
| keywords[0].score | 0.7057366967201233 |
| keywords[0].display_name | Modal |
| keywords[1].id | https://openalex.org/keywords/segmentation |
| keywords[1].score | 0.7048293948173523 |
| keywords[1].display_name | Segmentation |
| keywords[2].id | https://openalex.org/keywords/fusion |
| keywords[2].score | 0.612366259098053 |
| keywords[2].display_name | Fusion |
| keywords[3].id | https://openalex.org/keywords/computer-science |
| keywords[3].score | 0.5629415512084961 |
| keywords[3].display_name | Computer science |
| keywords[4].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[4].score | 0.5100796818733215 |
| keywords[4].display_name | Artificial intelligence |
| keywords[5].id | https://openalex.org/keywords/pattern-recognition |
| keywords[5].score | 0.3685615062713623 |
| keywords[5].display_name | Pattern recognition (psychology) |
| keywords[6].id | https://openalex.org/keywords/philosophy |
| keywords[6].score | 0.06713330745697021 |
| keywords[6].display_name | Philosophy |
| keywords[7].id | https://openalex.org/keywords/linguistics |
| keywords[7].score | 0.04793384671211243 |
| keywords[7].display_name | Linguistics |
| language | en |
| locations[0].id | doi:10.21203/rs.3.rs-44968/v1 |
| locations[0].is_oa | True |
| locations[0].source | |
| locations[0].license | cc-by |
| locations[0].pdf_url | https://www.researchsquare.com/article/rs-44968/v1.pdf |
| locations[0].version | acceptedVersion |
| locations[0].raw_type | posted-content |
| locations[0].license_id | https://openalex.org/licenses/cc-by |
| locations[0].is_accepted | True |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | https://doi.org/10.21203/rs.3.rs-44968/v1 |
| indexed_in | crossref |
| authorships[0].author.id | https://openalex.org/A5101564498 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-4755-9736 |
| authorships[0].author.display_name | Pai Peng |
| authorships[0].affiliations[0].raw_affiliation_string | Southeast University |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Pai Peng |
| authorships[0].is_corresponding | False |
| authorships[0].raw_affiliation_strings | Southeast University |
| authorships[1].author.id | https://openalex.org/A5063355672 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-9505-4260 |
| authorships[1].author.display_name | Keke Geng |
| authorships[1].affiliations[0].raw_affiliation_string | Southeast University |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Keke Geng |
| authorships[1].is_corresponding | False |
| authorships[1].raw_affiliation_strings | Southeast University |
| authorships[2].author.id | https://openalex.org/A5064394634 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-8437-7598 |
| authorships[2].author.display_name | Guodong Yin |
| authorships[2].affiliations[0].raw_affiliation_string | Southeast University |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Guodong YIN |
| authorships[2].is_corresponding | False |
| authorships[2].raw_affiliation_strings | Southeast University |
| authorships[3].author.id | https://openalex.org/A5102020887 |
| authorships[3].author.orcid | https://orcid.org/0000-0001-5711-7742 |
| authorships[3].author.display_name | Yanbo Lu |
| authorships[3].affiliations[0].raw_affiliation_string | Southeast University |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Yanbo Lu |
| authorships[3].is_corresponding | False |
| authorships[3].raw_affiliation_strings | Southeast University |
| authorships[4].author.id | https://openalex.org/A5083074361 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-4958-1446 |
| authorships[4].author.display_name | Weichao Zhuang |
| authorships[4].affiliations[0].raw_affiliation_string | Southeast University |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Weichao Zhuang |
| authorships[4].is_corresponding | False |
| authorships[4].raw_affiliation_strings | Southeast University |
| authorships[5].author.id | https://openalex.org/A5060240389 |
| authorships[5].author.orcid | https://orcid.org/0009-0009-3958-9020 |
| authorships[5].author.display_name | Shuaipeng Liu |
| authorships[5].affiliations[0].raw_affiliation_string | Southeast University |
| authorships[5].author_position | last |
| authorships[5].raw_author_name | Shuaipeng Liu |
| authorships[5].is_corresponding | False |
| authorships[5].raw_affiliation_strings | Southeast University |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://www.researchsquare.com/article/rs-44968/v1.pdf |
| open_access.oa_status | gold |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Adaptive Multi-Modal Fusion Instance Segmentation for CAEVs in Complex Conditions: Dataset, Framework and Verifications |
| has_fulltext | True |
| is_retracted | False |
| updated_date | 2025-11-06T03:46:38.306776 |
| primary_topic.id | https://openalex.org/T11099 |
| primary_topic.field.id | https://openalex.org/fields/22 |
| primary_topic.field.display_name | Engineering |
| primary_topic.score | 0.9865000247955322 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/2203 |
| primary_topic.subfield.display_name | Automotive Engineering |
| primary_topic.display_name | Autonomous Vehicle Technology and Safety |
| related_works | https://openalex.org/W2379392295, https://openalex.org/W3160965418, https://openalex.org/W4379231730, https://openalex.org/W613940353, https://openalex.org/W2320915480, https://openalex.org/W4389858081, https://openalex.org/W2099421762, https://openalex.org/W2362990116, https://openalex.org/W2033914206, https://openalex.org/W2042327336 |
| cited_by_count | 0 |
| locations_count | 1 |
| best_oa_location.id | doi:10.21203/rs.3.rs-44968/v1 |
| best_oa_location.is_oa | True |
| best_oa_location.source | |
| best_oa_location.license | cc-by |
| best_oa_location.pdf_url | https://www.researchsquare.com/article/rs-44968/v1.pdf |
| best_oa_location.version | acceptedVersion |
| best_oa_location.raw_type | posted-content |
| best_oa_location.license_id | https://openalex.org/licenses/cc-by |
| best_oa_location.is_accepted | True |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | https://doi.org/10.21203/rs.3.rs-44968/v1 |
| primary_location.id | doi:10.21203/rs.3.rs-44968/v1 |
| primary_location.is_oa | True |
| primary_location.source | |
| primary_location.license | cc-by |
| primary_location.pdf_url | https://www.researchsquare.com/article/rs-44968/v1.pdf |
| primary_location.version | acceptedVersion |
| primary_location.raw_type | posted-content |
| primary_location.license_id | https://openalex.org/licenses/cc-by |
| primary_location.is_accepted | True |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | https://doi.org/10.21203/rs.3.rs-44968/v1 |
| publication_date | 2020-07-23 |
| publication_year | 2020 |
| referenced_works | https://openalex.org/W2332721970, https://openalex.org/W2218489740, https://openalex.org/W2912493354, https://openalex.org/W2925468094, https://openalex.org/W2419281044, https://openalex.org/W2613718673, https://openalex.org/W6735463952, https://openalex.org/W2969583230, https://openalex.org/W3100811978, https://openalex.org/W2555618208, https://openalex.org/W2150066425, https://openalex.org/W2892614179, https://openalex.org/W2924214398, https://openalex.org/W2920625195, https://openalex.org/W6760739867, https://openalex.org/W6639102338, https://openalex.org/W1927251054, https://openalex.org/W2337429362, https://openalex.org/W6698013645, https://openalex.org/W2949145768, https://openalex.org/W2059319345, https://openalex.org/W260801291, https://openalex.org/W2460807593, https://openalex.org/W6736750069, https://openalex.org/W2795276939, https://openalex.org/W2809539339, https://openalex.org/W2913562503, https://openalex.org/W2522212509, https://openalex.org/W2774839435, https://openalex.org/W3099155473, https://openalex.org/W2899807236, https://openalex.org/W2950642167, https://openalex.org/W2774996270, https://openalex.org/W2895946091, https://openalex.org/W2760327656, https://openalex.org/W1485037422, https://openalex.org/W1903029394, https://openalex.org/W2788336029, https://openalex.org/W2739253758, https://openalex.org/W2609822318, https://openalex.org/W2340897893, https://openalex.org/W2780038208, https://openalex.org/W6730757611, https://openalex.org/W2803740064, https://openalex.org/W6687483927, https://openalex.org/W6730903564, https://openalex.org/W1533861849, https://openalex.org/W6785652829, https://openalex.org/W2341497066, https://openalex.org/W2059286516, https://openalex.org/W2558027072, https://openalex.org/W2560609797, https://openalex.org/W2300469113, https://openalex.org/W2968847206, https://openalex.org/W3106250896, https://openalex.org/W2962888833, https://openalex.org/W639708223, https://openalex.org/W2194775991, https://openalex.org/W2609825896, https://openalex.org/W2963038646, https://openalex.org/W2565639579, https://openalex.org/W2963400571, https://openalex.org/W2993182889, https://openalex.org/W1861492603, https://openalex.org/W3099024080, https://openalex.org/W2964211168, https://openalex.org/W3035574168, https://openalex.org/W2963516811, https://openalex.org/W2970095196, https://openalex.org/W2953920664, https://openalex.org/W2967740791, https://openalex.org/W2395611524, https://openalex.org/W611457968, https://openalex.org/W2964027659 |
| referenced_works_count | 74 |
| abstract_inverted_index.an | 6 |
| abstract_inverted_index.in | 27, 74, 120 |
| abstract_inverted_index.is | 71, 87 |
| abstract_inverted_index.of | 10, 21, 54, 57 |
| abstract_inverted_index.to | 4, 15, 89 |
| abstract_inverted_index.we | 46, 99 |
| abstract_inverted_index.RGB | 60 |
| abstract_inverted_index.and | 19, 30, 62 |
| abstract_inverted_index.for | 25 |
| abstract_inverted_index.our | 37, 77, 108 |
| abstract_inverted_index.the | 17, 22, 41, 48, 63, 83, 92, 104, 116 |
| abstract_inverted_index.7481 | 55 |
| abstract_inverted_index.SMoE | 84, 110 |
| abstract_inverted_index.This | 1 |
| abstract_inverted_index.aims | 3 |
| abstract_inverted_index.fine | 72 |
| abstract_inverted_index.from | 40, 95 |
| abstract_inverted_index.make | 36 |
| abstract_inverted_index.over | 115 |
| abstract_inverted_index.show | 106 |
| abstract_inverted_index.that | 107 |
| abstract_inverted_index.this | 69 |
| abstract_inverted_index.with | 76 |
| abstract_inverted_index.work | 38 |
| abstract_inverted_index.CAEVs | 26 |
| abstract_inverted_index.KITTI | 50, 59 |
| abstract_inverted_index.LiDAR | 65 |
| abstract_inverted_index.Three | 33 |
| abstract_inverted_index.dense | 66 |
| abstract_inverted_index.depth | 67 |
| abstract_inverted_index.learn | 91 |
| abstract_inverted_index.maps, | 68 |
| abstract_inverted_index.other | 117 |
| abstract_inverted_index.pairs | 56 |
| abstract_inverted_index.paper | 2 |
| abstract_inverted_index.which | 52 |
| abstract_inverted_index.yield | 112 |
| abstract_inverted_index.(SMoE) | 12 |
| abstract_inverted_index.First, | 45 |
| abstract_inverted_index.fusion | 13, 85, 118 |
| abstract_inverted_index.images | 61 |
| abstract_inverted_index.robust | 93 |
| abstract_inverted_index.Complex | 49 |
| abstract_inverted_index.Second, | 82 |
| abstract_inverted_index.adverse | 121 |
| abstract_inverted_index.complex | 28 |
| abstract_inverted_index.dataset | 51, 70 |
| abstract_inverted_index.develop | 5 |
| abstract_inverted_index.devised | 88 |
| abstract_inverted_index.experts | 11 |
| abstract_inverted_index.improve | 16 |
| abstract_inverted_index.kernels | 94 |
| abstract_inverted_index.method. | 81 |
| abstract_inverted_index.mixture | 9 |
| abstract_inverted_index.results | 105 |
| abstract_inverted_index.systems | 24 |
| abstract_inverted_index.weather | 31 |
| abstract_inverted_index.Finally, | 98 |
| abstract_inverted_index.accuracy | 20 |
| abstract_inverted_index.approach | 86 |
| abstract_inverted_index.consists | 53 |
| abstract_inverted_index.existing | 42 |
| abstract_inverted_index.modified | 58 |
| abstract_inverted_index.original | 34 |
| abstract_inverted_index.proposed | 78, 109 |
| abstract_inverted_index.relevant | 43 |
| abstract_inverted_index.annotated | 73 |
| abstract_inverted_index.framework | 14, 111 |
| abstract_inverted_index.generated | 64 |
| abstract_inverted_index.implement | 100 |
| abstract_inverted_index.introduce | 47 |
| abstract_inverted_index.adaptively | 90 |
| abstract_inverted_index.annotation | 80 |
| abstract_inverted_index.end-to-end | 7 |
| abstract_inverted_index.perception | 23 |
| abstract_inverted_index.robustness | 18 |
| abstract_inverted_index.sharpening | 8 |
| abstract_inverted_index.techniques | 119 |
| abstract_inverted_index.comparative | 102 |
| abstract_inverted_index.conditions. | 32, 123 |
| abstract_inverted_index.distinctive | 39 |
| abstract_inverted_index.literature. | 44 |
| abstract_inverted_index.modalities. | 97 |
| abstract_inverted_index.significant | 113 |
| abstract_inverted_index.experiments, | 103 |
| abstract_inverted_index.illumination | 29 |
| abstract_inverted_index.improvements | 114 |
| abstract_inverted_index.complementary | 96 |
| abstract_inverted_index.comprehensive | 101 |
| abstract_inverted_index.contributions | 35 |
| abstract_inverted_index.environmental | 122 |
| abstract_inverted_index.instance-level | 75 |
| abstract_inverted_index.semi-automatic | 79 |
| abstract_inverted_index.<title>Abstract</title> | 0 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 6 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/15 |
| sustainable_development_goals[0].score | 0.6100000143051147 |
| sustainable_development_goals[0].display_name | Life in Land |
| citation_normalized_percentile.value | 0.30045273 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | False |