Visuo-Tactile based Predictive Cross Modal Perception for Object Exploration in Robotics Article Swipe
Anirvan Dutta
,
Etienne Burdet
,
Mohsen Kaboli
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2405.12634
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2405.12634
Autonomously exploring the unknown physical properties of novel objects such as stiffness, mass, center of mass, friction coefficient, and shape is crucial for autonomous robotic systems operating continuously in unstructured environments. We introduce a novel visuo-tactile based predictive cross-modal perception framework where initial visual observations (shape) aid in obtaining an initial prior over the object properties (mass). The initial prior improves the efficiency of the object property estimation, which is autonomously inferred via interactive non-prehensile pushing and using a dual filtering approach. The inferred properties are then used to enhance the predictive capability of the cross-modal function efficiently by using a human-inspired `surprise' formulation. We evaluated our proposed framework in the real-robotic scenario, demonstrating superior performance.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2405.12634
- https://arxiv.org/pdf/2405.12634
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4398230218
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4398230218Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2405.12634Digital Object Identifier
- Title
-
Visuo-Tactile based Predictive Cross Modal Perception for Object Exploration in RoboticsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-05-21Full publication date if available
- Authors
-
Anirvan Dutta, Etienne Burdet, Mohsen KaboliList of authors in order
- Landing page
-
https://arxiv.org/abs/2405.12634Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2405.12634Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2405.12634Direct OA link when available
- Concepts
-
Artificial intelligence, Modal, Tactile perception, Robotics, Perception, Object (grammar), Computer science, Computer vision, Psychology, Robot, Neuroscience, Materials science, Polymer chemistryTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4398230218 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2405.12634 |
| ids.doi | https://doi.org/10.48550/arxiv.2405.12634 |
| ids.openalex | https://openalex.org/W4398230218 |
| fwci | 0.0 |
| type | preprint |
| title | Visuo-Tactile based Predictive Cross Modal Perception for Object Exploration in Robotics |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T11605 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9751999974250793 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1707 |
| topics[0].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[0].display_name | Visual Attention and Saliency Detection |
| topics[1].id | https://openalex.org/T10914 |
| topics[1].field.id | https://openalex.org/fields/28 |
| topics[1].field.display_name | Neuroscience |
| topics[1].score | 0.9686999917030334 |
| topics[1].domain.id | https://openalex.org/domains/1 |
| topics[1].domain.display_name | Life Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/2805 |
| topics[1].subfield.display_name | Cognitive Neuroscience |
| topics[1].display_name | Tactile and Sensory Interactions |
| topics[2].id | https://openalex.org/T10789 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9185000061988831 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1709 |
| topics[2].subfield.display_name | Human-Computer Interaction |
| topics[2].display_name | Interactive and Immersive Displays |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C154945302 |
| concepts[0].level | 1 |
| concepts[0].score | 0.6418696641921997 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[0].display_name | Artificial intelligence |
| concepts[1].id | https://openalex.org/C71139939 |
| concepts[1].level | 2 |
| concepts[1].score | 0.6048131585121155 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q910194 |
| concepts[1].display_name | Modal |
| concepts[2].id | https://openalex.org/C3017819093 |
| concepts[2].level | 3 |
| concepts[2].score | 0.5808171629905701 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q328835 |
| concepts[2].display_name | Tactile perception |
| concepts[3].id | https://openalex.org/C34413123 |
| concepts[3].level | 3 |
| concepts[3].score | 0.5527370572090149 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q170978 |
| concepts[3].display_name | Robotics |
| concepts[4].id | https://openalex.org/C26760741 |
| concepts[4].level | 2 |
| concepts[4].score | 0.5298920273780823 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q160402 |
| concepts[4].display_name | Perception |
| concepts[5].id | https://openalex.org/C2781238097 |
| concepts[5].level | 2 |
| concepts[5].score | 0.5091199278831482 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q175026 |
| concepts[5].display_name | Object (grammar) |
| concepts[6].id | https://openalex.org/C41008148 |
| concepts[6].level | 0 |
| concepts[6].score | 0.4947461783885956 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[6].display_name | Computer science |
| concepts[7].id | https://openalex.org/C31972630 |
| concepts[7].level | 1 |
| concepts[7].score | 0.3779591917991638 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q844240 |
| concepts[7].display_name | Computer vision |
| concepts[8].id | https://openalex.org/C15744967 |
| concepts[8].level | 0 |
| concepts[8].score | 0.2970924377441406 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q9418 |
| concepts[8].display_name | Psychology |
| concepts[9].id | https://openalex.org/C90509273 |
| concepts[9].level | 2 |
| concepts[9].score | 0.22453182935714722 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q11012 |
| concepts[9].display_name | Robot |
| concepts[10].id | https://openalex.org/C169760540 |
| concepts[10].level | 1 |
| concepts[10].score | 0.0786096453666687 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q207011 |
| concepts[10].display_name | Neuroscience |
| concepts[11].id | https://openalex.org/C192562407 |
| concepts[11].level | 0 |
| concepts[11].score | 0.0710824728012085 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q228736 |
| concepts[11].display_name | Materials science |
| concepts[12].id | https://openalex.org/C188027245 |
| concepts[12].level | 1 |
| concepts[12].score | 0.0 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q750446 |
| concepts[12].display_name | Polymer chemistry |
| keywords[0].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[0].score | 0.6418696641921997 |
| keywords[0].display_name | Artificial intelligence |
| keywords[1].id | https://openalex.org/keywords/modal |
| keywords[1].score | 0.6048131585121155 |
| keywords[1].display_name | Modal |
| keywords[2].id | https://openalex.org/keywords/tactile-perception |
| keywords[2].score | 0.5808171629905701 |
| keywords[2].display_name | Tactile perception |
| keywords[3].id | https://openalex.org/keywords/robotics |
| keywords[3].score | 0.5527370572090149 |
| keywords[3].display_name | Robotics |
| keywords[4].id | https://openalex.org/keywords/perception |
| keywords[4].score | 0.5298920273780823 |
| keywords[4].display_name | Perception |
| keywords[5].id | https://openalex.org/keywords/object |
| keywords[5].score | 0.5091199278831482 |
| keywords[5].display_name | Object (grammar) |
| keywords[6].id | https://openalex.org/keywords/computer-science |
| keywords[6].score | 0.4947461783885956 |
| keywords[6].display_name | Computer science |
| keywords[7].id | https://openalex.org/keywords/computer-vision |
| keywords[7].score | 0.3779591917991638 |
| keywords[7].display_name | Computer vision |
| keywords[8].id | https://openalex.org/keywords/psychology |
| keywords[8].score | 0.2970924377441406 |
| keywords[8].display_name | Psychology |
| keywords[9].id | https://openalex.org/keywords/robot |
| keywords[9].score | 0.22453182935714722 |
| keywords[9].display_name | Robot |
| keywords[10].id | https://openalex.org/keywords/neuroscience |
| keywords[10].score | 0.0786096453666687 |
| keywords[10].display_name | Neuroscience |
| keywords[11].id | https://openalex.org/keywords/materials-science |
| keywords[11].score | 0.0710824728012085 |
| keywords[11].display_name | Materials science |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2405.12634 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2405.12634 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2405.12634 |
| locations[1].id | pmh:oai:pure.tue.nl:openaire_cris_publications/df8c1eef-176f-4d9d-aafa-7a99063c7ecb |
| locations[1].is_oa | True |
| locations[1].source | |
| locations[1].license | other-oa |
| locations[1].pdf_url | |
| locations[1].version | submittedVersion |
| locations[1].raw_type | |
| locations[1].license_id | https://openalex.org/licenses/other-oa |
| locations[1].is_accepted | False |
| locations[1].is_published | False |
| locations[1].raw_source_name | Dutta , A , Burdet , E & Kaboli , M 2024 ' Visuo-Tactile based Predictive Cross Modal Perception for Object Exploration in Robotics ' arXiv.org . https://doi.org/10.48550/arXiv.2405.12634 |
| locations[1].landing_page_url | https://research.tue.nl/en/publications/df8c1eef-176f-4d9d-aafa-7a99063c7ecb |
| locations[2].id | doi:10.48550/arxiv.2405.12634 |
| locations[2].is_oa | True |
| locations[2].source.id | https://openalex.org/S4306400194 |
| locations[2].source.issn | |
| locations[2].source.type | repository |
| locations[2].source.is_oa | True |
| locations[2].source.issn_l | |
| locations[2].source.is_core | False |
| locations[2].source.is_in_doaj | False |
| locations[2].source.display_name | arXiv (Cornell University) |
| locations[2].source.host_organization | https://openalex.org/I205783295 |
| locations[2].source.host_organization_name | Cornell University |
| locations[2].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[2].license | |
| locations[2].pdf_url | |
| locations[2].version | |
| locations[2].raw_type | article |
| locations[2].license_id | |
| locations[2].is_accepted | False |
| locations[2].is_published | |
| locations[2].raw_source_name | |
| locations[2].landing_page_url | https://doi.org/10.48550/arxiv.2405.12634 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5073432337 |
| authorships[0].author.orcid | https://orcid.org/0000-0001-5857-4769 |
| authorships[0].author.display_name | Anirvan Dutta |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Dutta, Anirvan |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5025807459 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-2123-0185 |
| authorships[1].author.display_name | Etienne Burdet |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Burdet, Etienne |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5030520581 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-2320-5717 |
| authorships[2].author.display_name | Mohsen Kaboli |
| authorships[2].author_position | last |
| authorships[2].raw_author_name | Kaboli, Mohsen |
| authorships[2].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2405.12634 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Visuo-Tactile based Predictive Cross Modal Perception for Object Exploration in Robotics |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T11605 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9751999974250793 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1707 |
| primary_topic.subfield.display_name | Computer Vision and Pattern Recognition |
| primary_topic.display_name | Visual Attention and Saliency Detection |
| related_works | https://openalex.org/W1508899372, https://openalex.org/W2012658348, https://openalex.org/W2039460805, https://openalex.org/W4250956039, https://openalex.org/W4240485100, https://openalex.org/W4254970379, https://openalex.org/W2056130799, https://openalex.org/W2045758229, https://openalex.org/W405964254, https://openalex.org/W2297646692 |
| cited_by_count | 0 |
| locations_count | 3 |
| best_oa_location.id | pmh:oai:arXiv.org:2405.12634 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2405.12634 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2405.12634 |
| primary_location.id | pmh:oai:arXiv.org:2405.12634 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2405.12634 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2405.12634 |
| publication_date | 2024-05-21 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 33, 78, 100 |
| abstract_inverted_index.We | 31, 104 |
| abstract_inverted_index.an | 49 |
| abstract_inverted_index.as | 10 |
| abstract_inverted_index.by | 98 |
| abstract_inverted_index.in | 28, 47, 109 |
| abstract_inverted_index.is | 20, 69 |
| abstract_inverted_index.of | 6, 14, 63, 93 |
| abstract_inverted_index.to | 88 |
| abstract_inverted_index.The | 57, 82 |
| abstract_inverted_index.aid | 46 |
| abstract_inverted_index.and | 18, 76 |
| abstract_inverted_index.are | 85 |
| abstract_inverted_index.for | 22 |
| abstract_inverted_index.our | 106 |
| abstract_inverted_index.the | 2, 53, 61, 64, 90, 94, 110 |
| abstract_inverted_index.via | 72 |
| abstract_inverted_index.dual | 79 |
| abstract_inverted_index.over | 52 |
| abstract_inverted_index.such | 9 |
| abstract_inverted_index.then | 86 |
| abstract_inverted_index.used | 87 |
| abstract_inverted_index.based | 36 |
| abstract_inverted_index.mass, | 12, 15 |
| abstract_inverted_index.novel | 7, 34 |
| abstract_inverted_index.prior | 51, 59 |
| abstract_inverted_index.shape | 19 |
| abstract_inverted_index.using | 77, 99 |
| abstract_inverted_index.where | 41 |
| abstract_inverted_index.which | 68 |
| abstract_inverted_index.center | 13 |
| abstract_inverted_index.object | 54, 65 |
| abstract_inverted_index.visual | 43 |
| abstract_inverted_index.(mass). | 56 |
| abstract_inverted_index.(shape) | 45 |
| abstract_inverted_index.crucial | 21 |
| abstract_inverted_index.enhance | 89 |
| abstract_inverted_index.initial | 42, 50, 58 |
| abstract_inverted_index.objects | 8 |
| abstract_inverted_index.pushing | 75 |
| abstract_inverted_index.robotic | 24 |
| abstract_inverted_index.systems | 25 |
| abstract_inverted_index.unknown | 3 |
| abstract_inverted_index.friction | 16 |
| abstract_inverted_index.function | 96 |
| abstract_inverted_index.improves | 60 |
| abstract_inverted_index.inferred | 71, 83 |
| abstract_inverted_index.physical | 4 |
| abstract_inverted_index.property | 66 |
| abstract_inverted_index.proposed | 107 |
| abstract_inverted_index.superior | 114 |
| abstract_inverted_index.approach. | 81 |
| abstract_inverted_index.evaluated | 105 |
| abstract_inverted_index.exploring | 1 |
| abstract_inverted_index.filtering | 80 |
| abstract_inverted_index.framework | 40, 108 |
| abstract_inverted_index.introduce | 32 |
| abstract_inverted_index.obtaining | 48 |
| abstract_inverted_index.operating | 26 |
| abstract_inverted_index.scenario, | 112 |
| abstract_inverted_index.`surprise' | 102 |
| abstract_inverted_index.autonomous | 23 |
| abstract_inverted_index.capability | 92 |
| abstract_inverted_index.efficiency | 62 |
| abstract_inverted_index.perception | 39 |
| abstract_inverted_index.predictive | 37, 91 |
| abstract_inverted_index.properties | 5, 55, 84 |
| abstract_inverted_index.stiffness, | 11 |
| abstract_inverted_index.cross-modal | 38, 95 |
| abstract_inverted_index.efficiently | 97 |
| abstract_inverted_index.estimation, | 67 |
| abstract_inverted_index.interactive | 73 |
| abstract_inverted_index.Autonomously | 0 |
| abstract_inverted_index.autonomously | 70 |
| abstract_inverted_index.coefficient, | 17 |
| abstract_inverted_index.continuously | 27 |
| abstract_inverted_index.formulation. | 103 |
| abstract_inverted_index.observations | 44 |
| abstract_inverted_index.performance. | 115 |
| abstract_inverted_index.real-robotic | 111 |
| abstract_inverted_index.unstructured | 29 |
| abstract_inverted_index.demonstrating | 113 |
| abstract_inverted_index.environments. | 30 |
| abstract_inverted_index.visuo-tactile | 35 |
| abstract_inverted_index.human-inspired | 101 |
| abstract_inverted_index.non-prehensile | 74 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 3 |
| citation_normalized_percentile |