MIKE: A New Benchmark for Fine-grained Multimodal Entity Knowledge Editing Article Swipe
Jiaqi Li
,
Miaozeng Du
,
Chuanyi Zhang
,
Yongrui Chen
,
Nan Hu
,
Guilin Qi
,
Haiyun Jiang
,
Siyuan Cheng
,
Bozhong Tian
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2402.14835
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2402.14835
Multimodal knowledge editing represents a critical advancement in enhancing the capabilities of Multimodal Large Language Models (MLLMs). Despite its potential, current benchmarks predominantly focus on coarse-grained knowledge, leaving the intricacies of fine-grained (FG) multimodal entity knowledge largely unexplored. This gap presents a notable challenge, as FG entity recognition is pivotal for the practical deployment and effectiveness of MLLMs in diverse real-world scenarios. To bridge this gap, we introduce MIKE, a comprehensive benchmark and dataset specifically designed for the FG multimodal entity knowledge editing. MIKE encompasses a suite of tasks tailored to assess different perspectives, including Vanilla Name Answering, Entity-Level Caption, and Complex-Scenario Recognition. In addition, a new form of knowledge editing, Multi-step Editing, is introduced to evaluate the editing efficiency. Through our extensive evaluations, we demonstrate that the current state-of-the-art methods face significant challenges in tackling our proposed benchmark, underscoring the complexity of FG knowledge editing in MLLMs. Our findings spotlight the urgent need for novel approaches in this domain, setting a clear agenda for future research and development efforts within the community.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2402.14835
- https://arxiv.org/pdf/2402.14835
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4392181661
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4392181661Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2402.14835Digital Object Identifier
- Title
-
MIKE: A New Benchmark for Fine-grained Multimodal Entity Knowledge EditingWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-02-18Full publication date if available
- Authors
-
Jiaqi Li, Miaozeng Du, Chuanyi Zhang, Yongrui Chen, Nan Hu, Guilin Qi, Haiyun Jiang, Siyuan Cheng, Bozhong TianList of authors in order
- Landing page
-
https://arxiv.org/abs/2402.14835Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2402.14835Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2402.14835Direct OA link when available
- Concepts
-
Benchmark (surveying), Computer science, Artificial intelligence, Natural language processing, Data science, Geography, CartographyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4392181661 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2402.14835 |
| ids.doi | https://doi.org/10.48550/arxiv.2402.14835 |
| ids.openalex | https://openalex.org/W4392181661 |
| fwci | |
| type | preprint |
| title | MIKE: A New Benchmark for Fine-grained Multimodal Entity Knowledge Editing |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10181 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9708999991416931 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Natural Language Processing Techniques |
| topics[1].id | https://openalex.org/T10679 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.95660001039505 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1710 |
| topics[1].subfield.display_name | Information Systems |
| topics[1].display_name | Service-Oriented Architecture and Web Services |
| topics[2].id | https://openalex.org/T10215 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9434999823570251 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Semantic Web and Ontologies |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C185798385 |
| concepts[0].level | 2 |
| concepts[0].score | 0.8255147337913513 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q1161707 |
| concepts[0].display_name | Benchmark (surveying) |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.6673808097839355 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C154945302 |
| concepts[2].level | 1 |
| concepts[2].score | 0.41141384840011597 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[2].display_name | Artificial intelligence |
| concepts[3].id | https://openalex.org/C204321447 |
| concepts[3].level | 1 |
| concepts[3].score | 0.3423030972480774 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[3].display_name | Natural language processing |
| concepts[4].id | https://openalex.org/C2522767166 |
| concepts[4].level | 1 |
| concepts[4].score | 0.3409472703933716 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q2374463 |
| concepts[4].display_name | Data science |
| concepts[5].id | https://openalex.org/C205649164 |
| concepts[5].level | 0 |
| concepts[5].score | 0.09253683686256409 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q1071 |
| concepts[5].display_name | Geography |
| concepts[6].id | https://openalex.org/C58640448 |
| concepts[6].level | 1 |
| concepts[6].score | 0.08997038006782532 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q42515 |
| concepts[6].display_name | Cartography |
| keywords[0].id | https://openalex.org/keywords/benchmark |
| keywords[0].score | 0.8255147337913513 |
| keywords[0].display_name | Benchmark (surveying) |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.6673808097839355 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[2].score | 0.41141384840011597 |
| keywords[2].display_name | Artificial intelligence |
| keywords[3].id | https://openalex.org/keywords/natural-language-processing |
| keywords[3].score | 0.3423030972480774 |
| keywords[3].display_name | Natural language processing |
| keywords[4].id | https://openalex.org/keywords/data-science |
| keywords[4].score | 0.3409472703933716 |
| keywords[4].display_name | Data science |
| keywords[5].id | https://openalex.org/keywords/geography |
| keywords[5].score | 0.09253683686256409 |
| keywords[5].display_name | Geography |
| keywords[6].id | https://openalex.org/keywords/cartography |
| keywords[6].score | 0.08997038006782532 |
| keywords[6].display_name | Cartography |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2402.14835 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2402.14835 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2402.14835 |
| locations[1].id | doi:10.48550/arxiv.2402.14835 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2402.14835 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5015006932 |
| authorships[0].author.orcid | https://orcid.org/0009-0004-3393-5774 |
| authorships[0].author.display_name | Jiaqi Li |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Li, Jiaqi |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5100585277 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Miaozeng Du |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Du, Miaozeng |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5101465467 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-8724-5796 |
| authorships[2].author.display_name | Chuanyi Zhang |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Zhang, Chuanyi |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5103157273 |
| authorships[3].author.orcid | https://orcid.org/0000-0001-8934-3920 |
| authorships[3].author.display_name | Yongrui Chen |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Chen, Yongrui |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5020417364 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-9377-3904 |
| authorships[4].author.display_name | Nan Hu |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Hu, Nan |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5034606659 |
| authorships[5].author.orcid | https://orcid.org/0000-0003-0150-7236 |
| authorships[5].author.display_name | Guilin Qi |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Qi, Guilin |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5081262817 |
| authorships[6].author.orcid | https://orcid.org/0000-0001-7623-8902 |
| authorships[6].author.display_name | Haiyun Jiang |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Jiang, Haiyun |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5101863818 |
| authorships[7].author.orcid | https://orcid.org/0009-0008-2230-6875 |
| authorships[7].author.display_name | Siyuan Cheng |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Cheng, Siyuan |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5086969601 |
| authorships[8].author.orcid | https://orcid.org/0000-0001-6907-8496 |
| authorships[8].author.display_name | Bozhong Tian |
| authorships[8].author_position | last |
| authorships[8].raw_author_name | Tian, Bozhong |
| authorships[8].is_corresponding | False |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2402.14835 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-02-27T00:00:00 |
| display_name | MIKE: A New Benchmark for Fine-grained Multimodal Entity Knowledge Editing |
| has_fulltext | True |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10181 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9708999991416931 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Natural Language Processing Techniques |
| related_works | https://openalex.org/W2748952813, https://openalex.org/W2378211422, https://openalex.org/W2745001401, https://openalex.org/W4321353415, https://openalex.org/W2130974462, https://openalex.org/W972276598, https://openalex.org/W4246352526, https://openalex.org/W2028665553, https://openalex.org/W2086519370, https://openalex.org/W3204019825 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2402.14835 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2402.14835 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2402.14835 |
| primary_location.id | pmh:oai:arXiv.org:2402.14835 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2402.14835 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2402.14835 |
| publication_date | 2024-02-18 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 4, 41, 69, 85, 105, 161 |
| abstract_inverted_index.FG | 45, 78, 143 |
| abstract_inverted_index.In | 103 |
| abstract_inverted_index.To | 62 |
| abstract_inverted_index.as | 44 |
| abstract_inverted_index.in | 7, 58, 134, 146, 157 |
| abstract_inverted_index.is | 48, 113 |
| abstract_inverted_index.of | 11, 30, 56, 87, 108, 142 |
| abstract_inverted_index.on | 24 |
| abstract_inverted_index.to | 90, 115 |
| abstract_inverted_index.we | 66, 124 |
| abstract_inverted_index.Our | 148 |
| abstract_inverted_index.and | 54, 72, 100, 167 |
| abstract_inverted_index.for | 50, 76, 154, 164 |
| abstract_inverted_index.gap | 39 |
| abstract_inverted_index.its | 18 |
| abstract_inverted_index.new | 106 |
| abstract_inverted_index.our | 121, 136 |
| abstract_inverted_index.the | 9, 28, 51, 77, 117, 127, 140, 151, 171 |
| abstract_inverted_index.(FG) | 32 |
| abstract_inverted_index.MIKE | 83 |
| abstract_inverted_index.Name | 96 |
| abstract_inverted_index.This | 38 |
| abstract_inverted_index.face | 131 |
| abstract_inverted_index.form | 107 |
| abstract_inverted_index.gap, | 65 |
| abstract_inverted_index.need | 153 |
| abstract_inverted_index.that | 126 |
| abstract_inverted_index.this | 64, 158 |
| abstract_inverted_index.Large | 13 |
| abstract_inverted_index.MIKE, | 68 |
| abstract_inverted_index.MLLMs | 57 |
| abstract_inverted_index.clear | 162 |
| abstract_inverted_index.focus | 23 |
| abstract_inverted_index.novel | 155 |
| abstract_inverted_index.suite | 86 |
| abstract_inverted_index.tasks | 88 |
| abstract_inverted_index.MLLMs. | 147 |
| abstract_inverted_index.Models | 15 |
| abstract_inverted_index.agenda | 163 |
| abstract_inverted_index.assess | 91 |
| abstract_inverted_index.bridge | 63 |
| abstract_inverted_index.entity | 34, 46, 80 |
| abstract_inverted_index.future | 165 |
| abstract_inverted_index.urgent | 152 |
| abstract_inverted_index.within | 170 |
| abstract_inverted_index.Despite | 17 |
| abstract_inverted_index.Through | 120 |
| abstract_inverted_index.Vanilla | 95 |
| abstract_inverted_index.current | 20, 128 |
| abstract_inverted_index.dataset | 73 |
| abstract_inverted_index.diverse | 59 |
| abstract_inverted_index.domain, | 159 |
| abstract_inverted_index.editing | 2, 118, 145 |
| abstract_inverted_index.efforts | 169 |
| abstract_inverted_index.largely | 36 |
| abstract_inverted_index.leaving | 27 |
| abstract_inverted_index.methods | 130 |
| abstract_inverted_index.notable | 42 |
| abstract_inverted_index.pivotal | 49 |
| abstract_inverted_index.setting | 160 |
| abstract_inverted_index.(MLLMs). | 16 |
| abstract_inverted_index.Caption, | 99 |
| abstract_inverted_index.Editing, | 112 |
| abstract_inverted_index.Language | 14 |
| abstract_inverted_index.critical | 5 |
| abstract_inverted_index.designed | 75 |
| abstract_inverted_index.editing, | 110 |
| abstract_inverted_index.editing. | 82 |
| abstract_inverted_index.evaluate | 116 |
| abstract_inverted_index.findings | 149 |
| abstract_inverted_index.presents | 40 |
| abstract_inverted_index.proposed | 137 |
| abstract_inverted_index.research | 166 |
| abstract_inverted_index.tackling | 135 |
| abstract_inverted_index.tailored | 89 |
| abstract_inverted_index.addition, | 104 |
| abstract_inverted_index.benchmark | 71 |
| abstract_inverted_index.different | 92 |
| abstract_inverted_index.enhancing | 8 |
| abstract_inverted_index.extensive | 122 |
| abstract_inverted_index.including | 94 |
| abstract_inverted_index.introduce | 67 |
| abstract_inverted_index.knowledge | 1, 35, 81, 109, 144 |
| abstract_inverted_index.practical | 52 |
| abstract_inverted_index.spotlight | 150 |
| abstract_inverted_index.Answering, | 97 |
| abstract_inverted_index.Multi-step | 111 |
| abstract_inverted_index.Multimodal | 0, 12 |
| abstract_inverted_index.approaches | 156 |
| abstract_inverted_index.benchmark, | 138 |
| abstract_inverted_index.benchmarks | 21 |
| abstract_inverted_index.challenge, | 43 |
| abstract_inverted_index.challenges | 133 |
| abstract_inverted_index.community. | 172 |
| abstract_inverted_index.complexity | 141 |
| abstract_inverted_index.deployment | 53 |
| abstract_inverted_index.introduced | 114 |
| abstract_inverted_index.knowledge, | 26 |
| abstract_inverted_index.multimodal | 33, 79 |
| abstract_inverted_index.potential, | 19 |
| abstract_inverted_index.real-world | 60 |
| abstract_inverted_index.represents | 3 |
| abstract_inverted_index.scenarios. | 61 |
| abstract_inverted_index.advancement | 6 |
| abstract_inverted_index.demonstrate | 125 |
| abstract_inverted_index.development | 168 |
| abstract_inverted_index.efficiency. | 119 |
| abstract_inverted_index.encompasses | 84 |
| abstract_inverted_index.intricacies | 29 |
| abstract_inverted_index.recognition | 47 |
| abstract_inverted_index.significant | 132 |
| abstract_inverted_index.unexplored. | 37 |
| abstract_inverted_index.Entity-Level | 98 |
| abstract_inverted_index.Recognition. | 102 |
| abstract_inverted_index.capabilities | 10 |
| abstract_inverted_index.evaluations, | 123 |
| abstract_inverted_index.fine-grained | 31 |
| abstract_inverted_index.specifically | 74 |
| abstract_inverted_index.underscoring | 139 |
| abstract_inverted_index.comprehensive | 70 |
| abstract_inverted_index.effectiveness | 55 |
| abstract_inverted_index.perspectives, | 93 |
| abstract_inverted_index.predominantly | 22 |
| abstract_inverted_index.coarse-grained | 25 |
| abstract_inverted_index.Complex-Scenario | 101 |
| abstract_inverted_index.state-of-the-art | 129 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 9 |
| citation_normalized_percentile |