DIffSteISR: Harnessing Diffusion Prior for Superior Real-world Stereo Image Super-Resolution Article Swipe
Yuanbo Zhou
,
Xinlin Zhang
,
Wei Deng
,
Tao Wang
,
Tao Tan
,
Qinquan Gao
,
Tong Tong
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2408.07516
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2408.07516
We introduce DiffSteISR, a pioneering framework for reconstructing real-world stereo images. DiffSteISR utilizes the powerful prior knowledge embedded in pre-trained text-to-image model to efficiently recover the lost texture details in low-resolution stereo images. Specifically, DiffSteISR implements a time-aware stereo cross attention with temperature adapter (TASCATA) to guide the diffusion process, ensuring that the generated left and right views exhibit high texture consistency thereby reducing disparity error between the super-resolved images and the ground truth (GT) images. Additionally, a stereo omni attention control network (SOA ControlNet) is proposed to enhance the consistency of super-resolved images with GT images in the pixel, perceptual, and distribution space. Finally, DiffSteISR incorporates a stereo semantic extractor (SSE) to capture unique viewpoint soft semantic information and shared hard tag semantic information, thereby effectively improving the semantic accuracy and consistency of the generated left and right images. Extensive experimental results demonstrate that DiffSteISR accurately reconstructs natural and precise textures from low-resolution stereo images while maintaining a high consistency of semantic and texture between the left and right views.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2408.07516
- https://arxiv.org/pdf/2408.07516
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4402560445
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4402560445Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2408.07516Digital Object Identifier
- Title
-
DIffSteISR: Harnessing Diffusion Prior for Superior Real-world Stereo Image Super-ResolutionWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-08-14Full publication date if available
- Authors
-
Yuanbo Zhou, Xinlin Zhang, Wei Deng, Tao Wang, Tao Tan, Qinquan Gao, Tong TongList of authors in order
- Landing page
-
https://arxiv.org/abs/2408.07516Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2408.07516Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2408.07516Direct OA link when available
- Concepts
-
Computer vision, Diffusion, Superresolution, Computer science, Artificial intelligence, Image (mathematics), Resolution (logic), Physics, ThermodynamicsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4402560445 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2408.07516 |
| ids.doi | https://doi.org/10.48550/arxiv.2408.07516 |
| ids.openalex | https://openalex.org/W4402560445 |
| fwci | |
| type | preprint |
| title | DIffSteISR: Harnessing Diffusion Prior for Superior Real-world Stereo Image Super-Resolution |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T11105 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9991999864578247 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1707 |
| topics[0].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[0].display_name | Advanced Image Processing Techniques |
| topics[1].id | https://openalex.org/T10531 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9959999918937683 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1707 |
| topics[1].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[1].display_name | Advanced Vision and Imaging |
| topics[2].id | https://openalex.org/T13114 |
| topics[2].field.id | https://openalex.org/fields/22 |
| topics[2].field.display_name | Engineering |
| topics[2].score | 0.9958000183105469 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/2214 |
| topics[2].subfield.display_name | Media Technology |
| topics[2].display_name | Image Processing Techniques and Applications |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C31972630 |
| concepts[0].level | 1 |
| concepts[0].score | 0.5668525695800781 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q844240 |
| concepts[0].display_name | Computer vision |
| concepts[1].id | https://openalex.org/C69357855 |
| concepts[1].level | 2 |
| concepts[1].score | 0.5065580606460571 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q163214 |
| concepts[1].display_name | Diffusion |
| concepts[2].id | https://openalex.org/C141239990 |
| concepts[2].level | 3 |
| concepts[2].score | 0.4832773506641388 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q957423 |
| concepts[2].display_name | Superresolution |
| concepts[3].id | https://openalex.org/C41008148 |
| concepts[3].level | 0 |
| concepts[3].score | 0.4819912314414978 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[3].display_name | Computer science |
| concepts[4].id | https://openalex.org/C154945302 |
| concepts[4].level | 1 |
| concepts[4].score | 0.45439982414245605 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[4].display_name | Artificial intelligence |
| concepts[5].id | https://openalex.org/C115961682 |
| concepts[5].level | 2 |
| concepts[5].score | 0.4512863755226135 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q860623 |
| concepts[5].display_name | Image (mathematics) |
| concepts[6].id | https://openalex.org/C138268822 |
| concepts[6].level | 2 |
| concepts[6].score | 0.42701610922813416 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q1051925 |
| concepts[6].display_name | Resolution (logic) |
| concepts[7].id | https://openalex.org/C121332964 |
| concepts[7].level | 0 |
| concepts[7].score | 0.1464352309703827 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q413 |
| concepts[7].display_name | Physics |
| concepts[8].id | https://openalex.org/C97355855 |
| concepts[8].level | 1 |
| concepts[8].score | 0.0 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q11473 |
| concepts[8].display_name | Thermodynamics |
| keywords[0].id | https://openalex.org/keywords/computer-vision |
| keywords[0].score | 0.5668525695800781 |
| keywords[0].display_name | Computer vision |
| keywords[1].id | https://openalex.org/keywords/diffusion |
| keywords[1].score | 0.5065580606460571 |
| keywords[1].display_name | Diffusion |
| keywords[2].id | https://openalex.org/keywords/superresolution |
| keywords[2].score | 0.4832773506641388 |
| keywords[2].display_name | Superresolution |
| keywords[3].id | https://openalex.org/keywords/computer-science |
| keywords[3].score | 0.4819912314414978 |
| keywords[3].display_name | Computer science |
| keywords[4].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[4].score | 0.45439982414245605 |
| keywords[4].display_name | Artificial intelligence |
| keywords[5].id | https://openalex.org/keywords/image |
| keywords[5].score | 0.4512863755226135 |
| keywords[5].display_name | Image (mathematics) |
| keywords[6].id | https://openalex.org/keywords/resolution |
| keywords[6].score | 0.42701610922813416 |
| keywords[6].display_name | Resolution (logic) |
| keywords[7].id | https://openalex.org/keywords/physics |
| keywords[7].score | 0.1464352309703827 |
| keywords[7].display_name | Physics |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2408.07516 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2408.07516 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2408.07516 |
| locations[1].id | doi:10.48550/arxiv.2408.07516 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2408.07516 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5065946202 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-0136-3503 |
| authorships[0].author.display_name | Yuanbo Zhou |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Zhou, Yuanbo |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5101492717 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-1712-9241 |
| authorships[1].author.display_name | Xinlin Zhang |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Zhang, Xinlin |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5100426957 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-3671-5951 |
| authorships[2].author.display_name | Wei Deng |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Deng, Wei |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5100453495 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-2480-878X |
| authorships[3].author.display_name | Tao Wang |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Wang, Tao |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5101628586 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-5573-1105 |
| authorships[4].author.display_name | Tao Tan |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Tan, Tao |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5084595802 |
| authorships[5].author.orcid | https://orcid.org/0000-0003-0352-6027 |
| authorships[5].author.display_name | Qinquan Gao |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Gao, Qinquan |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5100745677 |
| authorships[6].author.orcid | https://orcid.org/0000-0003-0636-7359 |
| authorships[6].author.display_name | Tong Tong |
| authorships[6].author_position | last |
| authorships[6].raw_author_name | Tong, Tong |
| authorships[6].is_corresponding | False |
| has_content.pdf | True |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2408.07516 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-09-18T00:00:00 |
| display_name | DIffSteISR: Harnessing Diffusion Prior for Superior Real-world Stereo Image Super-Resolution |
| has_fulltext | True |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T11105 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9991999864578247 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1707 |
| primary_topic.subfield.display_name | Computer Vision and Pattern Recognition |
| primary_topic.display_name | Advanced Image Processing Techniques |
| related_works | https://openalex.org/W3216672236, https://openalex.org/W4250542427, https://openalex.org/W2216090681, https://openalex.org/W1517180214, https://openalex.org/W2082780921, https://openalex.org/W2025517136, https://openalex.org/W2143529858, https://openalex.org/W4367590696, https://openalex.org/W2056165575, https://openalex.org/W2358774654 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2408.07516 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2408.07516 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2408.07516 |
| primary_location.id | pmh:oai:arXiv.org:2408.07516 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2408.07516 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2408.07516 |
| publication_date | 2024-08-14 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 3, 36, 77, 107, 158 |
| abstract_inverted_index.GT | 95 |
| abstract_inverted_index.We | 0 |
| abstract_inverted_index.in | 18, 29, 97 |
| abstract_inverted_index.is | 85 |
| abstract_inverted_index.of | 91, 133, 161 |
| abstract_inverted_index.to | 22, 45, 87, 112 |
| abstract_inverted_index.and | 55, 70, 101, 119, 131, 137, 149, 163, 168 |
| abstract_inverted_index.for | 6 |
| abstract_inverted_index.tag | 122 |
| abstract_inverted_index.the | 13, 25, 47, 52, 67, 71, 89, 98, 128, 134, 166 |
| abstract_inverted_index.(GT) | 74 |
| abstract_inverted_index.(SOA | 83 |
| abstract_inverted_index.from | 152 |
| abstract_inverted_index.hard | 121 |
| abstract_inverted_index.high | 59, 159 |
| abstract_inverted_index.left | 54, 136, 167 |
| abstract_inverted_index.lost | 26 |
| abstract_inverted_index.omni | 79 |
| abstract_inverted_index.soft | 116 |
| abstract_inverted_index.that | 51, 144 |
| abstract_inverted_index.with | 41, 94 |
| abstract_inverted_index.(SSE) | 111 |
| abstract_inverted_index.cross | 39 |
| abstract_inverted_index.error | 65 |
| abstract_inverted_index.guide | 46 |
| abstract_inverted_index.model | 21 |
| abstract_inverted_index.prior | 15 |
| abstract_inverted_index.right | 56, 138, 169 |
| abstract_inverted_index.truth | 73 |
| abstract_inverted_index.views | 57 |
| abstract_inverted_index.while | 156 |
| abstract_inverted_index.ground | 72 |
| abstract_inverted_index.images | 69, 93, 96, 155 |
| abstract_inverted_index.pixel, | 99 |
| abstract_inverted_index.shared | 120 |
| abstract_inverted_index.space. | 103 |
| abstract_inverted_index.stereo | 9, 31, 38, 78, 108, 154 |
| abstract_inverted_index.unique | 114 |
| abstract_inverted_index.views. | 170 |
| abstract_inverted_index.adapter | 43 |
| abstract_inverted_index.between | 66, 165 |
| abstract_inverted_index.capture | 113 |
| abstract_inverted_index.control | 81 |
| abstract_inverted_index.details | 28 |
| abstract_inverted_index.enhance | 88 |
| abstract_inverted_index.exhibit | 58 |
| abstract_inverted_index.images. | 10, 32, 75, 139 |
| abstract_inverted_index.natural | 148 |
| abstract_inverted_index.network | 82 |
| abstract_inverted_index.precise | 150 |
| abstract_inverted_index.recover | 24 |
| abstract_inverted_index.results | 142 |
| abstract_inverted_index.texture | 27, 60, 164 |
| abstract_inverted_index.thereby | 62, 125 |
| abstract_inverted_index.Finally, | 104 |
| abstract_inverted_index.accuracy | 130 |
| abstract_inverted_index.embedded | 17 |
| abstract_inverted_index.ensuring | 50 |
| abstract_inverted_index.powerful | 14 |
| abstract_inverted_index.process, | 49 |
| abstract_inverted_index.proposed | 86 |
| abstract_inverted_index.reducing | 63 |
| abstract_inverted_index.semantic | 109, 117, 123, 129, 162 |
| abstract_inverted_index.textures | 151 |
| abstract_inverted_index.utilizes | 12 |
| abstract_inverted_index.(TASCATA) | 44 |
| abstract_inverted_index.Extensive | 140 |
| abstract_inverted_index.attention | 40, 80 |
| abstract_inverted_index.diffusion | 48 |
| abstract_inverted_index.disparity | 64 |
| abstract_inverted_index.extractor | 110 |
| abstract_inverted_index.framework | 5 |
| abstract_inverted_index.generated | 53, 135 |
| abstract_inverted_index.improving | 127 |
| abstract_inverted_index.introduce | 1 |
| abstract_inverted_index.knowledge | 16 |
| abstract_inverted_index.viewpoint | 115 |
| abstract_inverted_index.DiffSteISR | 11, 34, 105, 145 |
| abstract_inverted_index.accurately | 146 |
| abstract_inverted_index.implements | 35 |
| abstract_inverted_index.pioneering | 4 |
| abstract_inverted_index.real-world | 8 |
| abstract_inverted_index.time-aware | 37 |
| abstract_inverted_index.ControlNet) | 84 |
| abstract_inverted_index.DiffSteISR, | 2 |
| abstract_inverted_index.consistency | 61, 90, 132, 160 |
| abstract_inverted_index.demonstrate | 143 |
| abstract_inverted_index.effectively | 126 |
| abstract_inverted_index.efficiently | 23 |
| abstract_inverted_index.information | 118 |
| abstract_inverted_index.maintaining | 157 |
| abstract_inverted_index.perceptual, | 100 |
| abstract_inverted_index.pre-trained | 19 |
| abstract_inverted_index.temperature | 42 |
| abstract_inverted_index.distribution | 102 |
| abstract_inverted_index.experimental | 141 |
| abstract_inverted_index.incorporates | 106 |
| abstract_inverted_index.information, | 124 |
| abstract_inverted_index.reconstructs | 147 |
| abstract_inverted_index.Additionally, | 76 |
| abstract_inverted_index.Specifically, | 33 |
| abstract_inverted_index.text-to-image | 20 |
| abstract_inverted_index.low-resolution | 30, 153 |
| abstract_inverted_index.reconstructing | 7 |
| abstract_inverted_index.super-resolved | 68, 92 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 7 |
| citation_normalized_percentile |