Synth4Seg -- Learning Defect Data Synthesis for Defect Segmentation using Bi-level Optimization Article Swipe
Shancong Mou
,
Raviteja Vemulapalli
,
Shiyu Li
,
Yuxuan Liu
,
Christopher W. Thomas
,
Meng Cao
,
Haoping Bai
,
Oncel Tuzel
,
Ping Huang
,
Jiulong Shan
,
Jianjun Shi
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2410.18490
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2410.18490
Defect segmentation is crucial for quality control in advanced manufacturing, yet data scarcity poses challenges for state-of-the-art supervised deep learning. Synthetic defect data generation is a popular approach for mitigating data challenges. However, many current methods simply generate defects following a fixed set of rules, which may not directly relate to downstream task performance. This can lead to suboptimal performance and may even hinder the downstream task. To solve this problem, we leverage a novel bi-level optimization-based synthetic defect data generation framework. We use an online synthetic defect generation module grounded in the commonly-used Cut\&Paste framework, and adopt an efficient gradient-based optimization algorithm to solve the bi-level optimization problem. We achieve simultaneous training of the defect segmentation network, and learn various parameters of the data synthesis module by maximizing the validation performance of the trained defect segmentation network. Our experimental results on benchmark datasets under limited data settings show that the proposed bi-level optimization method can be used for learning the most effective locations for pasting synthetic defects thereby improving the segmentation performance by up to 18.3\% when compared to pasting defects at random locations. We also demonstrate up to 2.6\% performance gain by learning the importance weights for different augmentation-specific defect data sources when compared to giving equal importance to all the data sources.
Related Topics
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2410.18490
- https://arxiv.org/pdf/2410.18490
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4404306553
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4404306553Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2410.18490Digital Object Identifier
- Title
-
Synth4Seg -- Learning Defect Data Synthesis for Defect Segmentation using Bi-level OptimizationWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-10-24Full publication date if available
- Authors
-
Shancong Mou, Raviteja Vemulapalli, Shiyu Li, Yuxuan Liu, Christopher W. Thomas, Meng Cao, Haoping Bai, Oncel Tuzel, Ping Huang, Jiulong Shan, Jianjun ShiList of authors in order
- Landing page
-
https://arxiv.org/abs/2410.18490Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2410.18490Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2410.18490Direct OA link when available
- Concepts
-
Segmentation, Computer science, Artificial intelligence, Pattern recognition (psychology)Top concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4404306553 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2410.18490 |
| ids.doi | https://doi.org/10.48550/arxiv.2410.18490 |
| ids.openalex | https://openalex.org/W4404306553 |
| fwci | |
| type | preprint |
| title | Synth4Seg -- Learning Defect Data Synthesis for Defect Segmentation using Bi-level Optimization |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T12111 |
| topics[0].field.id | https://openalex.org/fields/22 |
| topics[0].field.display_name | Engineering |
| topics[0].score | 0.9970999956130981 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/2209 |
| topics[0].subfield.display_name | Industrial and Manufacturing Engineering |
| topics[0].display_name | Industrial Vision Systems and Defect Detection |
| topics[1].id | https://openalex.org/T11159 |
| topics[1].field.id | https://openalex.org/fields/22 |
| topics[1].field.display_name | Engineering |
| topics[1].score | 0.9872999787330627 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/2209 |
| topics[1].subfield.display_name | Industrial and Manufacturing Engineering |
| topics[1].display_name | Manufacturing Process and Optimization |
| topics[2].id | https://openalex.org/T11301 |
| topics[2].field.id | https://openalex.org/fields/22 |
| topics[2].field.display_name | Engineering |
| topics[2].score | 0.9606999754905701 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/2204 |
| topics[2].subfield.display_name | Biomedical Engineering |
| topics[2].display_name | Advanced Surface Polishing Techniques |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C89600930 |
| concepts[0].level | 2 |
| concepts[0].score | 0.6572405099868774 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q1423946 |
| concepts[0].display_name | Segmentation |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.4755382537841797 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C154945302 |
| concepts[2].level | 1 |
| concepts[2].score | 0.4410534203052521 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[2].display_name | Artificial intelligence |
| concepts[3].id | https://openalex.org/C153180895 |
| concepts[3].level | 2 |
| concepts[3].score | 0.34084421396255493 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q7148389 |
| concepts[3].display_name | Pattern recognition (psychology) |
| keywords[0].id | https://openalex.org/keywords/segmentation |
| keywords[0].score | 0.6572405099868774 |
| keywords[0].display_name | Segmentation |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.4755382537841797 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[2].score | 0.4410534203052521 |
| keywords[2].display_name | Artificial intelligence |
| keywords[3].id | https://openalex.org/keywords/pattern-recognition |
| keywords[3].score | 0.34084421396255493 |
| keywords[3].display_name | Pattern recognition (psychology) |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2410.18490 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2410.18490 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2410.18490 |
| locations[1].id | doi:10.48550/arxiv.2410.18490 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2410.18490 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5067794264 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-2481-3308 |
| authorships[0].author.display_name | Shancong Mou |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Mou, Shancong |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5071825172 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-0425-7797 |
| authorships[1].author.display_name | Raviteja Vemulapalli |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Vemulapalli, Raviteja |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5100413190 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-1990-7150 |
| authorships[2].author.display_name | Shiyu Li |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Li, Shiyu |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5114734508 |
| authorships[3].author.orcid | |
| authorships[3].author.display_name | Yuxuan Liu |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Liu, Yuxuan |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5052690772 |
| authorships[4].author.orcid | https://orcid.org/0000-0001-5371-7257 |
| authorships[4].author.display_name | Christopher W. Thomas |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Thomas, C |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5041610421 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-1008-5509 |
| authorships[5].author.display_name | Meng Cao |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Cao, Meng |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5025733574 |
| authorships[6].author.orcid | |
| authorships[6].author.display_name | Haoping Bai |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Bai, Haoping |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5028613002 |
| authorships[7].author.orcid | |
| authorships[7].author.display_name | Oncel Tuzel |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Tuzel, Oncel |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5042793025 |
| authorships[8].author.orcid | https://orcid.org/0000-0002-5295-8216 |
| authorships[8].author.display_name | Ping Huang |
| authorships[8].author_position | middle |
| authorships[8].raw_author_name | Huang, Ping |
| authorships[8].is_corresponding | False |
| authorships[9].author.id | https://openalex.org/A5068249077 |
| authorships[9].author.orcid | |
| authorships[9].author.display_name | Jiulong Shan |
| authorships[9].author_position | middle |
| authorships[9].raw_author_name | Shan, Jiulong |
| authorships[9].is_corresponding | False |
| authorships[10].author.id | https://openalex.org/A5022416209 |
| authorships[10].author.orcid | https://orcid.org/0000-0002-3774-9176 |
| authorships[10].author.display_name | Jianjun Shi |
| authorships[10].author_position | last |
| authorships[10].raw_author_name | Shi, Jianjun |
| authorships[10].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2410.18490 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-11-13T00:00:00 |
| display_name | Synth4Seg -- Learning Defect Data Synthesis for Defect Segmentation using Bi-level Optimization |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T12111 |
| primary_topic.field.id | https://openalex.org/fields/22 |
| primary_topic.field.display_name | Engineering |
| primary_topic.score | 0.9970999956130981 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/2209 |
| primary_topic.subfield.display_name | Industrial and Manufacturing Engineering |
| primary_topic.display_name | Industrial Vision Systems and Defect Detection |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W4391913857, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2033914206, https://openalex.org/W2042327336 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2410.18490 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2410.18490 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2410.18490 |
| primary_location.id | pmh:oai:arXiv.org:2410.18490 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2410.18490 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2410.18490 |
| publication_date | 2024-10-24 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 25, 40, 73 |
| abstract_inverted_index.To | 67 |
| abstract_inverted_index.We | 82, 109, 185 |
| abstract_inverted_index.an | 84, 98 |
| abstract_inverted_index.at | 182 |
| abstract_inverted_index.be | 156 |
| abstract_inverted_index.by | 127, 173, 193 |
| abstract_inverted_index.in | 7, 91 |
| abstract_inverted_index.is | 2, 24 |
| abstract_inverted_index.of | 43, 113, 122, 132 |
| abstract_inverted_index.on | 141 |
| abstract_inverted_index.to | 50, 57, 103, 175, 179, 189, 206, 210 |
| abstract_inverted_index.up | 174, 188 |
| abstract_inverted_index.we | 71 |
| abstract_inverted_index.Our | 138 |
| abstract_inverted_index.all | 211 |
| abstract_inverted_index.and | 60, 96, 118 |
| abstract_inverted_index.can | 55, 155 |
| abstract_inverted_index.for | 4, 15, 28, 158, 164, 198 |
| abstract_inverted_index.may | 46, 61 |
| abstract_inverted_index.not | 47 |
| abstract_inverted_index.set | 42 |
| abstract_inverted_index.the | 64, 92, 105, 114, 123, 129, 133, 150, 160, 170, 195, 212 |
| abstract_inverted_index.use | 83 |
| abstract_inverted_index.yet | 10 |
| abstract_inverted_index.This | 54 |
| abstract_inverted_index.also | 186 |
| abstract_inverted_index.data | 11, 22, 30, 79, 124, 146, 202, 213 |
| abstract_inverted_index.deep | 18 |
| abstract_inverted_index.even | 62 |
| abstract_inverted_index.gain | 192 |
| abstract_inverted_index.lead | 56 |
| abstract_inverted_index.many | 33 |
| abstract_inverted_index.most | 161 |
| abstract_inverted_index.show | 148 |
| abstract_inverted_index.task | 52 |
| abstract_inverted_index.that | 149 |
| abstract_inverted_index.this | 69 |
| abstract_inverted_index.used | 157 |
| abstract_inverted_index.when | 177, 204 |
| abstract_inverted_index.2.6\% | 190 |
| abstract_inverted_index.adopt | 97 |
| abstract_inverted_index.equal | 208 |
| abstract_inverted_index.fixed | 41 |
| abstract_inverted_index.learn | 119 |
| abstract_inverted_index.novel | 74 |
| abstract_inverted_index.poses | 13 |
| abstract_inverted_index.solve | 68, 104 |
| abstract_inverted_index.task. | 66 |
| abstract_inverted_index.under | 144 |
| abstract_inverted_index.which | 45 |
| abstract_inverted_index.18.3\% | 176 |
| abstract_inverted_index.Defect | 0 |
| abstract_inverted_index.defect | 21, 78, 87, 115, 135, 201 |
| abstract_inverted_index.giving | 207 |
| abstract_inverted_index.hinder | 63 |
| abstract_inverted_index.method | 154 |
| abstract_inverted_index.module | 89, 126 |
| abstract_inverted_index.online | 85 |
| abstract_inverted_index.random | 183 |
| abstract_inverted_index.relate | 49 |
| abstract_inverted_index.rules, | 44 |
| abstract_inverted_index.simply | 36 |
| abstract_inverted_index.achieve | 110 |
| abstract_inverted_index.control | 6 |
| abstract_inverted_index.crucial | 3 |
| abstract_inverted_index.current | 34 |
| abstract_inverted_index.defects | 38, 167, 181 |
| abstract_inverted_index.limited | 145 |
| abstract_inverted_index.methods | 35 |
| abstract_inverted_index.pasting | 165, 180 |
| abstract_inverted_index.popular | 26 |
| abstract_inverted_index.quality | 5 |
| abstract_inverted_index.results | 140 |
| abstract_inverted_index.sources | 203 |
| abstract_inverted_index.thereby | 168 |
| abstract_inverted_index.trained | 134 |
| abstract_inverted_index.various | 120 |
| abstract_inverted_index.weights | 197 |
| abstract_inverted_index.However, | 32 |
| abstract_inverted_index.advanced | 8 |
| abstract_inverted_index.approach | 27 |
| abstract_inverted_index.bi-level | 75, 106, 152 |
| abstract_inverted_index.compared | 178, 205 |
| abstract_inverted_index.datasets | 143 |
| abstract_inverted_index.directly | 48 |
| abstract_inverted_index.generate | 37 |
| abstract_inverted_index.grounded | 90 |
| abstract_inverted_index.learning | 159, 194 |
| abstract_inverted_index.leverage | 72 |
| abstract_inverted_index.network, | 117 |
| abstract_inverted_index.network. | 137 |
| abstract_inverted_index.problem, | 70 |
| abstract_inverted_index.problem. | 108 |
| abstract_inverted_index.proposed | 151 |
| abstract_inverted_index.scarcity | 12 |
| abstract_inverted_index.settings | 147 |
| abstract_inverted_index.sources. | 214 |
| abstract_inverted_index.training | 112 |
| abstract_inverted_index.Synthetic | 20 |
| abstract_inverted_index.algorithm | 102 |
| abstract_inverted_index.benchmark | 142 |
| abstract_inverted_index.different | 199 |
| abstract_inverted_index.effective | 162 |
| abstract_inverted_index.efficient | 99 |
| abstract_inverted_index.following | 39 |
| abstract_inverted_index.improving | 169 |
| abstract_inverted_index.learning. | 19 |
| abstract_inverted_index.locations | 163 |
| abstract_inverted_index.synthesis | 125 |
| abstract_inverted_index.synthetic | 77, 86, 166 |
| abstract_inverted_index.challenges | 14 |
| abstract_inverted_index.downstream | 51, 65 |
| abstract_inverted_index.framework, | 95 |
| abstract_inverted_index.framework. | 81 |
| abstract_inverted_index.generation | 23, 80, 88 |
| abstract_inverted_index.importance | 196, 209 |
| abstract_inverted_index.locations. | 184 |
| abstract_inverted_index.maximizing | 128 |
| abstract_inverted_index.mitigating | 29 |
| abstract_inverted_index.parameters | 121 |
| abstract_inverted_index.suboptimal | 58 |
| abstract_inverted_index.supervised | 17 |
| abstract_inverted_index.validation | 130 |
| abstract_inverted_index.challenges. | 31 |
| abstract_inverted_index.demonstrate | 187 |
| abstract_inverted_index.performance | 59, 131, 172, 191 |
| abstract_inverted_index.experimental | 139 |
| abstract_inverted_index.optimization | 101, 107, 153 |
| abstract_inverted_index.performance. | 53 |
| abstract_inverted_index.segmentation | 1, 116, 136, 171 |
| abstract_inverted_index.simultaneous | 111 |
| abstract_inverted_index.commonly-used | 93 |
| abstract_inverted_index.Cut\&Paste | 94 |
| abstract_inverted_index.gradient-based | 100 |
| abstract_inverted_index.manufacturing, | 9 |
| abstract_inverted_index.state-of-the-art | 16 |
| abstract_inverted_index.optimization-based | 76 |
| abstract_inverted_index.augmentation-specific | 200 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 11 |
| citation_normalized_percentile |