Rome was Not Built in a Single Step: Hierarchical Prompting for LLM-based Chip Design Article Swipe
Andre Nakkab
,
Sai Qian Zhang
,
Ramesh Karri
,
Siddharth Garg
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2407.18276
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2407.18276
Large Language Models (LLMs) are effective in computer hardware synthesis via hardware description language (HDL) generation. However, LLM-assisted approaches for HDL generation struggle when handling complex tasks. We introduce a suite of hierarchical prompting techniques which facilitate efficient stepwise design methods, and develop a generalizable automation pipeline for the process. To evaluate these techniques, we present a benchmark set of hardware designs which have solutions with or without architectural hierarchy. Using these benchmarks, we compare various open-source and proprietary LLMs, including our own fine-tuned Code Llama-Verilog model. Our hierarchical methods automatically produce successful designs for complex hardware modules that standard flat prompting methods cannot achieve, allowing smaller open-source LLMs to compete with large proprietary models. Hierarchical prompting reduces HDL generation time and yields savings on LLM costs. Our experiments detail which LLMs are capable of which applications, and how to apply hierarchical methods in various modes. We explore case studies of generating complex cores using automatic scripted hierarchical prompts, including the first-ever LLM-designed processor with no human feedback. Tools for the Recurrent Optimization via Machine Editing (ROME) method can be found at https://github.com/ajn313/ROME-LLM
Related Topics
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2407.18276
- https://arxiv.org/pdf/2407.18276
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4402466769
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4402466769Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2407.18276Digital Object Identifier
- Title
-
Rome was Not Built in a Single Step: Hierarchical Prompting for LLM-based Chip DesignWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-07-23Full publication date if available
- Authors
-
Andre Nakkab, Sai Qian Zhang, Ramesh Karri, Siddharth GargList of authors in order
- Landing page
-
https://arxiv.org/abs/2407.18276Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2407.18276Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2407.18276Direct OA link when available
- Concepts
-
Chip, Computer science, Computer architecture, TelecommunicationsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4402466769 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2407.18276 |
| ids.doi | https://doi.org/10.48550/arxiv.2407.18276 |
| ids.openalex | https://openalex.org/W4402466769 |
| fwci | |
| type | preprint |
| title | Rome was Not Built in a Single Step: Hierarchical Prompting for LLM-based Chip Design |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T11522 |
| topics[0].field.id | https://openalex.org/fields/22 |
| topics[0].field.display_name | Engineering |
| topics[0].score | 0.9854000210762024 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/2208 |
| topics[0].subfield.display_name | Electrical and Electronic Engineering |
| topics[0].display_name | VLSI and FPGA Design Techniques |
| topics[1].id | https://openalex.org/T11032 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.980400025844574 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1708 |
| topics[1].subfield.display_name | Hardware and Architecture |
| topics[1].display_name | VLSI and Analog Circuit Testing |
| topics[2].id | https://openalex.org/T10363 |
| topics[2].field.id | https://openalex.org/fields/22 |
| topics[2].field.display_name | Engineering |
| topics[2].score | 0.9700000286102295 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/2208 |
| topics[2].subfield.display_name | Electrical and Electronic Engineering |
| topics[2].display_name | Low-power high-performance VLSI design |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C165005293 |
| concepts[0].level | 2 |
| concepts[0].score | 0.6424900889396667 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q1074500 |
| concepts[0].display_name | Chip |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.53052818775177 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C118524514 |
| concepts[2].level | 1 |
| concepts[2].score | 0.45989754796028137 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q173212 |
| concepts[2].display_name | Computer architecture |
| concepts[3].id | https://openalex.org/C76155785 |
| concepts[3].level | 1 |
| concepts[3].score | 0.10511794686317444 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q418 |
| concepts[3].display_name | Telecommunications |
| keywords[0].id | https://openalex.org/keywords/chip |
| keywords[0].score | 0.6424900889396667 |
| keywords[0].display_name | Chip |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.53052818775177 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/computer-architecture |
| keywords[2].score | 0.45989754796028137 |
| keywords[2].display_name | Computer architecture |
| keywords[3].id | https://openalex.org/keywords/telecommunications |
| keywords[3].score | 0.10511794686317444 |
| keywords[3].display_name | Telecommunications |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2407.18276 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | cc-by-nc-sa |
| locations[0].pdf_url | https://arxiv.org/pdf/2407.18276 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | https://openalex.org/licenses/cc-by-nc-sa |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2407.18276 |
| locations[1].id | doi:10.48550/arxiv.2407.18276 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2407.18276 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5106967177 |
| authorships[0].author.orcid | https://orcid.org/0009-0006-1345-5444 |
| authorships[0].author.display_name | Andre Nakkab |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Nakkab, Andre |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5013517247 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-4815-9235 |
| authorships[1].author.display_name | Sai Qian Zhang |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Zhang, Sai Qian |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5059648257 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-7989-5617 |
| authorships[2].author.display_name | Ramesh Karri |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Karri, Ramesh |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5010950688 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-6158-9512 |
| authorships[3].author.display_name | Siddharth Garg |
| authorships[3].author_position | last |
| authorships[3].raw_author_name | Garg, Siddharth |
| authorships[3].is_corresponding | False |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2407.18276 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-09-12T00:00:00 |
| display_name | Rome was Not Built in a Single Step: Hierarchical Prompting for LLM-based Chip Design |
| has_fulltext | True |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T11522 |
| primary_topic.field.id | https://openalex.org/fields/22 |
| primary_topic.field.display_name | Engineering |
| primary_topic.score | 0.9854000210762024 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/2208 |
| primary_topic.subfield.display_name | Electrical and Electronic Engineering |
| primary_topic.display_name | VLSI and FPGA Design Techniques |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052, https://openalex.org/W2382290278, https://openalex.org/W4395014643 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2407.18276 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | cc-by-nc-sa |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2407.18276 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | https://openalex.org/licenses/cc-by-nc-sa |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2407.18276 |
| primary_location.id | pmh:oai:arXiv.org:2407.18276 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | cc-by-nc-sa |
| primary_location.pdf_url | https://arxiv.org/pdf/2407.18276 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | https://openalex.org/licenses/cc-by-nc-sa |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2407.18276 |
| publication_date | 2024-07-23 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 29, 43, 56 |
| abstract_inverted_index.To | 50 |
| abstract_inverted_index.We | 27, 146 |
| abstract_inverted_index.at | 181 |
| abstract_inverted_index.be | 179 |
| abstract_inverted_index.in | 6, 143 |
| abstract_inverted_index.no | 165 |
| abstract_inverted_index.of | 31, 59, 134, 150 |
| abstract_inverted_index.on | 124 |
| abstract_inverted_index.or | 66 |
| abstract_inverted_index.to | 109, 139 |
| abstract_inverted_index.we | 54, 73 |
| abstract_inverted_index.HDL | 20, 118 |
| abstract_inverted_index.LLM | 125 |
| abstract_inverted_index.Our | 87, 127 |
| abstract_inverted_index.and | 41, 77, 121, 137 |
| abstract_inverted_index.are | 4, 132 |
| abstract_inverted_index.can | 178 |
| abstract_inverted_index.for | 19, 47, 94, 169 |
| abstract_inverted_index.how | 138 |
| abstract_inverted_index.our | 81 |
| abstract_inverted_index.own | 82 |
| abstract_inverted_index.set | 58 |
| abstract_inverted_index.the | 48, 160, 170 |
| abstract_inverted_index.via | 10, 173 |
| abstract_inverted_index.Code | 84 |
| abstract_inverted_index.LLMs | 108, 131 |
| abstract_inverted_index.case | 148 |
| abstract_inverted_index.flat | 100 |
| abstract_inverted_index.have | 63 |
| abstract_inverted_index.that | 98 |
| abstract_inverted_index.time | 120 |
| abstract_inverted_index.when | 23 |
| abstract_inverted_index.with | 65, 111, 164 |
| abstract_inverted_index.(HDL) | 14 |
| abstract_inverted_index.LLMs, | 79 |
| abstract_inverted_index.Large | 0 |
| abstract_inverted_index.Tools | 168 |
| abstract_inverted_index.Using | 70 |
| abstract_inverted_index.apply | 140 |
| abstract_inverted_index.cores | 153 |
| abstract_inverted_index.found | 180 |
| abstract_inverted_index.human | 166 |
| abstract_inverted_index.large | 112 |
| abstract_inverted_index.suite | 30 |
| abstract_inverted_index.these | 52, 71 |
| abstract_inverted_index.using | 154 |
| abstract_inverted_index.which | 35, 62, 130, 135 |
| abstract_inverted_index.(LLMs) | 3 |
| abstract_inverted_index.(ROME) | 176 |
| abstract_inverted_index.Models | 2 |
| abstract_inverted_index.cannot | 103 |
| abstract_inverted_index.costs. | 126 |
| abstract_inverted_index.design | 39 |
| abstract_inverted_index.detail | 129 |
| abstract_inverted_index.method | 177 |
| abstract_inverted_index.model. | 86 |
| abstract_inverted_index.modes. | 145 |
| abstract_inverted_index.tasks. | 26 |
| abstract_inverted_index.yields | 122 |
| abstract_inverted_index.Editing | 175 |
| abstract_inverted_index.Machine | 174 |
| abstract_inverted_index.capable | 133 |
| abstract_inverted_index.compare | 74 |
| abstract_inverted_index.compete | 110 |
| abstract_inverted_index.complex | 25, 95, 152 |
| abstract_inverted_index.designs | 61, 93 |
| abstract_inverted_index.develop | 42 |
| abstract_inverted_index.explore | 147 |
| abstract_inverted_index.methods | 89, 102, 142 |
| abstract_inverted_index.models. | 114 |
| abstract_inverted_index.modules | 97 |
| abstract_inverted_index.present | 55 |
| abstract_inverted_index.produce | 91 |
| abstract_inverted_index.reduces | 117 |
| abstract_inverted_index.savings | 123 |
| abstract_inverted_index.smaller | 106 |
| abstract_inverted_index.studies | 149 |
| abstract_inverted_index.various | 75, 144 |
| abstract_inverted_index.without | 67 |
| abstract_inverted_index.However, | 16 |
| abstract_inverted_index.Language | 1 |
| abstract_inverted_index.achieve, | 104 |
| abstract_inverted_index.allowing | 105 |
| abstract_inverted_index.computer | 7 |
| abstract_inverted_index.evaluate | 51 |
| abstract_inverted_index.handling | 24 |
| abstract_inverted_index.hardware | 8, 11, 60, 96 |
| abstract_inverted_index.language | 13 |
| abstract_inverted_index.methods, | 40 |
| abstract_inverted_index.pipeline | 46 |
| abstract_inverted_index.process. | 49 |
| abstract_inverted_index.prompts, | 158 |
| abstract_inverted_index.scripted | 156 |
| abstract_inverted_index.standard | 99 |
| abstract_inverted_index.stepwise | 38 |
| abstract_inverted_index.struggle | 22 |
| abstract_inverted_index.Recurrent | 171 |
| abstract_inverted_index.automatic | 155 |
| abstract_inverted_index.benchmark | 57 |
| abstract_inverted_index.effective | 5 |
| abstract_inverted_index.efficient | 37 |
| abstract_inverted_index.feedback. | 167 |
| abstract_inverted_index.including | 80, 159 |
| abstract_inverted_index.introduce | 28 |
| abstract_inverted_index.processor | 163 |
| abstract_inverted_index.prompting | 33, 101, 116 |
| abstract_inverted_index.solutions | 64 |
| abstract_inverted_index.synthesis | 9 |
| abstract_inverted_index.approaches | 18 |
| abstract_inverted_index.automation | 45 |
| abstract_inverted_index.facilitate | 36 |
| abstract_inverted_index.fine-tuned | 83 |
| abstract_inverted_index.first-ever | 161 |
| abstract_inverted_index.generating | 151 |
| abstract_inverted_index.generation | 21, 119 |
| abstract_inverted_index.hierarchy. | 69 |
| abstract_inverted_index.successful | 92 |
| abstract_inverted_index.techniques | 34 |
| abstract_inverted_index.benchmarks, | 72 |
| abstract_inverted_index.description | 12 |
| abstract_inverted_index.experiments | 128 |
| abstract_inverted_index.generation. | 15 |
| abstract_inverted_index.open-source | 76, 107 |
| abstract_inverted_index.proprietary | 78, 113 |
| abstract_inverted_index.techniques, | 53 |
| abstract_inverted_index.Hierarchical | 115 |
| abstract_inverted_index.LLM-assisted | 17 |
| abstract_inverted_index.LLM-designed | 162 |
| abstract_inverted_index.Optimization | 172 |
| abstract_inverted_index.hierarchical | 32, 88, 141, 157 |
| abstract_inverted_index.Llama-Verilog | 85 |
| abstract_inverted_index.applications, | 136 |
| abstract_inverted_index.architectural | 68 |
| abstract_inverted_index.automatically | 90 |
| abstract_inverted_index.generalizable | 44 |
| abstract_inverted_index.https://github.com/ajn313/ROME-LLM | 182 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 4 |
| citation_normalized_percentile |