Web Archives Metadata Generation with GPT-4o: Challenges and Insights Article Swipe
A Huang
,
Abhijit Nair
,
Zhen Rong Goh
,
Tianrui Liu
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2411.05409
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2411.05409
Current metadata creation for web archives is time consuming and costly due to reliance on human effort. This paper explores the use of gpt-4o for metadata generation within the Web Archive Singapore, focusing on scalability, efficiency, and cost effectiveness. We processed 112 Web ARChive (WARC) files using data reduction techniques, achieving a notable 99.9% reduction in metadata generation costs. By prompt engineering, we generated titles and abstracts, which were evaluated both intrinsically using Levenshtein Distance and BERTScore, and extrinsically with human cataloguers using McNemar's test. Results indicate that while our method offers significant cost savings and efficiency gains, human curated metadata maintains an edge in quality. The study identifies key challenges including content inaccuracies, hallucinations, and translation issues, suggesting that Large Language Models (LLMs) should serve as complements rather than replacements for human cataloguers. Future work will focus on refining prompts, improving content filtering, and addressing privacy concerns through experimentation with smaller models. This research advances the integration of LLMs in web archiving, offering valuable insights into their current capabilities and outlining directions for future enhancements. The code is available at https://github.com/masamune-prog/warc2summary for further development and use by institutions facing similar challenges.
Related Topics
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2411.05409
- https://arxiv.org/pdf/2411.05409
- OA Status
- green
- Cited By
- 1
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4404389558
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4404389558Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2411.05409Digital Object Identifier
- Title
-
Web Archives Metadata Generation with GPT-4o: Challenges and InsightsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-11-08Full publication date if available
- Authors
-
A Huang, Abhijit Nair, Zhen Rong Goh, Tianrui LiuList of authors in order
- Landing page
-
https://arxiv.org/abs/2411.05409Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2411.05409Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2411.05409Direct OA link when available
- Concepts
-
Metadata, World Wide Web, Web application, Computer scienceTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
1Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4404389558 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2411.05409 |
| ids.doi | https://doi.org/10.48550/arxiv.2411.05409 |
| ids.openalex | https://openalex.org/W4404389558 |
| fwci | |
| type | preprint |
| title | Web Archives Metadata Generation with GPT-4o: Challenges and Insights |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T13523 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9570000171661377 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1703 |
| topics[0].subfield.display_name | Computational Theory and Mathematics |
| topics[0].display_name | Mathematics, Computing, and Information Processing |
| topics[1].id | https://openalex.org/T11986 |
| topics[1].field.id | https://openalex.org/fields/18 |
| topics[1].field.display_name | Decision Sciences |
| topics[1].score | 0.9197999835014343 |
| topics[1].domain.id | https://openalex.org/domains/2 |
| topics[1].domain.display_name | Social Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1802 |
| topics[1].subfield.display_name | Information Systems and Management |
| topics[1].display_name | Scientific Computing and Data Management |
| topics[2].id | https://openalex.org/T11937 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9161999821662903 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1710 |
| topics[2].subfield.display_name | Information Systems |
| topics[2].display_name | Research Data Management Practices |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C93518851 |
| concepts[0].level | 2 |
| concepts[0].score | 0.8661413788795471 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q180160 |
| concepts[0].display_name | Metadata |
| concepts[1].id | https://openalex.org/C136764020 |
| concepts[1].level | 1 |
| concepts[1].score | 0.5346924662590027 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q466 |
| concepts[1].display_name | World Wide Web |
| concepts[2].id | https://openalex.org/C118643609 |
| concepts[2].level | 2 |
| concepts[2].score | 0.52788245677948 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q189210 |
| concepts[2].display_name | Web application |
| concepts[3].id | https://openalex.org/C41008148 |
| concepts[3].level | 0 |
| concepts[3].score | 0.3928043842315674 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[3].display_name | Computer science |
| keywords[0].id | https://openalex.org/keywords/metadata |
| keywords[0].score | 0.8661413788795471 |
| keywords[0].display_name | Metadata |
| keywords[1].id | https://openalex.org/keywords/world-wide-web |
| keywords[1].score | 0.5346924662590027 |
| keywords[1].display_name | World Wide Web |
| keywords[2].id | https://openalex.org/keywords/web-application |
| keywords[2].score | 0.52788245677948 |
| keywords[2].display_name | Web application |
| keywords[3].id | https://openalex.org/keywords/computer-science |
| keywords[3].score | 0.3928043842315674 |
| keywords[3].display_name | Computer science |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2411.05409 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2411.05409 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2411.05409 |
| locations[1].id | doi:10.48550/arxiv.2411.05409 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2411.05409 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5071724766 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | A Huang |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Huang, Abigail Yongping |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5042109787 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-2506-0301 |
| authorships[1].author.display_name | Abhijit Nair |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Nair, Ashwin |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5114653520 |
| authorships[2].author.orcid | |
| authorships[2].author.display_name | Zhen Rong Goh |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Goh, Zhen Rong |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5102949546 |
| authorships[3].author.orcid | https://orcid.org/0000-0001-7926-3310 |
| authorships[3].author.display_name | Tianrui Liu |
| authorships[3].author_position | last |
| authorships[3].raw_author_name | Liu, Tianrui |
| authorships[3].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2411.05409 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-11-15T00:00:00 |
| display_name | Web Archives Metadata Generation with GPT-4o: Challenges and Insights |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T13523 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9570000171661377 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1703 |
| primary_topic.subfield.display_name | Computational Theory and Mathematics |
| primary_topic.display_name | Mathematics, Computing, and Information Processing |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2058118494, https://openalex.org/W2392768766, https://openalex.org/W2382021449, https://openalex.org/W2095118173, https://openalex.org/W2106424170, https://openalex.org/W1985426483, https://openalex.org/W2501188010 |
| cited_by_count | 1 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2411.05409 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2411.05409 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2411.05409 |
| primary_location.id | pmh:oai:arXiv.org:2411.05409 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2411.05409 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2411.05409 |
| publication_date | 2024-11-08 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 51 |
| abstract_inverted_index.By | 59 |
| abstract_inverted_index.We | 39 |
| abstract_inverted_index.an | 102 |
| abstract_inverted_index.as | 126 |
| abstract_inverted_index.at | 180 |
| abstract_inverted_index.by | 187 |
| abstract_inverted_index.in | 55, 104, 160 |
| abstract_inverted_index.is | 6, 178 |
| abstract_inverted_index.of | 22, 158 |
| abstract_inverted_index.on | 14, 33, 138 |
| abstract_inverted_index.to | 12 |
| abstract_inverted_index.we | 62 |
| abstract_inverted_index.112 | 41 |
| abstract_inverted_index.The | 106, 176 |
| abstract_inverted_index.Web | 29, 42 |
| abstract_inverted_index.and | 9, 36, 65, 75, 77, 95, 115, 144, 170, 185 |
| abstract_inverted_index.due | 11 |
| abstract_inverted_index.for | 3, 24, 131, 173, 182 |
| abstract_inverted_index.key | 109 |
| abstract_inverted_index.our | 89 |
| abstract_inverted_index.the | 20, 28, 156 |
| abstract_inverted_index.use | 21, 186 |
| abstract_inverted_index.web | 4, 161 |
| abstract_inverted_index.LLMs | 159 |
| abstract_inverted_index.This | 17, 153 |
| abstract_inverted_index.both | 70 |
| abstract_inverted_index.code | 177 |
| abstract_inverted_index.cost | 37, 93 |
| abstract_inverted_index.data | 47 |
| abstract_inverted_index.edge | 103 |
| abstract_inverted_index.into | 166 |
| abstract_inverted_index.than | 129 |
| abstract_inverted_index.that | 87, 119 |
| abstract_inverted_index.time | 7 |
| abstract_inverted_index.were | 68 |
| abstract_inverted_index.will | 136 |
| abstract_inverted_index.with | 79, 150 |
| abstract_inverted_index.work | 135 |
| abstract_inverted_index.99.9% | 53 |
| abstract_inverted_index.Large | 120 |
| abstract_inverted_index.files | 45 |
| abstract_inverted_index.focus | 137 |
| abstract_inverted_index.human | 15, 80, 98, 132 |
| abstract_inverted_index.paper | 18 |
| abstract_inverted_index.serve | 125 |
| abstract_inverted_index.study | 107 |
| abstract_inverted_index.test. | 84 |
| abstract_inverted_index.their | 167 |
| abstract_inverted_index.using | 46, 72, 82 |
| abstract_inverted_index.which | 67 |
| abstract_inverted_index.while | 88 |
| abstract_inverted_index.(LLMs) | 123 |
| abstract_inverted_index.(WARC) | 44 |
| abstract_inverted_index.Future | 134 |
| abstract_inverted_index.Models | 122 |
| abstract_inverted_index.costly | 10 |
| abstract_inverted_index.costs. | 58 |
| abstract_inverted_index.facing | 189 |
| abstract_inverted_index.future | 174 |
| abstract_inverted_index.gains, | 97 |
| abstract_inverted_index.gpt-4o | 23 |
| abstract_inverted_index.method | 90 |
| abstract_inverted_index.offers | 91 |
| abstract_inverted_index.prompt | 60 |
| abstract_inverted_index.rather | 128 |
| abstract_inverted_index.should | 124 |
| abstract_inverted_index.titles | 64 |
| abstract_inverted_index.within | 27 |
| abstract_inverted_index.ARChive | 43 |
| abstract_inverted_index.Archive | 30 |
| abstract_inverted_index.Current | 0 |
| abstract_inverted_index.Results | 85 |
| abstract_inverted_index.content | 112, 142 |
| abstract_inverted_index.curated | 99 |
| abstract_inverted_index.current | 168 |
| abstract_inverted_index.effort. | 16 |
| abstract_inverted_index.further | 183 |
| abstract_inverted_index.issues, | 117 |
| abstract_inverted_index.models. | 152 |
| abstract_inverted_index.notable | 52 |
| abstract_inverted_index.privacy | 146 |
| abstract_inverted_index.savings | 94 |
| abstract_inverted_index.similar | 190 |
| abstract_inverted_index.smaller | 151 |
| abstract_inverted_index.through | 148 |
| abstract_inverted_index.Distance | 74 |
| abstract_inverted_index.Language | 121 |
| abstract_inverted_index.advances | 155 |
| abstract_inverted_index.archives | 5 |
| abstract_inverted_index.concerns | 147 |
| abstract_inverted_index.creation | 2 |
| abstract_inverted_index.explores | 19 |
| abstract_inverted_index.focusing | 32 |
| abstract_inverted_index.indicate | 86 |
| abstract_inverted_index.insights | 165 |
| abstract_inverted_index.metadata | 1, 25, 56, 100 |
| abstract_inverted_index.offering | 163 |
| abstract_inverted_index.prompts, | 140 |
| abstract_inverted_index.quality. | 105 |
| abstract_inverted_index.refining | 139 |
| abstract_inverted_index.reliance | 13 |
| abstract_inverted_index.research | 154 |
| abstract_inverted_index.valuable | 164 |
| abstract_inverted_index.McNemar's | 83 |
| abstract_inverted_index.achieving | 50 |
| abstract_inverted_index.available | 179 |
| abstract_inverted_index.consuming | 8 |
| abstract_inverted_index.evaluated | 69 |
| abstract_inverted_index.generated | 63 |
| abstract_inverted_index.improving | 141 |
| abstract_inverted_index.including | 111 |
| abstract_inverted_index.maintains | 101 |
| abstract_inverted_index.outlining | 171 |
| abstract_inverted_index.processed | 40 |
| abstract_inverted_index.reduction | 48, 54 |
| abstract_inverted_index.BERTScore, | 76 |
| abstract_inverted_index.Singapore, | 31 |
| abstract_inverted_index.abstracts, | 66 |
| abstract_inverted_index.addressing | 145 |
| abstract_inverted_index.archiving, | 162 |
| abstract_inverted_index.challenges | 110 |
| abstract_inverted_index.directions | 172 |
| abstract_inverted_index.efficiency | 96 |
| abstract_inverted_index.filtering, | 143 |
| abstract_inverted_index.generation | 26, 57 |
| abstract_inverted_index.identifies | 108 |
| abstract_inverted_index.suggesting | 118 |
| abstract_inverted_index.Levenshtein | 73 |
| abstract_inverted_index.cataloguers | 81 |
| abstract_inverted_index.challenges. | 191 |
| abstract_inverted_index.complements | 127 |
| abstract_inverted_index.development | 184 |
| abstract_inverted_index.efficiency, | 35 |
| abstract_inverted_index.integration | 157 |
| abstract_inverted_index.significant | 92 |
| abstract_inverted_index.techniques, | 49 |
| abstract_inverted_index.translation | 116 |
| abstract_inverted_index.capabilities | 169 |
| abstract_inverted_index.cataloguers. | 133 |
| abstract_inverted_index.engineering, | 61 |
| abstract_inverted_index.institutions | 188 |
| abstract_inverted_index.replacements | 130 |
| abstract_inverted_index.scalability, | 34 |
| abstract_inverted_index.enhancements. | 175 |
| abstract_inverted_index.extrinsically | 78 |
| abstract_inverted_index.inaccuracies, | 113 |
| abstract_inverted_index.intrinsically | 71 |
| abstract_inverted_index.effectiveness. | 38 |
| abstract_inverted_index.experimentation | 149 |
| abstract_inverted_index.hallucinations, | 114 |
| abstract_inverted_index.https://github.com/masamune-prog/warc2summary | 181 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 4 |
| citation_normalized_percentile |