The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers Article Swipe
James Prather
,
Brent N. Reeves
,
Juho Leinonen
,
Stephen MacNeil
,
Arisoa S. Randrianasolo
,
Brett A. Becker
,
Bailey Kimmel
,
Jared Wright
,
Ben Briggs
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2405.17739
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2405.17739
Novice programmers often struggle through programming problem solving due to a lack of metacognitive awareness and strategies. Previous research has shown that novices can encounter multiple metacognitive difficulties while programming. Novices are typically unaware of how these difficulties are hindering their progress. Meanwhile, many novices are now programming with generative AI (GenAI), which can provide complete solutions to most introductory programming problems, code suggestions, hints for next steps when stuck, and explain cryptic error messages. Its impact on novice metacognition has only started to be explored. Here we replicate a previous study that examined novice programming problem solving behavior and extend it by incorporating GenAI tools. Through 21 lab sessions consisting of participant observation, interview, and eye tracking, we explore how novices are coding with GenAI tools. Although 20 of 21 students completed the assigned programming problem, our findings show an unfortunate divide in the use of GenAI tools between students who accelerated and students who struggled. Students who accelerated were able to use GenAI to create code they already intended to make and were able to ignore unhelpful or incorrect inline code suggestions. But for students who struggled, our findings indicate that previously known metacognitive difficulties persist, and that GenAI unfortunately can compound them and even introduce new metacognitive difficulties. Furthermore, struggling students often expressed cognitive dissonance about their problem solving ability, thought they performed better than they did, and finished with an illusion of competence. Based on our observations from both groups, we propose ways to scaffold the novice GenAI experience and make suggestions for future work.
Related Topics
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2405.17739
- https://arxiv.org/pdf/2405.17739
- OA Status
- green
- Cited By
- 3
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4399151505
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4399151505Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2405.17739Digital Object Identifier
- Title
-
The Widening Gap: The Benefits and Harms of Generative AI for Novice ProgrammersWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-05-28Full publication date if available
- Authors
-
James Prather, Brent N. Reeves, Juho Leinonen, Stephen MacNeil, Arisoa S. Randrianasolo, Brett A. Becker, Bailey Kimmel, Jared Wright, Ben BriggsList of authors in order
- Landing page
-
https://arxiv.org/abs/2405.17739Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2405.17739Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2405.17739Direct OA link when available
- Concepts
-
Generative grammar, Computer science, Artificial intelligenceTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
3Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 3Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4399151505 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2405.17739 |
| ids.doi | https://doi.org/10.48550/arxiv.2405.17739 |
| ids.openalex | https://openalex.org/W4399151505 |
| fwci | |
| type | preprint |
| title | The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T11636 |
| topics[0].field.id | https://openalex.org/fields/27 |
| topics[0].field.display_name | Medicine |
| topics[0].score | 0.9679999947547913 |
| topics[0].domain.id | https://openalex.org/domains/4 |
| topics[0].domain.display_name | Health Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/2718 |
| topics[0].subfield.display_name | Health Informatics |
| topics[0].display_name | Artificial Intelligence in Healthcare and Education |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C39890363 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7670329809188843 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q36108 |
| concepts[0].display_name | Generative grammar |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.4899284243583679 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C154945302 |
| concepts[2].level | 1 |
| concepts[2].score | 0.2763060927391052 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[2].display_name | Artificial intelligence |
| keywords[0].id | https://openalex.org/keywords/generative-grammar |
| keywords[0].score | 0.7670329809188843 |
| keywords[0].display_name | Generative grammar |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.4899284243583679 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[2].score | 0.2763060927391052 |
| keywords[2].display_name | Artificial intelligence |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2405.17739 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2405.17739 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2405.17739 |
| locations[1].id | doi:10.48550/arxiv.2405.17739 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2405.17739 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5065053503 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-2807-6042 |
| authorships[0].author.display_name | James Prather |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Prather, James |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5071345488 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-5781-1136 |
| authorships[1].author.display_name | Brent N. Reeves |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Reeves, Brent |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5041367899 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-6829-9449 |
| authorships[2].author.display_name | Juho Leinonen |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Leinonen, Juho |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5042822346 |
| authorships[3].author.orcid | https://orcid.org/0000-0003-2781-6619 |
| authorships[3].author.display_name | Stephen MacNeil |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | MacNeil, Stephen |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5030533178 |
| authorships[4].author.orcid | https://orcid.org/0000-0003-2481-1820 |
| authorships[4].author.display_name | Arisoa S. Randrianasolo |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Randrianasolo, Arisoa S. |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5054692443 |
| authorships[5].author.orcid | https://orcid.org/0000-0003-1446-647X |
| authorships[5].author.display_name | Brett A. Becker |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Becker, Brett |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5032143740 |
| authorships[6].author.orcid | https://orcid.org/0009-0000-6655-0564 |
| authorships[6].author.display_name | Bailey Kimmel |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Kimmel, Bailey |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5098937121 |
| authorships[7].author.orcid | |
| authorships[7].author.display_name | Jared Wright |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Wright, Jared |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5098937122 |
| authorships[8].author.orcid | |
| authorships[8].author.display_name | Ben Briggs |
| authorships[8].author_position | last |
| authorships[8].raw_author_name | Briggs, Ben |
| authorships[8].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2405.17739 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T11636 |
| primary_topic.field.id | https://openalex.org/fields/27 |
| primary_topic.field.display_name | Medicine |
| primary_topic.score | 0.9679999947547913 |
| primary_topic.domain.id | https://openalex.org/domains/4 |
| primary_topic.domain.display_name | Health Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/2718 |
| primary_topic.subfield.display_name | Health Informatics |
| primary_topic.display_name | Artificial Intelligence in Healthcare and Education |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2748952813, https://openalex.org/W2380075625, https://openalex.org/W2390279801, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052, https://openalex.org/W2382290278 |
| cited_by_count | 3 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 3 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2405.17739 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2405.17739 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2405.17739 |
| primary_location.id | pmh:oai:arXiv.org:2405.17739 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2405.17739 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2405.17739 |
| publication_date | 2024-05-28 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 10, 89 |
| abstract_inverted_index.20 | 128 |
| abstract_inverted_index.21 | 107, 130 |
| abstract_inverted_index.AI | 50 |
| abstract_inverted_index.an | 140, 233 |
| abstract_inverted_index.be | 84 |
| abstract_inverted_index.by | 102 |
| abstract_inverted_index.in | 143 |
| abstract_inverted_index.it | 101 |
| abstract_inverted_index.of | 12, 34, 111, 129, 146, 235 |
| abstract_inverted_index.on | 77, 238 |
| abstract_inverted_index.or | 179 |
| abstract_inverted_index.to | 9, 57, 83, 162, 165, 171, 176, 247 |
| abstract_inverted_index.we | 87, 118, 244 |
| abstract_inverted_index.But | 184 |
| abstract_inverted_index.Its | 75 |
| abstract_inverted_index.and | 15, 70, 99, 115, 153, 173, 198, 205, 230, 253 |
| abstract_inverted_index.are | 31, 38, 45, 122 |
| abstract_inverted_index.can | 23, 53, 202 |
| abstract_inverted_index.due | 8 |
| abstract_inverted_index.eye | 116 |
| abstract_inverted_index.for | 65, 185, 256 |
| abstract_inverted_index.has | 19, 80 |
| abstract_inverted_index.how | 35, 120 |
| abstract_inverted_index.lab | 108 |
| abstract_inverted_index.new | 208 |
| abstract_inverted_index.now | 46 |
| abstract_inverted_index.our | 137, 189, 239 |
| abstract_inverted_index.the | 133, 144, 249 |
| abstract_inverted_index.use | 145, 163 |
| abstract_inverted_index.who | 151, 155, 158, 187 |
| abstract_inverted_index.Here | 86 |
| abstract_inverted_index.able | 161, 175 |
| abstract_inverted_index.both | 242 |
| abstract_inverted_index.code | 62, 167, 182 |
| abstract_inverted_index.did, | 229 |
| abstract_inverted_index.even | 206 |
| abstract_inverted_index.from | 241 |
| abstract_inverted_index.lack | 11 |
| abstract_inverted_index.make | 172, 254 |
| abstract_inverted_index.many | 43 |
| abstract_inverted_index.most | 58 |
| abstract_inverted_index.next | 66 |
| abstract_inverted_index.only | 81 |
| abstract_inverted_index.show | 139 |
| abstract_inverted_index.than | 227 |
| abstract_inverted_index.that | 21, 92, 192, 199 |
| abstract_inverted_index.them | 204 |
| abstract_inverted_index.they | 168, 224, 228 |
| abstract_inverted_index.ways | 246 |
| abstract_inverted_index.were | 160, 174 |
| abstract_inverted_index.when | 68 |
| abstract_inverted_index.with | 48, 124, 232 |
| abstract_inverted_index.Based | 237 |
| abstract_inverted_index.GenAI | 104, 125, 147, 164, 200, 251 |
| abstract_inverted_index.about | 218 |
| abstract_inverted_index.error | 73 |
| abstract_inverted_index.hints | 64 |
| abstract_inverted_index.known | 194 |
| abstract_inverted_index.often | 2, 214 |
| abstract_inverted_index.shown | 20 |
| abstract_inverted_index.steps | 67 |
| abstract_inverted_index.study | 91 |
| abstract_inverted_index.their | 40, 219 |
| abstract_inverted_index.these | 36 |
| abstract_inverted_index.tools | 148 |
| abstract_inverted_index.which | 52 |
| abstract_inverted_index.while | 28 |
| abstract_inverted_index.work. | 258 |
| abstract_inverted_index.Novice | 0 |
| abstract_inverted_index.better | 226 |
| abstract_inverted_index.coding | 123 |
| abstract_inverted_index.create | 166 |
| abstract_inverted_index.divide | 142 |
| abstract_inverted_index.extend | 100 |
| abstract_inverted_index.future | 257 |
| abstract_inverted_index.ignore | 177 |
| abstract_inverted_index.impact | 76 |
| abstract_inverted_index.inline | 181 |
| abstract_inverted_index.novice | 78, 94, 250 |
| abstract_inverted_index.stuck, | 69 |
| abstract_inverted_index.tools. | 105, 126 |
| abstract_inverted_index.Novices | 30 |
| abstract_inverted_index.Through | 106 |
| abstract_inverted_index.already | 169 |
| abstract_inverted_index.between | 149 |
| abstract_inverted_index.cryptic | 72 |
| abstract_inverted_index.explain | 71 |
| abstract_inverted_index.explore | 119 |
| abstract_inverted_index.groups, | 243 |
| abstract_inverted_index.novices | 22, 44, 121 |
| abstract_inverted_index.problem | 6, 96, 220 |
| abstract_inverted_index.propose | 245 |
| abstract_inverted_index.provide | 54 |
| abstract_inverted_index.solving | 7, 97, 221 |
| abstract_inverted_index.started | 82 |
| abstract_inverted_index.thought | 223 |
| abstract_inverted_index.through | 4 |
| abstract_inverted_index.unaware | 33 |
| abstract_inverted_index.(GenAI), | 51 |
| abstract_inverted_index.Although | 127 |
| abstract_inverted_index.Previous | 17 |
| abstract_inverted_index.Students | 157 |
| abstract_inverted_index.ability, | 222 |
| abstract_inverted_index.assigned | 134 |
| abstract_inverted_index.behavior | 98 |
| abstract_inverted_index.complete | 55 |
| abstract_inverted_index.compound | 203 |
| abstract_inverted_index.examined | 93 |
| abstract_inverted_index.findings | 138, 190 |
| abstract_inverted_index.finished | 231 |
| abstract_inverted_index.illusion | 234 |
| abstract_inverted_index.indicate | 191 |
| abstract_inverted_index.intended | 170 |
| abstract_inverted_index.multiple | 25 |
| abstract_inverted_index.persist, | 197 |
| abstract_inverted_index.previous | 90 |
| abstract_inverted_index.problem, | 136 |
| abstract_inverted_index.research | 18 |
| abstract_inverted_index.scaffold | 248 |
| abstract_inverted_index.sessions | 109 |
| abstract_inverted_index.struggle | 3 |
| abstract_inverted_index.students | 131, 150, 154, 186, 213 |
| abstract_inverted_index.awareness | 14 |
| abstract_inverted_index.cognitive | 216 |
| abstract_inverted_index.completed | 132 |
| abstract_inverted_index.encounter | 24 |
| abstract_inverted_index.explored. | 85 |
| abstract_inverted_index.expressed | 215 |
| abstract_inverted_index.hindering | 39 |
| abstract_inverted_index.incorrect | 180 |
| abstract_inverted_index.introduce | 207 |
| abstract_inverted_index.messages. | 74 |
| abstract_inverted_index.performed | 225 |
| abstract_inverted_index.problems, | 61 |
| abstract_inverted_index.progress. | 41 |
| abstract_inverted_index.replicate | 88 |
| abstract_inverted_index.solutions | 56 |
| abstract_inverted_index.tracking, | 117 |
| abstract_inverted_index.typically | 32 |
| abstract_inverted_index.unhelpful | 178 |
| abstract_inverted_index.Meanwhile, | 42 |
| abstract_inverted_index.consisting | 110 |
| abstract_inverted_index.dissonance | 217 |
| abstract_inverted_index.experience | 252 |
| abstract_inverted_index.generative | 49 |
| abstract_inverted_index.interview, | 114 |
| abstract_inverted_index.previously | 193 |
| abstract_inverted_index.struggled, | 188 |
| abstract_inverted_index.struggled. | 156 |
| abstract_inverted_index.struggling | 212 |
| abstract_inverted_index.accelerated | 152, 159 |
| abstract_inverted_index.competence. | 236 |
| abstract_inverted_index.participant | 112 |
| abstract_inverted_index.programmers | 1 |
| abstract_inverted_index.programming | 5, 47, 60, 95, 135 |
| abstract_inverted_index.strategies. | 16 |
| abstract_inverted_index.suggestions | 255 |
| abstract_inverted_index.unfortunate | 141 |
| abstract_inverted_index.Furthermore, | 211 |
| abstract_inverted_index.difficulties | 27, 37, 196 |
| abstract_inverted_index.introductory | 59 |
| abstract_inverted_index.observation, | 113 |
| abstract_inverted_index.observations | 240 |
| abstract_inverted_index.programming. | 29 |
| abstract_inverted_index.suggestions, | 63 |
| abstract_inverted_index.suggestions. | 183 |
| abstract_inverted_index.difficulties. | 210 |
| abstract_inverted_index.incorporating | 103 |
| abstract_inverted_index.metacognition | 79 |
| abstract_inverted_index.metacognitive | 13, 26, 195, 209 |
| abstract_inverted_index.unfortunately | 201 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 9 |
| citation_normalized_percentile |