Analyzing LLM Usage in an Advanced Computing Class in India Article Swipe
Chaitanya Arora
,
Utkarsh Venaik
,
Pavit Singh
,
Sahil Goyal
,
Jatin Tyagi
,
Shyama Goel
,
Ujjwal Singhal
,
Dhruv Kumar
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2404.04603
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2404.04603
This study examines the use of large language models (LLMs) by undergraduate and graduate students for programming assignments in advanced computing classes. Unlike existing research, which primarily focuses on introductory classes and lacks in-depth analysis of actual student-LLM interactions, our work fills this gap. We conducted a comprehensive analysis involving 411 students from a Distributed Systems class at an Indian university, where they completed three programming assignments and shared their experiences through Google Form surveys. Our findings reveal that students leveraged LLMs for a variety of tasks, including code generation, debugging, conceptual inquiries, and test case creation. They employed a spectrum of prompting strategies, ranging from basic contextual prompts to advanced techniques like chain-of-thought prompting and iterative refinement. While students generally viewed LLMs as beneficial for enhancing productivity and learning, we noted a concerning trend of over-reliance, with many students submitting entire assignment descriptions to obtain complete solutions. Given the increasing use of LLMs in the software industry, our study highlights the need to update undergraduate curricula to include training on effective prompting strategies and to raise awareness about the benefits and potential drawbacks of LLM usage in academic settings.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2404.04603
- https://arxiv.org/pdf/2404.04603
- OA Status
- green
- Cited By
- 3
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4394654243
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4394654243Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2404.04603Digital Object Identifier
- Title
-
Analyzing LLM Usage in an Advanced Computing Class in IndiaWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-04-06Full publication date if available
- Authors
-
Chaitanya Arora, Utkarsh Venaik, Pavit Singh, Sahil Goyal, Jatin Tyagi, Shyama Goel, Ujjwal Singhal, Dhruv KumarList of authors in order
- Landing page
-
https://arxiv.org/abs/2404.04603Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2404.04603Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2404.04603Direct OA link when available
- Concepts
-
Class (philosophy), Computer science, Data science, Mathematics education, Mathematics, Artificial intelligenceTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
3Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 2, 2024: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4394654243 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2404.04603 |
| ids.doi | https://doi.org/10.48550/arxiv.2404.04603 |
| ids.openalex | https://openalex.org/W4394654243 |
| fwci | |
| type | preprint |
| title | Analyzing LLM Usage in an Advanced Computing Class in India |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T13999 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.5985999703407288 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1710 |
| topics[0].subfield.display_name | Information Systems |
| topics[0].display_name | Digital Rights Management and Security |
| topics[1].id | https://openalex.org/T13844 |
| topics[1].field.id | https://openalex.org/fields/33 |
| topics[1].field.display_name | Social Sciences |
| topics[1].score | 0.5496000051498413 |
| topics[1].domain.id | https://openalex.org/domains/2 |
| topics[1].domain.display_name | Social Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/3304 |
| topics[1].subfield.display_name | Education |
| topics[1].display_name | Higher Education Learning Practices |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C2777212361 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7290351390838623 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q5127848 |
| concepts[0].display_name | Class (philosophy) |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.5194513201713562 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C2522767166 |
| concepts[2].level | 1 |
| concepts[2].score | 0.3701947331428528 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q2374463 |
| concepts[2].display_name | Data science |
| concepts[3].id | https://openalex.org/C145420912 |
| concepts[3].level | 1 |
| concepts[3].score | 0.33037880063056946 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q853077 |
| concepts[3].display_name | Mathematics education |
| concepts[4].id | https://openalex.org/C33923547 |
| concepts[4].level | 0 |
| concepts[4].score | 0.20776531100273132 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[4].display_name | Mathematics |
| concepts[5].id | https://openalex.org/C154945302 |
| concepts[5].level | 1 |
| concepts[5].score | 0.20291215181350708 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[5].display_name | Artificial intelligence |
| keywords[0].id | https://openalex.org/keywords/class |
| keywords[0].score | 0.7290351390838623 |
| keywords[0].display_name | Class (philosophy) |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.5194513201713562 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/data-science |
| keywords[2].score | 0.3701947331428528 |
| keywords[2].display_name | Data science |
| keywords[3].id | https://openalex.org/keywords/mathematics-education |
| keywords[3].score | 0.33037880063056946 |
| keywords[3].display_name | Mathematics education |
| keywords[4].id | https://openalex.org/keywords/mathematics |
| keywords[4].score | 0.20776531100273132 |
| keywords[4].display_name | Mathematics |
| keywords[5].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[5].score | 0.20291215181350708 |
| keywords[5].display_name | Artificial intelligence |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2404.04603 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2404.04603 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2404.04603 |
| locations[1].id | doi:10.48550/arxiv.2404.04603 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2404.04603 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5022872824 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Chaitanya Arora |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Arora, Chaitanya |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5035471805 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-2561-1353 |
| authorships[1].author.display_name | Utkarsh Venaik |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Venaik, Utkarsh |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5109760949 |
| authorships[2].author.orcid | |
| authorships[2].author.display_name | Pavit Singh |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Singh, Pavit |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5091199866 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-8209-5167 |
| authorships[3].author.display_name | Sahil Goyal |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Goyal, Sahil |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5113173811 |
| authorships[4].author.orcid | |
| authorships[4].author.display_name | Jatin Tyagi |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Tyagi, Jatin |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5113173812 |
| authorships[5].author.orcid | |
| authorships[5].author.display_name | Shyama Goel |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Goel, Shyama |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5092958359 |
| authorships[6].author.orcid | https://orcid.org/0009-0003-6389-1170 |
| authorships[6].author.display_name | Ujjwal Singhal |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Singhal, Ujjwal |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5027859418 |
| authorships[7].author.orcid | https://orcid.org/0000-0003-2586-9397 |
| authorships[7].author.display_name | Dhruv Kumar |
| authorships[7].author_position | last |
| authorships[7].raw_author_name | Kumar, Dhruv |
| authorships[7].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2404.04603 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-04-11T00:00:00 |
| display_name | Analyzing LLM Usage in an Advanced Computing Class in India |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T13999 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.5985999703407288 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1710 |
| primary_topic.subfield.display_name | Information Systems |
| primary_topic.display_name | Digital Rights Management and Security |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W2358668433, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W2382290278, https://openalex.org/W4391913857, https://openalex.org/W2350741829, https://openalex.org/W2901074640 |
| cited_by_count | 3 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 2 |
| counts_by_year[1].year | 2024 |
| counts_by_year[1].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2404.04603 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2404.04603 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2404.04603 |
| primary_location.id | pmh:oai:arXiv.org:2404.04603 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2404.04603 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2404.04603 |
| publication_date | 2024-04-06 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 46, 53, 83, 99, 132 |
| abstract_inverted_index.We | 44 |
| abstract_inverted_index.an | 58 |
| abstract_inverted_index.as | 123 |
| abstract_inverted_index.at | 57 |
| abstract_inverted_index.by | 10 |
| abstract_inverted_index.in | 18, 154, 187 |
| abstract_inverted_index.of | 5, 35, 85, 101, 135, 152, 184 |
| abstract_inverted_index.on | 28, 170 |
| abstract_inverted_index.to | 109, 144, 163, 167, 175 |
| abstract_inverted_index.we | 130 |
| abstract_inverted_index.411 | 50 |
| abstract_inverted_index.LLM | 185 |
| abstract_inverted_index.Our | 75 |
| abstract_inverted_index.and | 12, 31, 67, 93, 115, 128, 174, 181 |
| abstract_inverted_index.for | 15, 82, 125 |
| abstract_inverted_index.our | 39, 158 |
| abstract_inverted_index.the | 3, 149, 155, 161, 179 |
| abstract_inverted_index.use | 4, 151 |
| abstract_inverted_index.Form | 73 |
| abstract_inverted_index.LLMs | 81, 122, 153 |
| abstract_inverted_index.They | 97 |
| abstract_inverted_index.This | 0 |
| abstract_inverted_index.case | 95 |
| abstract_inverted_index.code | 88 |
| abstract_inverted_index.from | 52, 105 |
| abstract_inverted_index.gap. | 43 |
| abstract_inverted_index.like | 112 |
| abstract_inverted_index.many | 138 |
| abstract_inverted_index.need | 162 |
| abstract_inverted_index.test | 94 |
| abstract_inverted_index.that | 78 |
| abstract_inverted_index.they | 62 |
| abstract_inverted_index.this | 42 |
| abstract_inverted_index.with | 137 |
| abstract_inverted_index.work | 40 |
| abstract_inverted_index.Given | 148 |
| abstract_inverted_index.While | 118 |
| abstract_inverted_index.about | 178 |
| abstract_inverted_index.basic | 106 |
| abstract_inverted_index.class | 56 |
| abstract_inverted_index.fills | 41 |
| abstract_inverted_index.lacks | 32 |
| abstract_inverted_index.large | 6 |
| abstract_inverted_index.noted | 131 |
| abstract_inverted_index.raise | 176 |
| abstract_inverted_index.study | 1, 159 |
| abstract_inverted_index.their | 69 |
| abstract_inverted_index.three | 64 |
| abstract_inverted_index.trend | 134 |
| abstract_inverted_index.usage | 186 |
| abstract_inverted_index.where | 61 |
| abstract_inverted_index.which | 25 |
| abstract_inverted_index.(LLMs) | 9 |
| abstract_inverted_index.Google | 72 |
| abstract_inverted_index.Indian | 59 |
| abstract_inverted_index.Unlike | 22 |
| abstract_inverted_index.actual | 36 |
| abstract_inverted_index.entire | 141 |
| abstract_inverted_index.models | 8 |
| abstract_inverted_index.obtain | 145 |
| abstract_inverted_index.reveal | 77 |
| abstract_inverted_index.shared | 68 |
| abstract_inverted_index.tasks, | 86 |
| abstract_inverted_index.update | 164 |
| abstract_inverted_index.viewed | 121 |
| abstract_inverted_index.Systems | 55 |
| abstract_inverted_index.classes | 30 |
| abstract_inverted_index.focuses | 27 |
| abstract_inverted_index.include | 168 |
| abstract_inverted_index.prompts | 108 |
| abstract_inverted_index.ranging | 104 |
| abstract_inverted_index.through | 71 |
| abstract_inverted_index.variety | 84 |
| abstract_inverted_index.academic | 188 |
| abstract_inverted_index.advanced | 19, 110 |
| abstract_inverted_index.analysis | 34, 48 |
| abstract_inverted_index.benefits | 180 |
| abstract_inverted_index.classes. | 21 |
| abstract_inverted_index.complete | 146 |
| abstract_inverted_index.employed | 98 |
| abstract_inverted_index.examines | 2 |
| abstract_inverted_index.existing | 23 |
| abstract_inverted_index.findings | 76 |
| abstract_inverted_index.graduate | 13 |
| abstract_inverted_index.in-depth | 33 |
| abstract_inverted_index.language | 7 |
| abstract_inverted_index.software | 156 |
| abstract_inverted_index.spectrum | 100 |
| abstract_inverted_index.students | 14, 51, 79, 119, 139 |
| abstract_inverted_index.surveys. | 74 |
| abstract_inverted_index.training | 169 |
| abstract_inverted_index.awareness | 177 |
| abstract_inverted_index.completed | 63 |
| abstract_inverted_index.computing | 20 |
| abstract_inverted_index.conducted | 45 |
| abstract_inverted_index.creation. | 96 |
| abstract_inverted_index.curricula | 166 |
| abstract_inverted_index.drawbacks | 183 |
| abstract_inverted_index.effective | 171 |
| abstract_inverted_index.enhancing | 126 |
| abstract_inverted_index.generally | 120 |
| abstract_inverted_index.including | 87 |
| abstract_inverted_index.industry, | 157 |
| abstract_inverted_index.involving | 49 |
| abstract_inverted_index.iterative | 116 |
| abstract_inverted_index.learning, | 129 |
| abstract_inverted_index.leveraged | 80 |
| abstract_inverted_index.potential | 182 |
| abstract_inverted_index.primarily | 26 |
| abstract_inverted_index.prompting | 102, 114, 172 |
| abstract_inverted_index.research, | 24 |
| abstract_inverted_index.settings. | 189 |
| abstract_inverted_index.assignment | 142 |
| abstract_inverted_index.beneficial | 124 |
| abstract_inverted_index.conceptual | 91 |
| abstract_inverted_index.concerning | 133 |
| abstract_inverted_index.contextual | 107 |
| abstract_inverted_index.debugging, | 90 |
| abstract_inverted_index.highlights | 160 |
| abstract_inverted_index.increasing | 150 |
| abstract_inverted_index.inquiries, | 92 |
| abstract_inverted_index.solutions. | 147 |
| abstract_inverted_index.strategies | 173 |
| abstract_inverted_index.submitting | 140 |
| abstract_inverted_index.techniques | 111 |
| abstract_inverted_index.Distributed | 54 |
| abstract_inverted_index.assignments | 17, 66 |
| abstract_inverted_index.experiences | 70 |
| abstract_inverted_index.generation, | 89 |
| abstract_inverted_index.programming | 16, 65 |
| abstract_inverted_index.refinement. | 117 |
| abstract_inverted_index.strategies, | 103 |
| abstract_inverted_index.student-LLM | 37 |
| abstract_inverted_index.university, | 60 |
| abstract_inverted_index.descriptions | 143 |
| abstract_inverted_index.introductory | 29 |
| abstract_inverted_index.productivity | 127 |
| abstract_inverted_index.comprehensive | 47 |
| abstract_inverted_index.interactions, | 38 |
| abstract_inverted_index.undergraduate | 11, 165 |
| abstract_inverted_index.over-reliance, | 136 |
| abstract_inverted_index.chain-of-thought | 113 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 8 |
| citation_normalized_percentile |