Large Language Models for Detecting Bias in Job Descriptions Article Swipe

PDF

Tristan Everitt , Paul Ryan , Brian M. Davis , Kolawole John Adebayo ·

YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.17454/ijcol102.05

This study explores the application of large language (LLM) models for detecting implicit bias in job descriptions, an important concern in human resources that shapes applicant pools and influences employer perception. We compare different LLM architectures—encoder, encoder-decoder, and decoder models—focusing on seven specific bias types. The research questions address the capability of foundation LLMs to detect implicit bias and the effectiveness of domain adaptation via fine-tuning versus prompt-tuning. Results indicate that fine-tuned models are more effective in detecting biases, with Flan-T5-XL emerging as the top performer, surpassing the zero-shot prompting of GPT-4o model. A labelled dataset consisting of verified gold-standard, silver-standard, and unverified bronze-standard data was created for this purpose and open-sourced1 to advance the field and serve as a valuable resource for future research.

Related Topics

Computer Science

Concepts

Computer science Natural language processing

Metadata

Type: article
Language: en
Landing Page: https://doi.org/10.17454/ijcol102.05
PDF: http://journals.openedition.org/ijcol/pdf/1454
OA Status: diamond
Related Works: 10
OpenAlex ID: https://openalex.org/W4411068020

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4411068020

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.17454/ijcol102.05

Digital Object Identifier
Title: Large Language Models for Detecting Bias in Job Descriptions

Work title
Type: article

OpenAlex work type
Language: en

Primary language
Publication year: 2024

Year of publication
Publication date: 2024-01-01

Full publication date if available
Authors: Tristan Everitt, Paul Ryan, Brian M. Davis, Kolawole John Adebayo

List of authors in order
Landing page: https://doi.org/10.17454/ijcol102.05

Publisher landing page
PDF URL: https://journals.openedition.org/ijcol/pdf/1454

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: diamond

Open access status per OpenAlex
OA URL: https://journals.openedition.org/ijcol/pdf/1454

Direct OA link when available
Concepts: Computer science, Natural language processing

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4411068020
doi	https://doi.org/10.17454/ijcol102.05
ids.doi	https://doi.org/10.17454/ijcol102.05
ids.openalex	https://openalex.org/W4411068020
fwci	0.0
type	article
title	Large Language Models for Detecting Bias in Job Descriptions
biblio.issue	2
biblio.volume	10
biblio.last_page	79
biblio.first_page	79
topics[0].id	https://openalex.org/T10809
topics[0].field.id	https://openalex.org/fields/36
topics[0].field.display_name	Health Professions
topics[0].score	0.9584000110626221
topics[0].domain.id	https://openalex.org/domains/4
topics[0].domain.display_name	Health Sciences
topics[0].subfield.id	https://openalex.org/subfields/3614
topics[0].subfield.display_name	Radiological and Ultrasound Technology
topics[0].display_name	Occupational Health and Safety Research
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C41008148
concepts[0].level	0
concepts[0].score	0.539740264415741
concepts[0].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[0].display_name	Computer science
concepts[1].id	https://openalex.org/C204321447
concepts[1].level	1
concepts[1].score	0.36411622166633606
concepts[1].wikidata	https://www.wikidata.org/wiki/Q30642
concepts[1].display_name	Natural language processing
keywords[0].id	https://openalex.org/keywords/computer-science
keywords[0].score	0.539740264415741
keywords[0].display_name	Computer science
keywords[1].id	https://openalex.org/keywords/natural-language-processing
keywords[1].score	0.36411622166633606
keywords[1].display_name	Natural language processing
language	en
locations[0].id	doi:10.17454/ijcol102.05
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4210219754
locations[0].source.issn	2499-4553
locations[0].source.type	journal
locations[0].source.is_oa	True
locations[0].source.issn_l	2499-4553
locations[0].source.is_core	True
locations[0].source.is_in_doaj	True
locations[0].source.display_name	Italian Journal of Computational Linguistics
locations[0].source.host_organization	https://openalex.org/P4310311504
locations[0].source.host_organization_name	UMR ESPACE et UMR LISST
locations[0].source.host_organization_lineage	https://openalex.org/P4310311504
locations[0].source.host_organization_lineage_names	UMR ESPACE et UMR LISST
locations[0].license	cc-by-nc-nd
locations[0].pdf_url	http://journals.openedition.org/ijcol/pdf/1454
locations[0].version	publishedVersion
locations[0].raw_type	journal-article
locations[0].license_id	https://openalex.org/licenses/cc-by-nc-nd
locations[0].is_accepted	True
locations[0].is_published	True
locations[0].raw_source_name	Italian Journal of Computational Linguistics
locations[0].landing_page_url	https://doi.org/10.17454/ijcol102.05
indexed_in	crossref, doaj
authorships[0].author.id	https://openalex.org/A5117844721
authorships[0].author.orcid
authorships[0].author.display_name	Tristan Everitt
authorships[0].author_position	first
authorships[0].raw_author_name	Tristan Everitt
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5054860855
authorships[1].author.orcid	https://orcid.org/0000-0003-0770-2737
authorships[1].author.display_name	Paul Ryan
authorships[1].author_position	middle
authorships[1].raw_author_name	Paul Ryan
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5078209252
authorships[2].author.orcid	https://orcid.org/0000-0002-4646-0569
authorships[2].author.display_name	Brian M. Davis
authorships[2].author_position	middle
authorships[2].raw_author_name	Brian Davis
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5017923726
authorships[3].author.orcid	https://orcid.org/0000-0001-7126-7026
authorships[3].author.display_name	Kolawole John Adebayo
authorships[3].author_position	last
authorships[3].raw_author_name	Kolawole J. Adebayo
authorships[3].is_corresponding	False
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	http://journals.openedition.org/ijcol/pdf/1454
open_access.oa_status	diamond
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Large Language Models for Detecting Bias in Job Descriptions
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T03:46:38.306776
primary_topic.id	https://openalex.org/T10809
primary_topic.field.id	https://openalex.org/fields/36
primary_topic.field.display_name	Health Professions
primary_topic.score	0.9584000110626221
primary_topic.domain.id	https://openalex.org/domains/4
primary_topic.domain.display_name	Health Sciences
primary_topic.subfield.id	https://openalex.org/subfields/3614
primary_topic.subfield.display_name	Radiological and Ultrasound Technology
primary_topic.display_name	Occupational Health and Safety Research
related_works	https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W4391913857, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052
cited_by_count	0
locations_count	1
best_oa_location.id	doi:10.17454/ijcol102.05
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4210219754
best_oa_location.source.issn	2499-4553
best_oa_location.source.type	journal
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l	2499-4553
best_oa_location.source.is_core	True
best_oa_location.source.is_in_doaj	True
best_oa_location.source.display_name	Italian Journal of Computational Linguistics
best_oa_location.source.host_organization	https://openalex.org/P4310311504
best_oa_location.source.host_organization_name	UMR ESPACE et UMR LISST
best_oa_location.source.host_organization_lineage	https://openalex.org/P4310311504
best_oa_location.source.host_organization_lineage_names	UMR ESPACE et UMR LISST
best_oa_location.license	cc-by-nc-nd
best_oa_location.pdf_url	http://journals.openedition.org/ijcol/pdf/1454
best_oa_location.version	publishedVersion
best_oa_location.raw_type	journal-article
best_oa_location.license_id	https://openalex.org/licenses/cc-by-nc-nd
best_oa_location.is_accepted	True
best_oa_location.is_published	True
best_oa_location.raw_source_name	Italian Journal of Computational Linguistics
best_oa_location.landing_page_url	https://doi.org/10.17454/ijcol102.05
primary_location.id	doi:10.17454/ijcol102.05
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4210219754
primary_location.source.issn	2499-4553
primary_location.source.type	journal
primary_location.source.is_oa	True
primary_location.source.issn_l	2499-4553
primary_location.source.is_core	True
primary_location.source.is_in_doaj	True
primary_location.source.display_name	Italian Journal of Computational Linguistics
primary_location.source.host_organization	https://openalex.org/P4310311504
primary_location.source.host_organization_name	UMR ESPACE et UMR LISST
primary_location.source.host_organization_lineage	https://openalex.org/P4310311504
primary_location.source.host_organization_lineage_names	UMR ESPACE et UMR LISST
primary_location.license	cc-by-nc-nd
primary_location.pdf_url	http://journals.openedition.org/ijcol/pdf/1454
primary_location.version	publishedVersion
primary_location.raw_type	journal-article
primary_location.license_id	https://openalex.org/licenses/cc-by-nc-nd
primary_location.is_accepted	True
primary_location.is_published	True
primary_location.raw_source_name	Italian Journal of Computational Linguistics
primary_location.landing_page_url	https://doi.org/10.17454/ijcol102.05
publication_date	2024-01-01
publication_year	2024
referenced_works_count	0
abstract_inverted_index.A	93
abstract_inverted_index.a	119
abstract_inverted_index.We	31
abstract_inverted_index.an	17
abstract_inverted_index.as	82, 118
abstract_inverted_index.in	14, 20, 76
abstract_inverted_index.of	5, 51, 61, 90, 97
abstract_inverted_index.on	40
abstract_inverted_index.to	54, 112
abstract_inverted_index.LLM	34
abstract_inverted_index.The	45
abstract_inverted_index.and	27, 37, 58, 101, 110, 116
abstract_inverted_index.are	73
abstract_inverted_index.for	10, 107, 122
abstract_inverted_index.job	15
abstract_inverted_index.the	3, 49, 59, 83, 87, 114
abstract_inverted_index.top	84
abstract_inverted_index.via	64
abstract_inverted_index.was	105
abstract_inverted_index.LLMs	53
abstract_inverted_index.This	0
abstract_inverted_index.bias	13, 43, 57
abstract_inverted_index.data	104
abstract_inverted_index.more	74
abstract_inverted_index.that	23, 70
abstract_inverted_index.this	108
abstract_inverted_index.with	79
abstract_inverted_index.(LLM)	8
abstract_inverted_index.field	115
abstract_inverted_index.human	21
abstract_inverted_index.large	6
abstract_inverted_index.pools	26
abstract_inverted_index.serve	117
abstract_inverted_index.seven	41
abstract_inverted_index.study	1
abstract_inverted_index.GPT-4o	91
abstract_inverted_index.detect	55
abstract_inverted_index.domain	62
abstract_inverted_index.future	123
abstract_inverted_index.model.	92
abstract_inverted_index.models	9, 72
abstract_inverted_index.shapes	24
abstract_inverted_index.types.	44
abstract_inverted_index.versus	66
abstract_inverted_index.Results	68
abstract_inverted_index.address	48
abstract_inverted_index.advance	113
abstract_inverted_index.biases,	78
abstract_inverted_index.compare	32
abstract_inverted_index.concern	19
abstract_inverted_index.created	106
abstract_inverted_index.dataset	95
abstract_inverted_index.decoder	38
abstract_inverted_index.purpose	109
abstract_inverted_index.emerging	81
abstract_inverted_index.employer	29
abstract_inverted_index.explores	2
abstract_inverted_index.implicit	12, 56
abstract_inverted_index.indicate	69
abstract_inverted_index.labelled	94
abstract_inverted_index.language	7
abstract_inverted_index.research	46
abstract_inverted_index.resource	121
abstract_inverted_index.specific	42
abstract_inverted_index.valuable	120
abstract_inverted_index.verified	98
abstract_inverted_index.applicant	25
abstract_inverted_index.detecting	11, 77
abstract_inverted_index.different	33
abstract_inverted_index.effective	75
abstract_inverted_index.important	18
abstract_inverted_index.prompting	89
abstract_inverted_index.questions	47
abstract_inverted_index.research.	124
abstract_inverted_index.resources	22
abstract_inverted_index.zero-shot	88
abstract_inverted_index.Flan-T5-XL	80
abstract_inverted_index.adaptation	63
abstract_inverted_index.capability	50
abstract_inverted_index.consisting	96
abstract_inverted_index.fine-tuned	71
abstract_inverted_index.foundation	52
abstract_inverted_index.influences	28
abstract_inverted_index.performer,	85
abstract_inverted_index.surpassing	86
abstract_inverted_index.unverified	102
abstract_inverted_index.application	4
abstract_inverted_index.fine-tuning	65
abstract_inverted_index.perception.	30
abstract_inverted_index.descriptions,	16
abstract_inverted_index.effectiveness	60
abstract_inverted_index.open-sourced1	111
abstract_inverted_index.gold-standard,	99
abstract_inverted_index.prompt-tuning.	67
abstract_inverted_index.bronze-standard	103
abstract_inverted_index.encoder-decoder,	36
abstract_inverted_index.silver-standard,	100
abstract_inverted_index.models—focusing	39
abstract_inverted_index.architectures—encoder,	35
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	4
citation_normalized_percentile.value	0.4796521
citation_normalized_percentile.is_in_top_1_percent	False
citation_normalized_percentile.is_in_top_10_percent	False