Standing on the shoulders of giants Article Swipe

PDF

Lucas Cardoso , José de Sousa Ribeiro Filho , Vitor Cirilo Araujo Santos , Regiane S. Kawasaki Francês , Ronnie Alves ·

YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2409.03151

Although fundamental to the advancement of Machine Learning, the classic evaluation metrics extracted from the confusion matrix, such as precision and F1, are limited. Such metrics only offer a quantitative view of the models' performance, without considering the complexity of the data or the quality of the hit. To overcome these limitations, recent research has introduced the use of psychometric metrics such as Item Response Theory (IRT), which allows an assessment at the level of latent characteristics of instances. This work investigates how IRT concepts can enrich a confusion matrix in order to identify which model is the most appropriate among options with similar performance. In the study carried out, IRT does not replace, but complements classical metrics by offering a new layer of evaluation and observation of the fine behavior of models in specific instances. It was also observed that there is 97% confidence that the score from the IRT has different contributions from 66% of the classical metrics analyzed.

Related Topics

Concepts

Shoulders Astronomy Economics Physics Medicine Surgery

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2409.03151
PDF: https://arxiv.org/pdf/2409.03151
OA Status: green
Related Works: 10
OpenAlex ID: https://openalex.org/W4403584135

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4403584135

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2409.03151

Digital Object Identifier
Title: Standing on the shoulders of giants

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2024

Year of publication
Publication date: 2024-09-05

Full publication date if available
Authors: Lucas Cardoso, José de Sousa Ribeiro Filho, Vitor Cirilo Araujo Santos, Regiane S. Kawasaki Francês, Ronnie Alves

List of authors in order
Landing page: https://arxiv.org/abs/2409.03151

Publisher landing page
PDF URL: https://arxiv.org/pdf/2409.03151

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2409.03151

Direct OA link when available
Concepts: Shoulders, Astronomy, Economics, Physics, Medicine, Surgery

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4403584135
doi	https://doi.org/10.48550/arxiv.2409.03151
ids.doi	https://doi.org/10.48550/arxiv.2409.03151
ids.openalex	https://openalex.org/W4403584135
fwci
type	preprint
title	Standing on the shoulders of giants
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C2777325788
concepts[0].level	2
concepts[0].score	0.8650903701782227
concepts[0].wikidata	https://www.wikidata.org/wiki/Q16363
concepts[0].display_name	Shoulders
concepts[1].id	https://openalex.org/C1276947
concepts[1].level	1
concepts[1].score	0.3478550910949707
concepts[1].wikidata	https://www.wikidata.org/wiki/Q333
concepts[1].display_name	Astronomy
concepts[2].id	https://openalex.org/C162324750
concepts[2].level	0
concepts[2].score	0.3242659270763397
concepts[2].wikidata	https://www.wikidata.org/wiki/Q8134
concepts[2].display_name	Economics
concepts[3].id	https://openalex.org/C121332964
concepts[3].level	0
concepts[3].score	0.24401184916496277
concepts[3].wikidata	https://www.wikidata.org/wiki/Q413
concepts[3].display_name	Physics
concepts[4].id	https://openalex.org/C71924100
concepts[4].level	0
concepts[4].score	0.14781638979911804
concepts[4].wikidata	https://www.wikidata.org/wiki/Q11190
concepts[4].display_name	Medicine
concepts[5].id	https://openalex.org/C141071460
concepts[5].level	1
concepts[5].score	0.0
concepts[5].wikidata	https://www.wikidata.org/wiki/Q40821
concepts[5].display_name	Surgery
keywords[0].id	https://openalex.org/keywords/shoulders
keywords[0].score	0.8650903701782227
keywords[0].display_name	Shoulders
keywords[1].id	https://openalex.org/keywords/astronomy
keywords[1].score	0.3478550910949707
keywords[1].display_name	Astronomy
keywords[2].id	https://openalex.org/keywords/economics
keywords[2].score	0.3242659270763397
keywords[2].display_name	Economics
keywords[3].id	https://openalex.org/keywords/physics
keywords[3].score	0.24401184916496277
keywords[3].display_name	Physics
keywords[4].id	https://openalex.org/keywords/medicine
keywords[4].score	0.14781638979911804
keywords[4].display_name	Medicine
language	en
locations[0].id	pmh:oai:arXiv.org:2409.03151
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2409.03151
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2409.03151
locations[1].id	doi:10.48550/arxiv.2409.03151
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license	cc-by
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id	https://openalex.org/licenses/cc-by
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2409.03151
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5101244526
authorships[0].author.orcid
authorships[0].author.display_name	Lucas Cardoso
authorships[0].author_position	first
authorships[0].raw_author_name	Cardoso, Lucas Felipe Ferraro
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5024245764
authorships[1].author.orcid	https://orcid.org/0000-0002-8836-4188
authorships[1].author.display_name	José de Sousa Ribeiro Filho
authorships[1].author_position	middle
authorships[1].raw_author_name	Filho, José de Sousa Ribeiro
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5103073179
authorships[2].author.orcid	https://orcid.org/0000-0002-7960-3079
authorships[2].author.display_name	Vitor Cirilo Araujo Santos
authorships[2].author_position	middle
authorships[2].raw_author_name	Santos, Vitor Cirilo Araujo
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5068068844
authorships[3].author.orcid	https://orcid.org/0000-0003-3958-064X
authorships[3].author.display_name	Regiane S. Kawasaki Francês
authorships[3].author_position	middle
authorships[3].raw_author_name	Frances, Regiane Silva Kawasaki
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5000457683
authorships[4].author.orcid	https://orcid.org/0000-0003-4139-0562
authorships[4].author.display_name	Ronnie Alves
authorships[4].author_position	last
authorships[4].raw_author_name	Alves, Ronnie Cley de Oliveira
authorships[4].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2409.03151
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Standing on the shoulders of giants
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic
related_works	https://openalex.org/W2748952813, https://openalex.org/W4391375266, https://openalex.org/W2097168487, https://openalex.org/W2069513325, https://openalex.org/W2097970179, https://openalex.org/W2791520673, https://openalex.org/W1144894844, https://openalex.org/W1914928029, https://openalex.org/W2050109828, https://openalex.org/W2137863352
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2409.03151
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2409.03151
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2409.03151
primary_location.id	pmh:oai:arXiv.org:2409.03151
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2409.03151
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2409.03151
publication_date	2024-09-05
publication_year	2024
referenced_works_count	0
abstract_inverted_index.a	28, 87, 120
abstract_inverted_index.In	105
abstract_inverted_index.It	136
abstract_inverted_index.To	48
abstract_inverted_index.an	69
abstract_inverted_index.as	18, 62
abstract_inverted_index.at	71
abstract_inverted_index.by	118
abstract_inverted_index.in	90, 133
abstract_inverted_index.is	96, 142
abstract_inverted_index.of	5, 31, 39, 45, 58, 74, 77, 123, 127, 131, 156
abstract_inverted_index.or	42
abstract_inverted_index.to	2, 92
abstract_inverted_index.66%	155
abstract_inverted_index.97%	143
abstract_inverted_index.F1,	21
abstract_inverted_index.IRT	83, 110, 150
abstract_inverted_index.and	20, 125
abstract_inverted_index.are	22
abstract_inverted_index.but	114
abstract_inverted_index.can	85
abstract_inverted_index.has	54, 151
abstract_inverted_index.how	82
abstract_inverted_index.new	121
abstract_inverted_index.not	112
abstract_inverted_index.the	3, 8, 14, 32, 37, 40, 43, 46, 56, 72, 97, 106, 128, 146, 149, 157
abstract_inverted_index.use	57
abstract_inverted_index.was	137
abstract_inverted_index.Item	63
abstract_inverted_index.Such	24
abstract_inverted_index.This	79
abstract_inverted_index.also	138
abstract_inverted_index.data	41
abstract_inverted_index.does	111
abstract_inverted_index.fine	129
abstract_inverted_index.from	13, 148, 154
abstract_inverted_index.hit.	47
abstract_inverted_index.most	98
abstract_inverted_index.only	26
abstract_inverted_index.out,	109
abstract_inverted_index.such	17, 61
abstract_inverted_index.that	140, 145
abstract_inverted_index.view	30
abstract_inverted_index.with	102
abstract_inverted_index.work	80
abstract_inverted_index.among	100
abstract_inverted_index.layer	122
abstract_inverted_index.level	73
abstract_inverted_index.model	95
abstract_inverted_index.offer	27
abstract_inverted_index.order	91
abstract_inverted_index.score	147
abstract_inverted_index.study	107
abstract_inverted_index.there	141
abstract_inverted_index.these	50
abstract_inverted_index.which	67, 94
abstract_inverted_index.(IRT),	66
abstract_inverted_index.Theory	65
abstract_inverted_index.allows	68
abstract_inverted_index.enrich	86
abstract_inverted_index.latent	75
abstract_inverted_index.matrix	89
abstract_inverted_index.models	132
abstract_inverted_index.recent	52
abstract_inverted_index.Machine	6
abstract_inverted_index.carried	108
abstract_inverted_index.classic	9
abstract_inverted_index.matrix,	16
abstract_inverted_index.metrics	11, 25, 60, 117, 159
abstract_inverted_index.models'	33
abstract_inverted_index.options	101
abstract_inverted_index.quality	44
abstract_inverted_index.similar	103
abstract_inverted_index.without	35
abstract_inverted_index.Although	0
abstract_inverted_index.Response	64
abstract_inverted_index.behavior	130
abstract_inverted_index.concepts	84
abstract_inverted_index.identify	93
abstract_inverted_index.limited.	23
abstract_inverted_index.observed	139
abstract_inverted_index.offering	119
abstract_inverted_index.overcome	49
abstract_inverted_index.replace,	113
abstract_inverted_index.research	53
abstract_inverted_index.specific	134
abstract_inverted_index.Learning,	7
abstract_inverted_index.analyzed.	160
abstract_inverted_index.classical	116, 158
abstract_inverted_index.confusion	15, 88
abstract_inverted_index.different	152
abstract_inverted_index.extracted	12
abstract_inverted_index.precision	19
abstract_inverted_index.assessment	70
abstract_inverted_index.complexity	38
abstract_inverted_index.confidence	144
abstract_inverted_index.evaluation	10, 124
abstract_inverted_index.instances.	78, 135
abstract_inverted_index.introduced	55
abstract_inverted_index.advancement	4
abstract_inverted_index.appropriate	99
abstract_inverted_index.complements	115
abstract_inverted_index.considering	36
abstract_inverted_index.fundamental	1
abstract_inverted_index.observation	126
abstract_inverted_index.investigates	81
abstract_inverted_index.limitations,	51
abstract_inverted_index.performance,	34
abstract_inverted_index.performance.	104
abstract_inverted_index.psychometric	59
abstract_inverted_index.quantitative	29
abstract_inverted_index.contributions	153
abstract_inverted_index.characteristics	76
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	5
citation_normalized_percentile