Estimating and Evaluating the Uncertainty of Rating Predictions and Top-n Recommendations in Recommender Systems Article Swipe
Victor Coscrato
,
Derek Bridge
·
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.1145/3584021
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.1145/3584021
Uncertainty is a characteristic of every data-driven application, including recommender systems. The quantification of uncertainty can be key to increasing user trust in recommendations or choosing which recommendations should be accompanied by an explanation; uncertainty estimates can be used to accomplish recommender tasks such as active learning and co-training. Many uncertainty estimators are available, but to date, the literature has lacked a comprehensive survey and a detailed comparison. In this article, we fulfill these needs. We review the existing methods for uncertainty estimation and metrics for evaluating uncertainty estimates, while also proposing some estimation methods and evaluation metrics of our own. Using two datasets, we compare the methods using the evaluation metrics that we describe, and we discuss their strengths and potential issues. The goal of this work is to provide a foundation to the field of uncertainty estimation in recommender systems, on which further research can be built.
Related Topics
Concepts
Recommender system
Computer science
Estimator
Estimation
Uncertainty quantification
Field (mathematics)
Uncertainty analysis
Key (lock)
Data mining
Sensitivity analysis
Machine learning
Measurement uncertainty
Artificial intelligence
Data science
Statistics
Simulation
Mathematics
Engineering
Computer security
Systems engineering
Pure mathematics
Metadata
- Type
- article
- Language
- en
- Landing Page
- https://doi.org/10.1145/3584021
- OA Status
- green
- Cited By
- 13
- References
- 35
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4321109316
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4321109316Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.1145/3584021Digital Object Identifier
- Title
-
Estimating and Evaluating the Uncertainty of Rating Predictions and Top-n Recommendations in Recommender SystemsWork title
- Type
-
articleOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2023Year of publication
- Publication date
-
2023-02-16Full publication date if available
- Authors
-
Victor Coscrato, Derek BridgeList of authors in order
- Landing page
-
https://doi.org/10.1145/3584021Publisher landing page
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://cora.ucc.ie/bitstreams/b5747c78-75e0-4d5a-b935-f3c7f4a39d1d/downloadDirect OA link when available
- Concepts
-
Recommender system, Computer science, Estimator, Estimation, Uncertainty quantification, Field (mathematics), Uncertainty analysis, Key (lock), Data mining, Sensitivity analysis, Machine learning, Measurement uncertainty, Artificial intelligence, Data science, Statistics, Simulation, Mathematics, Engineering, Computer security, Systems engineering, Pure mathematicsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
13Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 6, 2024: 5, 2023: 2Per-year citation counts (last 5 years)
- References (count)
-
35Number of works referenced by this work
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4321109316 |
|---|---|
| doi | https://doi.org/10.1145/3584021 |
| ids.doi | https://doi.org/10.1145/3584021 |
| ids.openalex | https://openalex.org/W4321109316 |
| fwci | 8.04033915 |
| type | article |
| title | Estimating and Evaluating the Uncertainty of Rating Predictions and Top-n Recommendations in Recommender Systems |
| awards[0].id | https://openalex.org/G2394637485 |
| awards[0].funder_id | https://openalex.org/F4320320847 |
| awards[0].display_name | |
| awards[0].funder_award_id | 18/CRT/6223 |
| awards[0].funder_display_name | Science Foundation Ireland |
| biblio.issue | 2 |
| biblio.volume | 1 |
| biblio.last_page | 34 |
| biblio.first_page | 1 |
| topics[0].id | https://openalex.org/T10203 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9997000098228455 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1710 |
| topics[0].subfield.display_name | Information Systems |
| topics[0].display_name | Recommender Systems and Techniques |
| topics[1].id | https://openalex.org/T12101 |
| topics[1].field.id | https://openalex.org/fields/18 |
| topics[1].field.display_name | Decision Sciences |
| topics[1].score | 0.9991999864578247 |
| topics[1].domain.id | https://openalex.org/domains/2 |
| topics[1].domain.display_name | Social Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1803 |
| topics[1].subfield.display_name | Management Science and Operations Research |
| topics[1].display_name | Advanced Bandit Algorithms Research |
| topics[2].id | https://openalex.org/T12761 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9894999861717224 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Data Stream Mining Techniques |
| funders[0].id | https://openalex.org/F4320320847 |
| funders[0].ror | https://ror.org/0271asj38 |
| funders[0].display_name | Science Foundation Ireland |
| funders[1].id | https://openalex.org/F4320335322 |
| funders[1].ror | https://ror.org/00k4n6c32 |
| funders[1].display_name | European Regional Development Fund |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C557471498 |
| concepts[0].level | 2 |
| concepts[0].score | 0.8748457431793213 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q554950 |
| concepts[0].display_name | Recommender system |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.7773169279098511 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C185429906 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6576180458068848 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q1130160 |
| concepts[2].display_name | Estimator |
| concepts[3].id | https://openalex.org/C96250715 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5818238258361816 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q965330 |
| concepts[3].display_name | Estimation |
| concepts[4].id | https://openalex.org/C32230216 |
| concepts[4].level | 2 |
| concepts[4].score | 0.5427818894386292 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q7882499 |
| concepts[4].display_name | Uncertainty quantification |
| concepts[5].id | https://openalex.org/C9652623 |
| concepts[5].level | 2 |
| concepts[5].score | 0.5406176447868347 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q190109 |
| concepts[5].display_name | Field (mathematics) |
| concepts[6].id | https://openalex.org/C177803969 |
| concepts[6].level | 2 |
| concepts[6].score | 0.5063964128494263 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q29205 |
| concepts[6].display_name | Uncertainty analysis |
| concepts[7].id | https://openalex.org/C26517878 |
| concepts[7].level | 2 |
| concepts[7].score | 0.4628232419490814 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q228039 |
| concepts[7].display_name | Key (lock) |
| concepts[8].id | https://openalex.org/C124101348 |
| concepts[8].level | 1 |
| concepts[8].score | 0.44870439171791077 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q172491 |
| concepts[8].display_name | Data mining |
| concepts[9].id | https://openalex.org/C176147448 |
| concepts[9].level | 3 |
| concepts[9].score | 0.4351784586906433 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q1889114 |
| concepts[9].display_name | Sensitivity analysis |
| concepts[10].id | https://openalex.org/C119857082 |
| concepts[10].level | 1 |
| concepts[10].score | 0.42547863721847534 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[10].display_name | Machine learning |
| concepts[11].id | https://openalex.org/C137209882 |
| concepts[11].level | 2 |
| concepts[11].score | 0.4152575135231018 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q1403517 |
| concepts[11].display_name | Measurement uncertainty |
| concepts[12].id | https://openalex.org/C154945302 |
| concepts[12].level | 1 |
| concepts[12].score | 0.3436892628669739 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[12].display_name | Artificial intelligence |
| concepts[13].id | https://openalex.org/C2522767166 |
| concepts[13].level | 1 |
| concepts[13].score | 0.3284046947956085 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q2374463 |
| concepts[13].display_name | Data science |
| concepts[14].id | https://openalex.org/C105795698 |
| concepts[14].level | 1 |
| concepts[14].score | 0.11012458801269531 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q12483 |
| concepts[14].display_name | Statistics |
| concepts[15].id | https://openalex.org/C44154836 |
| concepts[15].level | 1 |
| concepts[15].score | 0.0920407772064209 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q45045 |
| concepts[15].display_name | Simulation |
| concepts[16].id | https://openalex.org/C33923547 |
| concepts[16].level | 0 |
| concepts[16].score | 0.07757580280303955 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[16].display_name | Mathematics |
| concepts[17].id | https://openalex.org/C127413603 |
| concepts[17].level | 0 |
| concepts[17].score | 0.06844380497932434 |
| concepts[17].wikidata | https://www.wikidata.org/wiki/Q11023 |
| concepts[17].display_name | Engineering |
| concepts[18].id | https://openalex.org/C38652104 |
| concepts[18].level | 1 |
| concepts[18].score | 0.0 |
| concepts[18].wikidata | https://www.wikidata.org/wiki/Q3510521 |
| concepts[18].display_name | Computer security |
| concepts[19].id | https://openalex.org/C201995342 |
| concepts[19].level | 1 |
| concepts[19].score | 0.0 |
| concepts[19].wikidata | https://www.wikidata.org/wiki/Q682496 |
| concepts[19].display_name | Systems engineering |
| concepts[20].id | https://openalex.org/C202444582 |
| concepts[20].level | 1 |
| concepts[20].score | 0.0 |
| concepts[20].wikidata | https://www.wikidata.org/wiki/Q837863 |
| concepts[20].display_name | Pure mathematics |
| keywords[0].id | https://openalex.org/keywords/recommender-system |
| keywords[0].score | 0.8748457431793213 |
| keywords[0].display_name | Recommender system |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.7773169279098511 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/estimator |
| keywords[2].score | 0.6576180458068848 |
| keywords[2].display_name | Estimator |
| keywords[3].id | https://openalex.org/keywords/estimation |
| keywords[3].score | 0.5818238258361816 |
| keywords[3].display_name | Estimation |
| keywords[4].id | https://openalex.org/keywords/uncertainty-quantification |
| keywords[4].score | 0.5427818894386292 |
| keywords[4].display_name | Uncertainty quantification |
| keywords[5].id | https://openalex.org/keywords/field |
| keywords[5].score | 0.5406176447868347 |
| keywords[5].display_name | Field (mathematics) |
| keywords[6].id | https://openalex.org/keywords/uncertainty-analysis |
| keywords[6].score | 0.5063964128494263 |
| keywords[6].display_name | Uncertainty analysis |
| keywords[7].id | https://openalex.org/keywords/key |
| keywords[7].score | 0.4628232419490814 |
| keywords[7].display_name | Key (lock) |
| keywords[8].id | https://openalex.org/keywords/data-mining |
| keywords[8].score | 0.44870439171791077 |
| keywords[8].display_name | Data mining |
| keywords[9].id | https://openalex.org/keywords/sensitivity-analysis |
| keywords[9].score | 0.4351784586906433 |
| keywords[9].display_name | Sensitivity analysis |
| keywords[10].id | https://openalex.org/keywords/machine-learning |
| keywords[10].score | 0.42547863721847534 |
| keywords[10].display_name | Machine learning |
| keywords[11].id | https://openalex.org/keywords/measurement-uncertainty |
| keywords[11].score | 0.4152575135231018 |
| keywords[11].display_name | Measurement uncertainty |
| keywords[12].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[12].score | 0.3436892628669739 |
| keywords[12].display_name | Artificial intelligence |
| keywords[13].id | https://openalex.org/keywords/data-science |
| keywords[13].score | 0.3284046947956085 |
| keywords[13].display_name | Data science |
| keywords[14].id | https://openalex.org/keywords/statistics |
| keywords[14].score | 0.11012458801269531 |
| keywords[14].display_name | Statistics |
| keywords[15].id | https://openalex.org/keywords/simulation |
| keywords[15].score | 0.0920407772064209 |
| keywords[15].display_name | Simulation |
| keywords[16].id | https://openalex.org/keywords/mathematics |
| keywords[16].score | 0.07757580280303955 |
| keywords[16].display_name | Mathematics |
| keywords[17].id | https://openalex.org/keywords/engineering |
| keywords[17].score | 0.06844380497932434 |
| keywords[17].display_name | Engineering |
| language | en |
| locations[0].id | doi:10.1145/3584021 |
| locations[0].is_oa | False |
| locations[0].source.id | https://openalex.org/S4387284325 |
| locations[0].source.issn | 2770-6699 |
| locations[0].source.type | journal |
| locations[0].source.is_oa | False |
| locations[0].source.issn_l | 2770-6699 |
| locations[0].source.is_core | True |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | ACM Transactions on Recommender Systems |
| locations[0].source.host_organization | https://openalex.org/P4310319798 |
| locations[0].source.host_organization_name | Association for Computing Machinery |
| locations[0].source.host_organization_lineage | https://openalex.org/P4310319798 |
| locations[0].source.host_organization_lineage_names | Association for Computing Machinery |
| locations[0].license | |
| locations[0].pdf_url | |
| locations[0].version | publishedVersion |
| locations[0].raw_type | journal-article |
| locations[0].license_id | |
| locations[0].is_accepted | True |
| locations[0].is_published | True |
| locations[0].raw_source_name | ACM Transactions on Recommender Systems |
| locations[0].landing_page_url | https://doi.org/10.1145/3584021 |
| locations[1].id | pmh:oai:cora.ucc.ie:10468/14296 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400766 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | False |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | Cork Open Research Archive (University College Cork) |
| locations[1].source.host_organization | https://openalex.org/I27577105 |
| locations[1].source.host_organization_name | University College Cork |
| locations[1].source.host_organization_lineage | https://openalex.org/I27577105 |
| locations[1].license | |
| locations[1].pdf_url | https://cora.ucc.ie/bitstreams/b5747c78-75e0-4d5a-b935-f3c7f4a39d1d/download |
| locations[1].version | submittedVersion |
| locations[1].raw_type | info:eu-repo/semantics/article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | False |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | http://hdl.handle.net/10468/14296 |
| indexed_in | crossref |
| authorships[0].author.id | https://openalex.org/A5064054034 |
| authorships[0].author.orcid | https://orcid.org/0000-0003-2779-3082 |
| authorships[0].author.display_name | Victor Coscrato |
| authorships[0].countries | IE |
| authorships[0].affiliations[0].institution_ids | https://openalex.org/I27577105 |
| authorships[0].affiliations[0].raw_affiliation_string | University College Cork, Ireland |
| authorships[0].institutions[0].id | https://openalex.org/I27577105 |
| authorships[0].institutions[0].ror | https://ror.org/03265fv13 |
| authorships[0].institutions[0].type | education |
| authorships[0].institutions[0].lineage | https://openalex.org/I27577105 |
| authorships[0].institutions[0].country_code | IE |
| authorships[0].institutions[0].display_name | University College Cork |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Victor Coscrato |
| authorships[0].is_corresponding | False |
| authorships[0].raw_affiliation_strings | University College Cork, Ireland |
| authorships[1].author.id | https://openalex.org/A5008002610 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-8720-3876 |
| authorships[1].author.display_name | Derek Bridge |
| authorships[1].countries | IE |
| authorships[1].affiliations[0].institution_ids | https://openalex.org/I27577105 |
| authorships[1].affiliations[0].raw_affiliation_string | University College Cork, Ireland |
| authorships[1].institutions[0].id | https://openalex.org/I27577105 |
| authorships[1].institutions[0].ror | https://ror.org/03265fv13 |
| authorships[1].institutions[0].type | education |
| authorships[1].institutions[0].lineage | https://openalex.org/I27577105 |
| authorships[1].institutions[0].country_code | IE |
| authorships[1].institutions[0].display_name | University College Cork |
| authorships[1].author_position | last |
| authorships[1].raw_author_name | Derek Bridge |
| authorships[1].is_corresponding | False |
| authorships[1].raw_affiliation_strings | University College Cork, Ireland |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://cora.ucc.ie/bitstreams/b5747c78-75e0-4d5a-b935-f3c7f4a39d1d/download |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Estimating and Evaluating the Uncertainty of Rating Predictions and Top-n Recommendations in Recommender Systems |
| has_fulltext | True |
| is_retracted | False |
| updated_date | 2025-11-06T03:46:38.306776 |
| primary_topic.id | https://openalex.org/T10203 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9997000098228455 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1710 |
| primary_topic.subfield.display_name | Information Systems |
| primary_topic.display_name | Recommender Systems and Techniques |
| related_works | https://openalex.org/W4205181462, https://openalex.org/W3212153563, https://openalex.org/W4249135812, https://openalex.org/W2116089868, https://openalex.org/W4312671192, https://openalex.org/W2376953431, https://openalex.org/W93577605, https://openalex.org/W2387053421, https://openalex.org/W2607177949, https://openalex.org/W4386584851 |
| cited_by_count | 13 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 6 |
| counts_by_year[1].year | 2024 |
| counts_by_year[1].cited_by_count | 5 |
| counts_by_year[2].year | 2023 |
| counts_by_year[2].cited_by_count | 2 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:cora.ucc.ie:10468/14296 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400766 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | False |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | Cork Open Research Archive (University College Cork) |
| best_oa_location.source.host_organization | https://openalex.org/I27577105 |
| best_oa_location.source.host_organization_name | University College Cork |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I27577105 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://cora.ucc.ie/bitstreams/b5747c78-75e0-4d5a-b935-f3c7f4a39d1d/download |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | info:eu-repo/semantics/article |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://hdl.handle.net/10468/14296 |
| primary_location.id | doi:10.1145/3584021 |
| primary_location.is_oa | False |
| primary_location.source.id | https://openalex.org/S4387284325 |
| primary_location.source.issn | 2770-6699 |
| primary_location.source.type | journal |
| primary_location.source.is_oa | False |
| primary_location.source.issn_l | 2770-6699 |
| primary_location.source.is_core | True |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | ACM Transactions on Recommender Systems |
| primary_location.source.host_organization | https://openalex.org/P4310319798 |
| primary_location.source.host_organization_name | Association for Computing Machinery |
| primary_location.source.host_organization_lineage | https://openalex.org/P4310319798 |
| primary_location.source.host_organization_lineage_names | Association for Computing Machinery |
| primary_location.license | |
| primary_location.pdf_url | |
| primary_location.version | publishedVersion |
| primary_location.raw_type | journal-article |
| primary_location.license_id | |
| primary_location.is_accepted | True |
| primary_location.is_published | True |
| primary_location.raw_source_name | ACM Transactions on Recommender Systems |
| primary_location.landing_page_url | https://doi.org/10.1145/3584021 |
| publication_date | 2023-02-16 |
| publication_year | 2023 |
| referenced_works | https://openalex.org/W1872493208, https://openalex.org/W2904422658, https://openalex.org/W2592771984, https://openalex.org/W2954069222, https://openalex.org/W2012575681, https://openalex.org/W2791544193, https://openalex.org/W90253967, https://openalex.org/W281665770, https://openalex.org/W2765564115, https://openalex.org/W2810522746, https://openalex.org/W2739749877, https://openalex.org/W2219888463, https://openalex.org/W1971040550, https://openalex.org/W3014596384, https://openalex.org/W2964691483, https://openalex.org/W1971460272, https://openalex.org/W2151052953, https://openalex.org/W2031849834, https://openalex.org/W4207015948, https://openalex.org/W2070689335, https://openalex.org/W3033949471, https://openalex.org/W3088724481, https://openalex.org/W1486317198, https://openalex.org/W2085040216, https://openalex.org/W2108630796, https://openalex.org/W2963870721, https://openalex.org/W4297162925, https://openalex.org/W2051532229, https://openalex.org/W2802558476, https://openalex.org/W4242635839, https://openalex.org/W2990138404, https://openalex.org/W4385453288, https://openalex.org/W3134774296, https://openalex.org/W3100945072, https://openalex.org/W2604505959 |
| referenced_works_count | 35 |
| abstract_inverted_index.a | 2, 61, 65, 131 |
| abstract_inverted_index.In | 68 |
| abstract_inverted_index.We | 75 |
| abstract_inverted_index.an | 32 |
| abstract_inverted_index.as | 44 |
| abstract_inverted_index.be | 16, 29, 37, 147 |
| abstract_inverted_index.by | 31 |
| abstract_inverted_index.in | 22, 139 |
| abstract_inverted_index.is | 1, 128 |
| abstract_inverted_index.of | 4, 13, 98, 125, 136 |
| abstract_inverted_index.on | 142 |
| abstract_inverted_index.or | 24 |
| abstract_inverted_index.to | 18, 39, 55, 129, 133 |
| abstract_inverted_index.we | 71, 104, 113, 116 |
| abstract_inverted_index.The | 11, 123 |
| abstract_inverted_index.and | 47, 64, 83, 95, 115, 120 |
| abstract_inverted_index.are | 52 |
| abstract_inverted_index.but | 54 |
| abstract_inverted_index.can | 15, 36, 146 |
| abstract_inverted_index.for | 80, 85 |
| abstract_inverted_index.has | 59 |
| abstract_inverted_index.key | 17 |
| abstract_inverted_index.our | 99 |
| abstract_inverted_index.the | 57, 77, 106, 109, 134 |
| abstract_inverted_index.two | 102 |
| abstract_inverted_index.Many | 49 |
| abstract_inverted_index.also | 90 |
| abstract_inverted_index.goal | 124 |
| abstract_inverted_index.own. | 100 |
| abstract_inverted_index.some | 92 |
| abstract_inverted_index.such | 43 |
| abstract_inverted_index.that | 112 |
| abstract_inverted_index.this | 69, 126 |
| abstract_inverted_index.used | 38 |
| abstract_inverted_index.user | 20 |
| abstract_inverted_index.work | 127 |
| abstract_inverted_index.Using | 101 |
| abstract_inverted_index.date, | 56 |
| abstract_inverted_index.every | 5 |
| abstract_inverted_index.field | 135 |
| abstract_inverted_index.tasks | 42 |
| abstract_inverted_index.their | 118 |
| abstract_inverted_index.these | 73 |
| abstract_inverted_index.trust | 21 |
| abstract_inverted_index.using | 108 |
| abstract_inverted_index.which | 26, 143 |
| abstract_inverted_index.while | 89 |
| abstract_inverted_index.active | 45 |
| abstract_inverted_index.built. | 148 |
| abstract_inverted_index.lacked | 60 |
| abstract_inverted_index.needs. | 74 |
| abstract_inverted_index.review | 76 |
| abstract_inverted_index.should | 28 |
| abstract_inverted_index.survey | 63 |
| abstract_inverted_index.compare | 105 |
| abstract_inverted_index.discuss | 117 |
| abstract_inverted_index.fulfill | 72 |
| abstract_inverted_index.further | 144 |
| abstract_inverted_index.issues. | 122 |
| abstract_inverted_index.methods | 79, 94, 107 |
| abstract_inverted_index.metrics | 84, 97, 111 |
| abstract_inverted_index.provide | 130 |
| abstract_inverted_index.article, | 70 |
| abstract_inverted_index.choosing | 25 |
| abstract_inverted_index.detailed | 66 |
| abstract_inverted_index.existing | 78 |
| abstract_inverted_index.learning | 46 |
| abstract_inverted_index.research | 145 |
| abstract_inverted_index.systems, | 141 |
| abstract_inverted_index.systems. | 10 |
| abstract_inverted_index.datasets, | 103 |
| abstract_inverted_index.describe, | 114 |
| abstract_inverted_index.estimates | 35 |
| abstract_inverted_index.including | 8 |
| abstract_inverted_index.potential | 121 |
| abstract_inverted_index.proposing | 91 |
| abstract_inverted_index.strengths | 119 |
| abstract_inverted_index.accomplish | 40 |
| abstract_inverted_index.available, | 53 |
| abstract_inverted_index.estimates, | 88 |
| abstract_inverted_index.estimation | 82, 93, 138 |
| abstract_inverted_index.estimators | 51 |
| abstract_inverted_index.evaluating | 86 |
| abstract_inverted_index.evaluation | 96, 110 |
| abstract_inverted_index.foundation | 132 |
| abstract_inverted_index.increasing | 19 |
| abstract_inverted_index.literature | 58 |
| abstract_inverted_index.Uncertainty | 0 |
| abstract_inverted_index.accompanied | 30 |
| abstract_inverted_index.comparison. | 67 |
| abstract_inverted_index.data-driven | 6 |
| abstract_inverted_index.recommender | 9, 41, 140 |
| abstract_inverted_index.uncertainty | 14, 34, 50, 81, 87, 137 |
| abstract_inverted_index.application, | 7 |
| abstract_inverted_index.co-training. | 48 |
| abstract_inverted_index.explanation; | 33 |
| abstract_inverted_index.comprehensive | 62 |
| abstract_inverted_index.characteristic | 3 |
| abstract_inverted_index.quantification | 12 |
| abstract_inverted_index.recommendations | 23, 27 |
| cited_by_percentile_year.max | 99 |
| cited_by_percentile_year.min | 94 |
| countries_distinct_count | 1 |
| institutions_distinct_count | 2 |
| citation_normalized_percentile.value | 0.966604 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | True |