Opinion Marks : A Human-Based Computation Approach to Instill Structure into Unstructured Text on the Web Article Swipe
Bum Chul Kwon
,
Jaegul Choo
,
Sung-Hee Kim
,
Daniel A. Keim
,
Haesun Park
,
Ji Soo Yi
·
YOU?
·
· 2015
· Open Access
·
YOU?
·
· 2015
· Open Access
·
Despite recent improvements in computational approaches such asmachine learning, natural language processing, and computationallinguistics, making a computer understand human-generated unstructuredtext still remains a difficult problem to solve. To alleviatethe challenges, we propose an approach called “Opinion Marks,”which enables writers to mark positive and negative aspects of atopic on their own text. In addition, Opinion Marks incorporates anautomatic marking suggestion algorithm to offload a user’s markingeffort. The phrases marked with Opinion Marks can be furtherused to clarify the sentiments of other text in a similar context.We implemented Opinion Marks at a question answering websitehttp://caniask.net. To test the efficacy of Opinion Marks, weconducted a crowdsourced experiment with 144 participants in abetween-subject design under three different conditions: 1) humanmarking only; 2) machine marking only (automatic marking suggestion);and 3) human-machine collaboration (Opinion Marks).This study revealed that Opinion Marks significantly improves thequality of marked phrases and usability of the system.
Related Topics
Concepts
Computer science
Context (archaeology)
Sentiment analysis
Computational linguistics
Natural language processing
Artificial intelligence
Subject (documents)
Quality (philosophy)
Usability
Information retrieval
Question answering
World Wide Web
Human–computer interaction
Epistemology
Philosophy
Paleontology
Biology
Metadata
- Type
- article
- Language
- en
- Landing Page
- http://nbn-resolving.de/urn:nbn:de:bsz:352-0-301049
- http://nbn-resolving.de/urn:nbn:de:bsz:352-0-301049
- OA Status
- green
- Cited By
- 1
- References
- 34
- Related Works
- 20
- OpenAlex ID
- https://openalex.org/W2236337979
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W2236337979Canonical identifier for this work in OpenAlex
- Title
-
Opinion Marks : A Human-Based Computation Approach to Instill Structure into Unstructured Text on the WebWork title
- Type
-
articleOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2015Year of publication
- Publication date
-
2015-01-01Full publication date if available
- Authors
-
Bum Chul Kwon, Jaegul Choo, Sung-Hee Kim, Daniel A. Keim, Haesun Park, Ji Soo YiList of authors in order
- Landing page
-
https://nbn-resolving.de/urn:nbn:de:bsz:352-0-301049Publisher landing page
- PDF URL
-
https://nbn-resolving.de/urn:nbn:de:bsz:352-0-301049Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://nbn-resolving.de/urn:nbn:de:bsz:352-0-301049Direct OA link when available
- Concepts
-
Computer science, Context (archaeology), Sentiment analysis, Computational linguistics, Natural language processing, Artificial intelligence, Subject (documents), Quality (philosophy), Usability, Information retrieval, Question answering, World Wide Web, Human–computer interaction, Epistemology, Philosophy, Paleontology, BiologyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
1Total citation count in OpenAlex
- Citations by year (recent)
-
2016: 1Per-year citation counts (last 5 years)
- References (count)
-
34Number of works referenced by this work
- Related works (count)
-
20Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W2236337979 |
|---|---|
| doi | |
| ids.mag | 2236337979 |
| ids.openalex | https://openalex.org/W2236337979 |
| fwci | 0.31430091 |
| type | article |
| title | Opinion Marks : A Human-Based Computation Approach to Instill Structure into Unstructured Text on the Web |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | 55 |
| biblio.first_page | 47 |
| topics[0].id | https://openalex.org/T10028 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9983999729156494 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Topic Modeling |
| topics[1].id | https://openalex.org/T13083 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9976000189781189 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Advanced Text Analysis Techniques |
| topics[2].id | https://openalex.org/T10260 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9973999857902527 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1710 |
| topics[2].subfield.display_name | Information Systems |
| topics[2].display_name | Software Engineering Research |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.794417679309845 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C2779343474 |
| concepts[1].level | 2 |
| concepts[1].score | 0.6194586753845215 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q3109175 |
| concepts[1].display_name | Context (archaeology) |
| concepts[2].id | https://openalex.org/C66402592 |
| concepts[2].level | 2 |
| concepts[2].score | 0.5506373047828674 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q2271421 |
| concepts[2].display_name | Sentiment analysis |
| concepts[3].id | https://openalex.org/C155092808 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5415828824043274 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q182557 |
| concepts[3].display_name | Computational linguistics |
| concepts[4].id | https://openalex.org/C204321447 |
| concepts[4].level | 1 |
| concepts[4].score | 0.5117659568786621 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[4].display_name | Natural language processing |
| concepts[5].id | https://openalex.org/C154945302 |
| concepts[5].level | 1 |
| concepts[5].score | 0.5110658407211304 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[5].display_name | Artificial intelligence |
| concepts[6].id | https://openalex.org/C2777855551 |
| concepts[6].level | 2 |
| concepts[6].score | 0.4958823621273041 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q12310021 |
| concepts[6].display_name | Subject (documents) |
| concepts[7].id | https://openalex.org/C2779530757 |
| concepts[7].level | 2 |
| concepts[7].score | 0.47469040751457214 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q1207505 |
| concepts[7].display_name | Quality (philosophy) |
| concepts[8].id | https://openalex.org/C170130773 |
| concepts[8].level | 2 |
| concepts[8].score | 0.4670397639274597 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q216378 |
| concepts[8].display_name | Usability |
| concepts[9].id | https://openalex.org/C23123220 |
| concepts[9].level | 1 |
| concepts[9].score | 0.4307267367839813 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q816826 |
| concepts[9].display_name | Information retrieval |
| concepts[10].id | https://openalex.org/C44291984 |
| concepts[10].level | 2 |
| concepts[10].score | 0.42175719141960144 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q1074173 |
| concepts[10].display_name | Question answering |
| concepts[11].id | https://openalex.org/C136764020 |
| concepts[11].level | 1 |
| concepts[11].score | 0.4005570113658905 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q466 |
| concepts[11].display_name | World Wide Web |
| concepts[12].id | https://openalex.org/C107457646 |
| concepts[12].level | 1 |
| concepts[12].score | 0.20187869668006897 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q207434 |
| concepts[12].display_name | Human–computer interaction |
| concepts[13].id | https://openalex.org/C111472728 |
| concepts[13].level | 1 |
| concepts[13].score | 0.0 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q9471 |
| concepts[13].display_name | Epistemology |
| concepts[14].id | https://openalex.org/C138885662 |
| concepts[14].level | 0 |
| concepts[14].score | 0.0 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q5891 |
| concepts[14].display_name | Philosophy |
| concepts[15].id | https://openalex.org/C151730666 |
| concepts[15].level | 1 |
| concepts[15].score | 0.0 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q7205 |
| concepts[15].display_name | Paleontology |
| concepts[16].id | https://openalex.org/C86803240 |
| concepts[16].level | 0 |
| concepts[16].score | 0.0 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q420 |
| concepts[16].display_name | Biology |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.794417679309845 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/context |
| keywords[1].score | 0.6194586753845215 |
| keywords[1].display_name | Context (archaeology) |
| keywords[2].id | https://openalex.org/keywords/sentiment-analysis |
| keywords[2].score | 0.5506373047828674 |
| keywords[2].display_name | Sentiment analysis |
| keywords[3].id | https://openalex.org/keywords/computational-linguistics |
| keywords[3].score | 0.5415828824043274 |
| keywords[3].display_name | Computational linguistics |
| keywords[4].id | https://openalex.org/keywords/natural-language-processing |
| keywords[4].score | 0.5117659568786621 |
| keywords[4].display_name | Natural language processing |
| keywords[5].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[5].score | 0.5110658407211304 |
| keywords[5].display_name | Artificial intelligence |
| keywords[6].id | https://openalex.org/keywords/subject |
| keywords[6].score | 0.4958823621273041 |
| keywords[6].display_name | Subject (documents) |
| keywords[7].id | https://openalex.org/keywords/quality |
| keywords[7].score | 0.47469040751457214 |
| keywords[7].display_name | Quality (philosophy) |
| keywords[8].id | https://openalex.org/keywords/usability |
| keywords[8].score | 0.4670397639274597 |
| keywords[8].display_name | Usability |
| keywords[9].id | https://openalex.org/keywords/information-retrieval |
| keywords[9].score | 0.4307267367839813 |
| keywords[9].display_name | Information retrieval |
| keywords[10].id | https://openalex.org/keywords/question-answering |
| keywords[10].score | 0.42175719141960144 |
| keywords[10].display_name | Question answering |
| keywords[11].id | https://openalex.org/keywords/world-wide-web |
| keywords[11].score | 0.4005570113658905 |
| keywords[11].display_name | World Wide Web |
| keywords[12].id | https://openalex.org/keywords/human–computer-interaction |
| keywords[12].score | 0.20187869668006897 |
| keywords[12].display_name | Human–computer interaction |
| language | en |
| locations[0].id | pmh:oai:kops.uni-konstanz.de:123456789/32597 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306401487 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | False |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | KOPS (University of Konstanz) |
| locations[0].source.host_organization | https://openalex.org/I189712700 |
| locations[0].source.host_organization_name | University of Konstanz |
| locations[0].source.host_organization_lineage | https://openalex.org/I189712700 |
| locations[0].license | |
| locations[0].pdf_url | http://nbn-resolving.de/urn:nbn:de:bsz:352-0-301049 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | doc-type:conferenceObject |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | Proceedings of the ACM SIGKDD Workshop on Interactive Data Exploration and Analytics Permission / Polo Chau et al. (Hrsg.). - New York : ACM, 2015. - S. 47-55 |
| locations[0].landing_page_url | http://nbn-resolving.de/urn:nbn:de:bsz:352-0-301049 |
| locations[1].id | mag:2236337979 |
| locations[1].is_oa | False |
| locations[1].source | |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | http://poloclub.gatech.edu/idea2015/papers/p47-kwon.pdf |
| authorships[0].author.id | https://openalex.org/A5064480048 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-9391-6274 |
| authorships[0].author.display_name | Bum Chul Kwon |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Bum Chul Kwon |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5047912015 |
| authorships[1].author.orcid | https://orcid.org/0000-0003-1071-4835 |
| authorships[1].author.display_name | Jaegul Choo |
| authorships[1].affiliations[0].raw_affiliation_string | Korea University |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Jaegul Choo |
| authorships[1].is_corresponding | False |
| authorships[1].raw_affiliation_strings | Korea University |
| authorships[2].author.id | https://openalex.org/A5100654454 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-9716-8349 |
| authorships[2].author.display_name | Sung-Hee Kim |
| authorships[2].countries | US |
| authorships[2].affiliations[0].institution_ids | https://openalex.org/I219193219 |
| authorships[2].affiliations[0].raw_affiliation_string | PURDUE UNIVERSITY. |
| authorships[2].institutions[0].id | https://openalex.org/I219193219 |
| authorships[2].institutions[0].ror | https://ror.org/02dqehb95 |
| authorships[2].institutions[0].type | education |
| authorships[2].institutions[0].lineage | https://openalex.org/I219193219 |
| authorships[2].institutions[0].country_code | US |
| authorships[2].institutions[0].display_name | Purdue University West Lafayette |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Sung-Hee Kim |
| authorships[2].is_corresponding | False |
| authorships[2].raw_affiliation_strings | PURDUE UNIVERSITY. |
| authorships[3].author.id | https://openalex.org/A5073919282 |
| authorships[3].author.orcid | https://orcid.org/0000-0001-7966-9740 |
| authorships[3].author.display_name | Daniel A. Keim |
| authorships[3].countries | DE |
| authorships[3].affiliations[0].institution_ids | https://openalex.org/I189712700 |
| authorships[3].affiliations[0].raw_affiliation_string | University of Konstanz. |
| authorships[3].institutions[0].id | https://openalex.org/I189712700 |
| authorships[3].institutions[0].ror | https://ror.org/0546hnb39 |
| authorships[3].institutions[0].type | education |
| authorships[3].institutions[0].lineage | https://openalex.org/I189712700 |
| authorships[3].institutions[0].country_code | DE |
| authorships[3].institutions[0].display_name | University of Konstanz |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Daniel Keim |
| authorships[3].is_corresponding | False |
| authorships[3].raw_affiliation_strings | University of Konstanz. |
| authorships[4].author.id | https://openalex.org/A5101728710 |
| authorships[4].author.orcid | https://orcid.org/0000-0001-6259-7170 |
| authorships[4].author.display_name | Haesun Park |
| authorships[4].countries | US |
| authorships[4].affiliations[0].institution_ids | https://openalex.org/I130701444 |
| authorships[4].affiliations[0].raw_affiliation_string | #N#‡#N#Georgia Institute of Technology#N# |
| authorships[4].institutions[0].id | https://openalex.org/I130701444 |
| authorships[4].institutions[0].ror | https://ror.org/01zkghx44 |
| authorships[4].institutions[0].type | education |
| authorships[4].institutions[0].lineage | https://openalex.org/I130701444 |
| authorships[4].institutions[0].country_code | US |
| authorships[4].institutions[0].display_name | Georgia Institute of Technology |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Haesun Park |
| authorships[4].is_corresponding | False |
| authorships[4].raw_affiliation_strings | #N#‡#N#Georgia Institute of Technology#N# |
| authorships[5].author.id | https://openalex.org/A5053349373 |
| authorships[5].author.orcid | https://orcid.org/0000-0002-7545-3878 |
| authorships[5].author.display_name | Ji Soo Yi |
| authorships[5].countries | US |
| authorships[5].affiliations[0].institution_ids | https://openalex.org/I219193219 |
| authorships[5].affiliations[0].raw_affiliation_string | PURDUE UNIVERSITY. |
| authorships[5].institutions[0].id | https://openalex.org/I219193219 |
| authorships[5].institutions[0].ror | https://ror.org/02dqehb95 |
| authorships[5].institutions[0].type | education |
| authorships[5].institutions[0].lineage | https://openalex.org/I219193219 |
| authorships[5].institutions[0].country_code | US |
| authorships[5].institutions[0].display_name | Purdue University West Lafayette |
| authorships[5].author_position | last |
| authorships[5].raw_author_name | Ji Soo Yi |
| authorships[5].is_corresponding | False |
| authorships[5].raw_affiliation_strings | PURDUE UNIVERSITY. |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | http://nbn-resolving.de/urn:nbn:de:bsz:352-0-301049 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Opinion Marks : A Human-Based Computation Approach to Instill Structure into Unstructured Text on the Web |
| has_fulltext | True |
| is_retracted | False |
| updated_date | 2025-11-06T04:12:42.849631 |
| primary_topic.id | https://openalex.org/T10028 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9983999729156494 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Topic Modeling |
| related_works | https://openalex.org/W2226920895, https://openalex.org/W2114251074, https://openalex.org/W3102354094, https://openalex.org/W3100195825, https://openalex.org/W2949393284, https://openalex.org/W3155626385, https://openalex.org/W2964786431, https://openalex.org/W2111008930, https://openalex.org/W2970507142, https://openalex.org/W2133264657, https://openalex.org/W2186335449, https://openalex.org/W3014534958, https://openalex.org/W2183883706, https://openalex.org/W2560750187, https://openalex.org/W2798745006, https://openalex.org/W2941907145, https://openalex.org/W2266945365, https://openalex.org/W1702721705, https://openalex.org/W3201938637, https://openalex.org/W2181664276 |
| cited_by_count | 1 |
| counts_by_year[0].year | 2016 |
| counts_by_year[0].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:kops.uni-konstanz.de:123456789/32597 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306401487 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | False |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | KOPS (University of Konstanz) |
| best_oa_location.source.host_organization | https://openalex.org/I189712700 |
| best_oa_location.source.host_organization_name | University of Konstanz |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I189712700 |
| best_oa_location.license | |
| best_oa_location.pdf_url | http://nbn-resolving.de/urn:nbn:de:bsz:352-0-301049 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | doc-type:conferenceObject |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | Proceedings of the ACM SIGKDD Workshop on Interactive Data Exploration and Analytics Permission / Polo Chau et al. (Hrsg.). - New York : ACM, 2015. - S. 47-55 |
| best_oa_location.landing_page_url | http://nbn-resolving.de/urn:nbn:de:bsz:352-0-301049 |
| primary_location.id | pmh:oai:kops.uni-konstanz.de:123456789/32597 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306401487 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | False |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | KOPS (University of Konstanz) |
| primary_location.source.host_organization | https://openalex.org/I189712700 |
| primary_location.source.host_organization_name | University of Konstanz |
| primary_location.source.host_organization_lineage | https://openalex.org/I189712700 |
| primary_location.license | |
| primary_location.pdf_url | http://nbn-resolving.de/urn:nbn:de:bsz:352-0-301049 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | doc-type:conferenceObject |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | Proceedings of the ACM SIGKDD Workshop on Interactive Data Exploration and Analytics Permission / Polo Chau et al. (Hrsg.). - New York : ACM, 2015. - S. 47-55 |
| primary_location.landing_page_url | http://nbn-resolving.de/urn:nbn:de:bsz:352-0-301049 |
| publication_date | 2015-01-01 |
| publication_year | 2015 |
| referenced_works | https://openalex.org/W2141631351, https://openalex.org/W2141282920, https://openalex.org/W2160660844, https://openalex.org/W2054741990, https://openalex.org/W2251939518, https://openalex.org/W1964613733, https://openalex.org/W2169606435, https://openalex.org/W2034721576, https://openalex.org/W2153207410, https://openalex.org/W50799546, https://openalex.org/W2128996912, https://openalex.org/W2027323723, https://openalex.org/W2101196063, https://openalex.org/W2049493498, https://openalex.org/W2110764733, https://openalex.org/W2090491854, https://openalex.org/W2022204871, https://openalex.org/W2295555357, https://openalex.org/W1986725403, https://openalex.org/W2163659824, https://openalex.org/W2150071643, https://openalex.org/W2149310982, https://openalex.org/W2087382273, https://openalex.org/W2119281377, https://openalex.org/W2160052288, https://openalex.org/W2132166724, https://openalex.org/W2097726431, https://openalex.org/W2034351017, https://openalex.org/W2251426493, https://openalex.org/W2136518070, https://openalex.org/W66373487, https://openalex.org/W2128272588, https://openalex.org/W2913879771, https://openalex.org/W2105567784 |
| referenced_works_count | 34 |
| abstract_inverted_index.a | 17, 25, 69, 91, 99, 113 |
| abstract_inverted_index.1) | 127 |
| abstract_inverted_index.2) | 131 |
| abstract_inverted_index.3) | 139 |
| abstract_inverted_index.In | 57 |
| abstract_inverted_index.To | 30, 104 |
| abstract_inverted_index.an | 36 |
| abstract_inverted_index.at | 98 |
| abstract_inverted_index.be | 80 |
| abstract_inverted_index.in | 3, 90, 119 |
| abstract_inverted_index.of | 50, 87, 108, 154, 159 |
| abstract_inverted_index.on | 53 |
| abstract_inverted_index.to | 28, 44, 67, 83 |
| abstract_inverted_index.we | 34 |
| abstract_inverted_index.144 | 117 |
| abstract_inverted_index.The | 73 |
| abstract_inverted_index.and | 13, 47, 157 |
| abstract_inverted_index.can | 79 |
| abstract_inverted_index.own | 55 |
| abstract_inverted_index.the | 85, 106, 160 |
| abstract_inverted_index.mark | 45 |
| abstract_inverted_index.only | 134 |
| abstract_inverted_index.such | 6 |
| abstract_inverted_index.test | 105 |
| abstract_inverted_index.text | 89 |
| abstract_inverted_index.that | 147 |
| abstract_inverted_index.with | 76, 116 |
| abstract_inverted_index.Marks | 60, 78, 97, 149 |
| abstract_inverted_index.only; | 130 |
| abstract_inverted_index.other | 88 |
| abstract_inverted_index.still | 23 |
| abstract_inverted_index.study | 145 |
| abstract_inverted_index.text. | 56 |
| abstract_inverted_index.their | 54 |
| abstract_inverted_index.three | 124 |
| abstract_inverted_index.under | 123 |
| abstract_inverted_index.Marks, | 110 |
| abstract_inverted_index.called | 38 |
| abstract_inverted_index.design | 122 |
| abstract_inverted_index.making | 16 |
| abstract_inverted_index.marked | 75, 155 |
| abstract_inverted_index.recent | 1 |
| abstract_inverted_index.solve. | 29 |
| abstract_inverted_index./>We | 94 |
| abstract_inverted_index.Despite | 0 |
| abstract_inverted_index.Opinion | 59, 77, 96, 109, 148 |
| abstract_inverted_index.a<br | 51, 120 |
| abstract_inverted_index.aspects | 49 |
| abstract_inverted_index.clarify | 84 |
| abstract_inverted_index.enables | 42 |
| abstract_inverted_index.machine | 132 |
| abstract_inverted_index.marking | 64, 133, 136 |
| abstract_inverted_index.natural | 10 |
| abstract_inverted_index.offload | 68 |
| abstract_inverted_index.phrases | 74, 156 |
| abstract_inverted_index.problem | 27 |
| abstract_inverted_index.propose | 35 |
| abstract_inverted_index.remains | 24 |
| abstract_inverted_index.similar | 92 |
| abstract_inverted_index.system. | 161 |
| abstract_inverted_index.writers | 43 |
| abstract_inverted_index.(Opinion | 142 |
| abstract_inverted_index./>and | 138 |
| abstract_inverted_index./>the | 32 |
| abstract_inverted_index.an<br | 62 |
| abstract_inverted_index.approach | 37 |
| abstract_inverted_index.as<br | 7 |
| abstract_inverted_index.computer | 18 |
| abstract_inverted_index.efficacy | 107 |
| abstract_inverted_index.improves | 151 |
| abstract_inverted_index.language | 11 |
| abstract_inverted_index.negative | 48 |
| abstract_inverted_index.positive | 46 |
| abstract_inverted_index.question | 100 |
| abstract_inverted_index.revealed | 146 |
| abstract_inverted_index.user’s | 70 |
| abstract_inverted_index.we<br | 111 |
| abstract_inverted_index./>This | 144 |
| abstract_inverted_index./>text | 22 |
| abstract_inverted_index./>used | 82 |
| abstract_inverted_index.addition, | 58 |
| abstract_inverted_index.algorithm | 66 |
| abstract_inverted_index.answering | 101 |
| abstract_inverted_index.different | 125 |
| abstract_inverted_index.difficult | 26 |
| abstract_inverted_index.learning, | 9 |
| abstract_inverted_index.the<br | 152 |
| abstract_inverted_index.usability | 158 |
| abstract_inverted_index.(automatic | 135 |
| abstract_inverted_index./>topic | 52 |
| abstract_inverted_index./>which | 41 |
| abstract_inverted_index.approaches | 5 |
| abstract_inverted_index.experiment | 115 |
| abstract_inverted_index.sentiments | 86 |
| abstract_inverted_index.suggestion | 65 |
| abstract_inverted_index.understand | 19 |
| abstract_inverted_index.“Opinion | 39 |
| abstract_inverted_index.challenges, | 33 |
| abstract_inverted_index.conditions: | 126 |
| abstract_inverted_index.human<br | 128 |
| abstract_inverted_index.implemented | 95 |
| abstract_inverted_index.processing, | 12 |
| abstract_inverted_index./>effort. | 72 |
| abstract_inverted_index./>machine | 8 |
| abstract_inverted_index./>marking | 129 |
| abstract_inverted_index./>quality | 153 |
| abstract_inverted_index.crowdsourced | 114 |
| abstract_inverted_index.improvements | 2 |
| abstract_inverted_index.incorporates | 61 |
| abstract_inverted_index.participants | 118 |
| abstract_inverted_index.Marks).<br | 143 |
| abstract_inverted_index.collaboration | 141 |
| abstract_inverted_index.computational | 4 |
| abstract_inverted_index.further<br | 81 |
| abstract_inverted_index.human-machine | 140 |
| abstract_inverted_index.marking<br | 71 |
| abstract_inverted_index.significantly | 150 |
| abstract_inverted_index.website<br | 102 |
| abstract_inverted_index./>automatic | 63 |
| abstract_inverted_index./>conducted | 112 |
| abstract_inverted_index.context.<br | 93 |
| abstract_inverted_index.Marks,”<br | 40 |
| abstract_inverted_index.alleviate<br | 31 |
| abstract_inverted_index.human-generated | 20 |
| abstract_inverted_index./>linguistics, | 15 |
| abstract_inverted_index.suggestion);<br | 137 |
| abstract_inverted_index.unstructured<br | 21 |
| abstract_inverted_index.computational<br | 14 |
| abstract_inverted_index./>between-subject | 121 |
| abstract_inverted_index./>http://caniask.net. | 103 |
| cited_by_percentile_year.max | 94 |
| cited_by_percentile_year.min | 90 |
| countries_distinct_count | 2 |
| institutions_distinct_count | 6 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/4 |
| sustainable_development_goals[0].score | 0.6000000238418579 |
| sustainable_development_goals[0].display_name | Quality Education |
| citation_normalized_percentile.value | 0.80471702 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | False |