MHS-STMA: Multimodal Hate Speech Detection via Scalable Transformer-Based Multilevel Attention Framework Article Swipe
Anusha Chhabra
,
Dinesh Kumar Vishwakarma
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2409.05136
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2409.05136
Social media has a significant impact on people's lives. Hate speech on social media has emerged as one of society's most serious issues in recent years. Text and pictures are two forms of multimodal data that are distributed within articles. Unimodal analysis has been the primary emphasis of earlier approaches. Additionally, when doing multimodal analysis, researchers neglect to preserve the distinctive qualities associated with each modality. To address these shortcomings, the present article suggests a scalable architecture for multimodal hate content detection called transformer-based multilevel attention (STMA). This architecture consists of three main parts: a combined attention-based deep learning mechanism, a vision attention-mechanism encoder, and a caption attention-mechanism encoder. To identify hate content, each component uses various attention processes and handles multimodal data in a unique way. Several studies employing multiple assessment criteria on three hate speech datasets such as Hateful memes, MultiOff, and MMHS150K, validate the suggested architecture's efficacy. The outcomes demonstrate that on all three datasets, the suggested strategy performs better than the baseline approaches.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2409.05136
- https://arxiv.org/pdf/2409.05136
- OA Status
- green
- Cited By
- 1
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4403703688
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4403703688Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2409.05136Digital Object Identifier
- Title
-
MHS-STMA: Multimodal Hate Speech Detection via Scalable Transformer-Based Multilevel Attention FrameworkWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-09-08Full publication date if available
- Authors
-
Anusha Chhabra, Dinesh Kumar VishwakarmaList of authors in order
- Landing page
-
https://arxiv.org/abs/2409.05136Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2409.05136Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2409.05136Direct OA link when available
- Concepts
-
Transformer, Computer science, Speech recognition, Scalability, Voice activity detection, Speech processing, Engineering, Electrical engineering, Voltage, DatabaseTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
1Total citation count in OpenAlex
- Citations by year (recent)
-
2025: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4403703688 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2409.05136 |
| ids.doi | https://doi.org/10.48550/arxiv.2409.05136 |
| ids.openalex | https://openalex.org/W4403703688 |
| fwci | |
| type | preprint |
| title | MHS-STMA: Multimodal Hate Speech Detection via Scalable Transformer-Based Multilevel Attention Framework |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T12262 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.972000002861023 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Hate Speech and Cyberbullying Detection |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C66322947 |
| concepts[0].level | 3 |
| concepts[0].score | 0.6054052114486694 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q11658 |
| concepts[0].display_name | Transformer |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.5606614351272583 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C28490314 |
| concepts[2].level | 1 |
| concepts[2].score | 0.54084312915802 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q189436 |
| concepts[2].display_name | Speech recognition |
| concepts[3].id | https://openalex.org/C48044578 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5019192695617676 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q727490 |
| concepts[3].display_name | Scalability |
| concepts[4].id | https://openalex.org/C204201278 |
| concepts[4].level | 3 |
| concepts[4].score | 0.42925459146499634 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q1332614 |
| concepts[4].display_name | Voice activity detection |
| concepts[5].id | https://openalex.org/C61328038 |
| concepts[5].level | 2 |
| concepts[5].score | 0.1629713475704193 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q3358061 |
| concepts[5].display_name | Speech processing |
| concepts[6].id | https://openalex.org/C127413603 |
| concepts[6].level | 0 |
| concepts[6].score | 0.15807759761810303 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q11023 |
| concepts[6].display_name | Engineering |
| concepts[7].id | https://openalex.org/C119599485 |
| concepts[7].level | 1 |
| concepts[7].score | 0.08571666479110718 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q43035 |
| concepts[7].display_name | Electrical engineering |
| concepts[8].id | https://openalex.org/C165801399 |
| concepts[8].level | 2 |
| concepts[8].score | 0.04469412565231323 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q25428 |
| concepts[8].display_name | Voltage |
| concepts[9].id | https://openalex.org/C77088390 |
| concepts[9].level | 1 |
| concepts[9].score | 0.0 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q8513 |
| concepts[9].display_name | Database |
| keywords[0].id | https://openalex.org/keywords/transformer |
| keywords[0].score | 0.6054052114486694 |
| keywords[0].display_name | Transformer |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.5606614351272583 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/speech-recognition |
| keywords[2].score | 0.54084312915802 |
| keywords[2].display_name | Speech recognition |
| keywords[3].id | https://openalex.org/keywords/scalability |
| keywords[3].score | 0.5019192695617676 |
| keywords[3].display_name | Scalability |
| keywords[4].id | https://openalex.org/keywords/voice-activity-detection |
| keywords[4].score | 0.42925459146499634 |
| keywords[4].display_name | Voice activity detection |
| keywords[5].id | https://openalex.org/keywords/speech-processing |
| keywords[5].score | 0.1629713475704193 |
| keywords[5].display_name | Speech processing |
| keywords[6].id | https://openalex.org/keywords/engineering |
| keywords[6].score | 0.15807759761810303 |
| keywords[6].display_name | Engineering |
| keywords[7].id | https://openalex.org/keywords/electrical-engineering |
| keywords[7].score | 0.08571666479110718 |
| keywords[7].display_name | Electrical engineering |
| keywords[8].id | https://openalex.org/keywords/voltage |
| keywords[8].score | 0.04469412565231323 |
| keywords[8].display_name | Voltage |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2409.05136 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2409.05136 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2409.05136 |
| locations[1].id | doi:10.48550/arxiv.2409.05136 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2409.05136 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5083748222 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Anusha Chhabra |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Chhabra, Anusha |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5021449557 |
| authorships[1].author.orcid | https://orcid.org/0000-0002-1026-0047 |
| authorships[1].author.display_name | Dinesh Kumar Vishwakarma |
| authorships[1].author_position | last |
| authorships[1].raw_author_name | Vishwakarma, Dinesh Kumar |
| authorships[1].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2409.05136 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | MHS-STMA: Multimodal Hate Speech Detection via Scalable Transformer-Based Multilevel Attention Framework |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T12262 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.972000002861023 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Hate Speech and Cyberbullying Detection |
| related_works | https://openalex.org/W191108438, https://openalex.org/W3135230428, https://openalex.org/W2904739811, https://openalex.org/W249088392, https://openalex.org/W2152158029, https://openalex.org/W2012540220, https://openalex.org/W2131711534, https://openalex.org/W2559837139, https://openalex.org/W1151175420, https://openalex.org/W2407342067 |
| cited_by_count | 1 |
| counts_by_year[0].year | 2025 |
| counts_by_year[0].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2409.05136 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2409.05136 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2409.05136 |
| primary_location.id | pmh:oai:arXiv.org:2409.05136 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2409.05136 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2409.05136 |
| publication_date | 2024-09-08 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 3, 74, 94, 100, 105, 124 |
| abstract_inverted_index.To | 66, 109 |
| abstract_inverted_index.as | 16, 139 |
| abstract_inverted_index.in | 23, 123 |
| abstract_inverted_index.of | 18, 32, 47, 90 |
| abstract_inverted_index.on | 6, 11, 133, 154 |
| abstract_inverted_index.to | 57 |
| abstract_inverted_index.The | 150 |
| abstract_inverted_index.all | 155 |
| abstract_inverted_index.and | 27, 104, 119, 143 |
| abstract_inverted_index.are | 29, 36 |
| abstract_inverted_index.for | 77 |
| abstract_inverted_index.has | 2, 14, 42 |
| abstract_inverted_index.one | 17 |
| abstract_inverted_index.the | 44, 59, 70, 146, 158, 164 |
| abstract_inverted_index.two | 30 |
| abstract_inverted_index.Hate | 9 |
| abstract_inverted_index.Text | 26 |
| abstract_inverted_index.This | 87 |
| abstract_inverted_index.been | 43 |
| abstract_inverted_index.data | 34, 122 |
| abstract_inverted_index.deep | 97 |
| abstract_inverted_index.each | 64, 113 |
| abstract_inverted_index.hate | 79, 111, 135 |
| abstract_inverted_index.main | 92 |
| abstract_inverted_index.most | 20 |
| abstract_inverted_index.such | 138 |
| abstract_inverted_index.than | 163 |
| abstract_inverted_index.that | 35, 153 |
| abstract_inverted_index.uses | 115 |
| abstract_inverted_index.way. | 126 |
| abstract_inverted_index.when | 51 |
| abstract_inverted_index.with | 63 |
| abstract_inverted_index.doing | 52 |
| abstract_inverted_index.forms | 31 |
| abstract_inverted_index.media | 1, 13 |
| abstract_inverted_index.these | 68 |
| abstract_inverted_index.three | 91, 134, 156 |
| abstract_inverted_index.Social | 0 |
| abstract_inverted_index.better | 162 |
| abstract_inverted_index.called | 82 |
| abstract_inverted_index.impact | 5 |
| abstract_inverted_index.issues | 22 |
| abstract_inverted_index.lives. | 8 |
| abstract_inverted_index.memes, | 141 |
| abstract_inverted_index.parts: | 93 |
| abstract_inverted_index.recent | 24 |
| abstract_inverted_index.social | 12 |
| abstract_inverted_index.speech | 10, 136 |
| abstract_inverted_index.unique | 125 |
| abstract_inverted_index.vision | 101 |
| abstract_inverted_index.within | 38 |
| abstract_inverted_index.years. | 25 |
| abstract_inverted_index.(STMA). | 86 |
| abstract_inverted_index.Hateful | 140 |
| abstract_inverted_index.Several | 127 |
| abstract_inverted_index.address | 67 |
| abstract_inverted_index.article | 72 |
| abstract_inverted_index.caption | 106 |
| abstract_inverted_index.content | 80 |
| abstract_inverted_index.earlier | 48 |
| abstract_inverted_index.emerged | 15 |
| abstract_inverted_index.handles | 120 |
| abstract_inverted_index.neglect | 56 |
| abstract_inverted_index.present | 71 |
| abstract_inverted_index.primary | 45 |
| abstract_inverted_index.serious | 21 |
| abstract_inverted_index.studies | 128 |
| abstract_inverted_index.various | 116 |
| abstract_inverted_index.Unimodal | 40 |
| abstract_inverted_index.analysis | 41 |
| abstract_inverted_index.baseline | 165 |
| abstract_inverted_index.combined | 95 |
| abstract_inverted_index.consists | 89 |
| abstract_inverted_index.content, | 112 |
| abstract_inverted_index.criteria | 132 |
| abstract_inverted_index.datasets | 137 |
| abstract_inverted_index.emphasis | 46 |
| abstract_inverted_index.encoder, | 103 |
| abstract_inverted_index.encoder. | 108 |
| abstract_inverted_index.identify | 110 |
| abstract_inverted_index.learning | 98 |
| abstract_inverted_index.multiple | 130 |
| abstract_inverted_index.outcomes | 151 |
| abstract_inverted_index.people's | 7 |
| abstract_inverted_index.performs | 161 |
| abstract_inverted_index.pictures | 28 |
| abstract_inverted_index.preserve | 58 |
| abstract_inverted_index.scalable | 75 |
| abstract_inverted_index.strategy | 160 |
| abstract_inverted_index.suggests | 73 |
| abstract_inverted_index.validate | 145 |
| abstract_inverted_index.MMHS150K, | 144 |
| abstract_inverted_index.MultiOff, | 142 |
| abstract_inverted_index.analysis, | 54 |
| abstract_inverted_index.articles. | 39 |
| abstract_inverted_index.attention | 85, 117 |
| abstract_inverted_index.component | 114 |
| abstract_inverted_index.datasets, | 157 |
| abstract_inverted_index.detection | 81 |
| abstract_inverted_index.efficacy. | 149 |
| abstract_inverted_index.employing | 129 |
| abstract_inverted_index.modality. | 65 |
| abstract_inverted_index.processes | 118 |
| abstract_inverted_index.qualities | 61 |
| abstract_inverted_index.society's | 19 |
| abstract_inverted_index.suggested | 147, 159 |
| abstract_inverted_index.assessment | 131 |
| abstract_inverted_index.associated | 62 |
| abstract_inverted_index.mechanism, | 99 |
| abstract_inverted_index.multilevel | 84 |
| abstract_inverted_index.multimodal | 33, 53, 78, 121 |
| abstract_inverted_index.approaches. | 49, 166 |
| abstract_inverted_index.demonstrate | 152 |
| abstract_inverted_index.distinctive | 60 |
| abstract_inverted_index.distributed | 37 |
| abstract_inverted_index.researchers | 55 |
| abstract_inverted_index.significant | 4 |
| abstract_inverted_index.architecture | 76, 88 |
| abstract_inverted_index.Additionally, | 50 |
| abstract_inverted_index.shortcomings, | 69 |
| abstract_inverted_index.architecture's | 148 |
| abstract_inverted_index.attention-based | 96 |
| abstract_inverted_index.transformer-based | 83 |
| abstract_inverted_index.attention-mechanism | 102, 107 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 2 |
| citation_normalized_percentile |