Implementing Active Learning in Cybersecurity: Detecting Anomalies in Redacted Emails Article Swipe

MAKE IMPACT

Mu-Huan , Chung , Lu Wang , Sharon Sharon , LI . , Yuhong Yuhong , Yang Yang , Calvin Giang , Khilan Jerath , Abhay Raman , David Lie , Mark Chignell · YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2303.00870

Research on email anomaly detection has typically relied on specially prepared datasets that may not adequately reflect the type of data that occurs in industry settings. In our research, at a major financial services company, privacy concerns prevented inspection of the bodies of emails and attachment details (although subject headings and attachment filenames were available). This made labeling possible anomalies in the resulting redacted emails more difficult. Another source of difficulty is the high volume of emails combined with the scarcity of resources making machine learning (ML) a necessity, but also creating a need for more efficient human training of ML models. Active learning (AL) has been proposed as a way to make human training of ML models more efficient. However, the implementation of Active Learning methods is a human-centered AI challenge due to potential human analyst uncertainty, and the labeling task can be further complicated in domains such as the cybersecurity domain (or healthcare, aviation, etc.) where mistakes in labeling can have highly adverse consequences. In this paper we present research results concerning the application of Active Learning to anomaly detection in redacted emails, comparing the utility of different methods for implementing active learning in this context. We evaluate different AL strategies and their impact on resulting model performance. We also examine how ratings of confidence that experts have in their labels can inform AL. The results obtained are discussed in terms of their implications for AL methodology and for the role of experts in model-assisted email anomaly screening.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2303.00870

PDF

https://arxiv.org/pdf/2303.00870

OA Status

green

Cited By

3

Related Works

10

OpenAlex ID

https://openalex.org/W4323207639

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4323207639

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2303.00870

Digital Object Identifier

Title

Implementing Active Learning in Cybersecurity: Detecting Anomalies in Redacted Emails

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2023

Year of publication

Publication date

2023-03-01

Full publication date if available

Authors

Mu-Huan, Chung, Lu Wang, Sharon Sharon, LI ., Yuhong Yuhong, Yang Yang, Calvin Giang, Khilan Jerath, Abhay Raman, David Lie, Mark Chignell

List of authors in order

Landing page

https://arxiv.org/abs/2303.00870

Publisher landing page

PDF URL

https://arxiv.org/pdf/2303.00870

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2303.00870

Direct OA link when available

Concepts

Computer science, Anomaly detection, Context (archaeology), Task (project management), Subject-matter expert, Artificial intelligence, Domain (mathematical analysis), Active learning (machine learning), Deep learning, Supervised learning, Machine learning, Data science, Engineering, Expert system, Paleontology, Mathematical analysis, Systems engineering, Mathematics, Biology, Artificial neural network

Top concepts (fields/topics) attached by OpenAlex

Cited by

3

Total citation count in OpenAlex

Citations by year (recent)

2024: 2, 2023: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload