Audit-LLM: Multi-Agent Collaboration for Log-based Insider Threat Detection Article Swipe

PDF

Chengyu Song , Linru Ma , Jianming Zheng , Jinzhi Liao , Hongyu Kuang , Lin Yang ·

YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2408.08902

Log-based insider threat detection (ITD) detects malicious user activities by auditing log entries. Recently, large language models (LLMs) with strong common sense knowledge have emerged in the domain of ITD. Nevertheless, diverse activity types and overlong log files pose a significant challenge for LLMs in directly discerning malicious ones within myriads of normal activities. Furthermore, the faithfulness hallucination issue from LLMs aggravates its application difficulty in ITD, as the generated conclusion may not align with user commands and activity context. In response to these challenges, we introduce Audit-LLM, a multi-agent log-based insider threat detection framework comprising three collaborative agents: (i) the Decomposer agent, breaking down the complex ITD task into manageable sub-tasks using Chain-of-Thought (COT) reasoning;(ii) the Tool Builder agent, creating reusable tools for sub-tasks to overcome context length limitations in LLMs; and (iii) the Executor agent, generating the final detection conclusion by invoking constructed tools. To enhance conclusion accuracy, we propose a pair-wise Evidence-based Multi-agent Debate (EMAD) mechanism, where two independent Executors iteratively refine their conclusions through reasoning exchange to reach a consensus. Comprehensive experiments conducted on three publicly available ITD datasets-CERT r4.2, CERT r5.2, and PicoDomain-demonstrate the superiority of our method over existing baselines and show that the proposed EMAD significantly improves the faithfulness of explanations generated by LLMs.

Related Topics

Concepts

Insider threat Audit Insider Business Computer science Accounting Political science Law

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2408.08902
PDF: https://arxiv.org/pdf/2408.08902
OA Status: green
Cited By: 1
Related Works: 10
OpenAlex ID: https://openalex.org/W4402502890

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4402502890

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2408.08902

Digital Object Identifier
Title: Audit-LLM: Multi-Agent Collaboration for Log-based Insider Threat Detection

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2024

Year of publication
Publication date: 2024-08-12

Full publication date if available
Authors: Chengyu Song, Linru Ma, Jianming Zheng, Jinzhi Liao, Hongyu Kuang, Lin Yang

List of authors in order
Landing page: https://arxiv.org/abs/2408.08902

Publisher landing page
PDF URL: https://arxiv.org/pdf/2408.08902

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2408.08902

Direct OA link when available
Concepts: Insider threat, Audit, Insider, Business, Computer science, Accounting, Political science, Law

Top concepts (fields/topics) attached by OpenAlex
Cited by: 1

Total citation count in OpenAlex
Citations by year (recent): 2025: 1

Per-year citation counts (last 5 years)
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4402502890
doi	https://doi.org/10.48550/arxiv.2408.08902
ids.doi	https://doi.org/10.48550/arxiv.2408.08902
ids.openalex	https://openalex.org/W4402502890
fwci
type	preprint
title	Audit-LLM: Multi-Agent Collaboration for Log-based Insider Threat Detection
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10400
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9904999732971191
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1705
topics[0].subfield.display_name	Computer Networks and Communications
topics[0].display_name	Network Security and Intrusion Detection
topics[1].id	https://openalex.org/T11241
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9878000020980835
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1711
topics[1].subfield.display_name	Signal Processing
topics[1].display_name	Advanced Malware Detection Techniques
topics[2].id	https://openalex.org/T10734
topics[2].field.id	https://openalex.org/fields/17
topics[2].field.display_name	Computer Science
topics[2].score	0.9846000075340271
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/1710
topics[2].subfield.display_name	Information Systems
topics[2].display_name	Information and Cyber Security
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C2776633304
concepts[0].level	3
concepts[0].score	0.7239079475402832
concepts[0].wikidata	https://www.wikidata.org/wiki/Q6038026
concepts[0].display_name	Insider threat
concepts[1].id	https://openalex.org/C199521495
concepts[1].level	2
concepts[1].score	0.6648865342140198
concepts[1].wikidata	https://www.wikidata.org/wiki/Q181487
concepts[1].display_name	Audit
concepts[2].id	https://openalex.org/C2778971194
concepts[2].level	2
concepts[2].score	0.6476158499717712
concepts[2].wikidata	https://www.wikidata.org/wiki/Q1664551
concepts[2].display_name	Insider
concepts[3].id	https://openalex.org/C144133560
concepts[3].level	0
concepts[3].score	0.5336993932723999
concepts[3].wikidata	https://www.wikidata.org/wiki/Q4830453
concepts[3].display_name	Business
concepts[4].id	https://openalex.org/C41008148
concepts[4].level	0
concepts[4].score	0.3910177946090698
concepts[4].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[4].display_name	Computer science
concepts[5].id	https://openalex.org/C121955636
concepts[5].level	1
concepts[5].score	0.352754682302475
concepts[5].wikidata	https://www.wikidata.org/wiki/Q4116214
concepts[5].display_name	Accounting
concepts[6].id	https://openalex.org/C17744445
concepts[6].level	0
concepts[6].score	0.19335612654685974
concepts[6].wikidata	https://www.wikidata.org/wiki/Q36442
concepts[6].display_name	Political science
concepts[7].id	https://openalex.org/C199539241
concepts[7].level	1
concepts[7].score	0.06675472855567932
concepts[7].wikidata	https://www.wikidata.org/wiki/Q7748
concepts[7].display_name	Law
keywords[0].id	https://openalex.org/keywords/insider-threat
keywords[0].score	0.7239079475402832
keywords[0].display_name	Insider threat
keywords[1].id	https://openalex.org/keywords/audit
keywords[1].score	0.6648865342140198
keywords[1].display_name	Audit
keywords[2].id	https://openalex.org/keywords/insider
keywords[2].score	0.6476158499717712
keywords[2].display_name	Insider
keywords[3].id	https://openalex.org/keywords/business
keywords[3].score	0.5336993932723999
keywords[3].display_name	Business
keywords[4].id	https://openalex.org/keywords/computer-science
keywords[4].score	0.3910177946090698
keywords[4].display_name	Computer science
keywords[5].id	https://openalex.org/keywords/accounting
keywords[5].score	0.352754682302475
keywords[5].display_name	Accounting
keywords[6].id	https://openalex.org/keywords/political-science
keywords[6].score	0.19335612654685974
keywords[6].display_name	Political science
keywords[7].id	https://openalex.org/keywords/law
keywords[7].score	0.06675472855567932
keywords[7].display_name	Law
language	en
locations[0].id	pmh:oai:arXiv.org:2408.08902
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2408.08902
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2408.08902
locations[1].id	doi:10.48550/arxiv.2408.08902
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2408.08902
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5032690266
authorships[0].author.orcid	https://orcid.org/0000-0001-7020-7603
authorships[0].author.display_name	Chengyu Song
authorships[0].author_position	first
authorships[0].raw_author_name	Song, Chengyu
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5067868458
authorships[1].author.orcid	https://orcid.org/0000-0002-4530-5401
authorships[1].author.display_name	Linru Ma
authorships[1].author_position	middle
authorships[1].raw_author_name	Ma, Linru
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5023144456
authorships[2].author.orcid	https://orcid.org/0000-0002-4928-8194
authorships[2].author.display_name	Jianming Zheng
authorships[2].author_position	middle
authorships[2].raw_author_name	Zheng, Jianming
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5052860228
authorships[3].author.orcid	https://orcid.org/0000-0002-2898-6559
authorships[3].author.display_name	Jinzhi Liao
authorships[3].author_position	middle
authorships[3].raw_author_name	Liao, Jinzhi
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5029015256
authorships[4].author.orcid	https://orcid.org/0000-0002-2339-612X
authorships[4].author.display_name	Hongyu Kuang
authorships[4].author_position	middle
authorships[4].raw_author_name	Kuang, Hongyu
authorships[4].is_corresponding	False
authorships[5].author.id	https://openalex.org/A5101475284
authorships[5].author.orcid	https://orcid.org/0000-0002-6956-8177
authorships[5].author.display_name	Lin Yang
authorships[5].author_position	last
authorships[5].raw_author_name	Yang, Lin
authorships[5].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2408.08902
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2024-09-14T00:00:00
display_name	Audit-LLM: Multi-Agent Collaboration for Log-based Insider Threat Detection
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T10400
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9904999732971191
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1705
primary_topic.subfield.display_name	Computer Networks and Communications
primary_topic.display_name	Network Security and Intrusion Detection
related_works	https://openalex.org/W2766781562, https://openalex.org/W4205304595, https://openalex.org/W2979782961, https://openalex.org/W308359497, https://openalex.org/W1499596878, https://openalex.org/W3136170567, https://openalex.org/W2947769183, https://openalex.org/W2018332730, https://openalex.org/W4387194049, https://openalex.org/W2286217954
cited_by_count	1
counts_by_year[0].year	2025
counts_by_year[0].cited_by_count	1
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2408.08902
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2408.08902
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2408.08902
primary_location.id	pmh:oai:arXiv.org:2408.08902
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2408.08902
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2408.08902
publication_date	2024-08-12
publication_year	2024
referenced_works_count	0
abstract_inverted_index.a	39, 88, 152, 172
abstract_inverted_index.In	80
abstract_inverted_index.To	146
abstract_inverted_index.as	67
abstract_inverted_index.by	9, 142, 209
abstract_inverted_index.in	25, 44, 65, 130
abstract_inverted_index.of	28, 51, 190, 206
abstract_inverted_index.on	177
abstract_inverted_index.to	82, 125, 170
abstract_inverted_index.we	85, 150
abstract_inverted_index.(i)	99
abstract_inverted_index.ITD	107, 181
abstract_inverted_index.and	34, 77, 132, 186, 196
abstract_inverted_index.for	42, 123
abstract_inverted_index.its	62
abstract_inverted_index.log	11, 36
abstract_inverted_index.may	71
abstract_inverted_index.not	72
abstract_inverted_index.our	191
abstract_inverted_index.the	26, 55, 68, 100, 105, 116, 134, 138, 188, 199, 204
abstract_inverted_index.two	160
abstract_inverted_index.CERT	184
abstract_inverted_index.EMAD	201
abstract_inverted_index.ITD,	66
abstract_inverted_index.ITD.	29
abstract_inverted_index.LLMs	43, 60
abstract_inverted_index.Tool	117
abstract_inverted_index.down	104
abstract_inverted_index.from	59
abstract_inverted_index.have	23
abstract_inverted_index.into	109
abstract_inverted_index.ones	48
abstract_inverted_index.over	193
abstract_inverted_index.pose	38
abstract_inverted_index.show	197
abstract_inverted_index.task	108
abstract_inverted_index.that	198
abstract_inverted_index.user	7, 75
abstract_inverted_index.with	18, 74
abstract_inverted_index.(COT)	114
abstract_inverted_index.(ITD)	4
abstract_inverted_index.(iii)	133
abstract_inverted_index.LLMs.	210
abstract_inverted_index.LLMs;	131
abstract_inverted_index.align	73
abstract_inverted_index.files	37
abstract_inverted_index.final	139
abstract_inverted_index.issue	58
abstract_inverted_index.large	14
abstract_inverted_index.r4.2,	183
abstract_inverted_index.r5.2,	185
abstract_inverted_index.reach	171
abstract_inverted_index.sense	21
abstract_inverted_index.their	165
abstract_inverted_index.these	83
abstract_inverted_index.three	96, 178
abstract_inverted_index.tools	122
abstract_inverted_index.types	33
abstract_inverted_index.using	112
abstract_inverted_index.where	159
abstract_inverted_index.(EMAD)	157
abstract_inverted_index.(LLMs)	17
abstract_inverted_index.Debate	156
abstract_inverted_index.agent,	102, 119, 136
abstract_inverted_index.common	20
abstract_inverted_index.domain	27
abstract_inverted_index.length	128
abstract_inverted_index.method	192
abstract_inverted_index.models	16
abstract_inverted_index.normal	52
abstract_inverted_index.refine	164
abstract_inverted_index.strong	19
abstract_inverted_index.threat	2, 92
abstract_inverted_index.tools.	145
abstract_inverted_index.within	49
abstract_inverted_index.Builder	118
abstract_inverted_index.agents:	98
abstract_inverted_index.complex	106
abstract_inverted_index.context	127
abstract_inverted_index.detects	5
abstract_inverted_index.diverse	31
abstract_inverted_index.emerged	24
abstract_inverted_index.enhance	147
abstract_inverted_index.insider	1, 91
abstract_inverted_index.myriads	50
abstract_inverted_index.propose	151
abstract_inverted_index.through	167
abstract_inverted_index.Executor	135
abstract_inverted_index.activity	32, 78
abstract_inverted_index.auditing	10
abstract_inverted_index.breaking	103
abstract_inverted_index.commands	76
abstract_inverted_index.context.	79
abstract_inverted_index.creating	120
abstract_inverted_index.directly	45
abstract_inverted_index.entries.	12
abstract_inverted_index.exchange	169
abstract_inverted_index.existing	194
abstract_inverted_index.improves	203
abstract_inverted_index.invoking	143
abstract_inverted_index.language	15
abstract_inverted_index.overcome	126
abstract_inverted_index.overlong	35
abstract_inverted_index.proposed	200
abstract_inverted_index.publicly	179
abstract_inverted_index.response	81
abstract_inverted_index.reusable	121
abstract_inverted_index.Executors	162
abstract_inverted_index.Log-based	0
abstract_inverted_index.Recently,	13
abstract_inverted_index.accuracy,	149
abstract_inverted_index.available	180
abstract_inverted_index.baselines	195
abstract_inverted_index.challenge	41
abstract_inverted_index.conducted	176
abstract_inverted_index.detection	3, 93, 140
abstract_inverted_index.framework	94
abstract_inverted_index.generated	69, 208
abstract_inverted_index.introduce	86
abstract_inverted_index.knowledge	22
abstract_inverted_index.log-based	90
abstract_inverted_index.malicious	6, 47
abstract_inverted_index.pair-wise	153
abstract_inverted_index.reasoning	168
abstract_inverted_index.sub-tasks	111, 124
abstract_inverted_index.Audit-LLM,	87
abstract_inverted_index.Decomposer	101
abstract_inverted_index.activities	8
abstract_inverted_index.aggravates	61
abstract_inverted_index.comprising	95
abstract_inverted_index.conclusion	70, 141, 148
abstract_inverted_index.consensus.	173
abstract_inverted_index.difficulty	64
abstract_inverted_index.discerning	46
abstract_inverted_index.generating	137
abstract_inverted_index.manageable	110
abstract_inverted_index.mechanism,	158
abstract_inverted_index.Multi-agent	155
abstract_inverted_index.activities.	53
abstract_inverted_index.application	63
abstract_inverted_index.challenges,	84
abstract_inverted_index.conclusions	166
abstract_inverted_index.constructed	144
abstract_inverted_index.experiments	175
abstract_inverted_index.independent	161
abstract_inverted_index.iteratively	163
abstract_inverted_index.limitations	129
abstract_inverted_index.multi-agent	89
abstract_inverted_index.significant	40
abstract_inverted_index.superiority	189
abstract_inverted_index.Furthermore,	54
abstract_inverted_index.explanations	207
abstract_inverted_index.faithfulness	56, 205
abstract_inverted_index.Comprehensive	174
abstract_inverted_index.Nevertheless,	30
abstract_inverted_index.collaborative	97
abstract_inverted_index.datasets-CERT	182
abstract_inverted_index.hallucination	57
abstract_inverted_index.significantly	202
abstract_inverted_index.Evidence-based	154
abstract_inverted_index.reasoning;(ii)	115
abstract_inverted_index.Chain-of-Thought	113
abstract_inverted_index.PicoDomain-demonstrate	187
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	6
citation_normalized_percentile