Adaptive Monitoring and Real-World Evaluation of Agentic AI Systems Article Swipe

PDF

Manish Shukla ·

YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.21203/rs.3.rs-7497109/v1

Agentic artificial intelligence (AI) — multi-agent systems that combine large language models with external tools and autonomous planning — are rapidly transitioning from research laboratories into high-stakes domains. Our earlier “Basic” paper introduced a five-axis framework and proposed preliminary metrics such as goal drift and harm reduction but did not provide an algorithmic instantiation or empirical evidence. This “Advanced” sequel fills that gap. First, we revisit recent benchmarks and industrial deployments to show that technical metrics still dominate evaluations: a systematic review of 84 papers from 2023–2025 found that 83% report capability metrics while only 30% consider human-centred or economic axes[2]. Second, we formalise an Adaptive Multi-Dimensional Monitoring (AMDM) algorithm that normalises heterogeneous metrics, applies per-axis exponentially weighted moving-average thresholds and performs joint anomaly detection via the Mahalanobis distance. Third, we conduct simulations and real-world experiments. AMDM cuts anomaly-detection latency from 12.3 s to 5.6 s on simulated goal drift and reduces false-positive rates from 4.5% to 0.9% compared with static thresholds. We present a comparison table and ROC/PR curves, and we reanalyse case studies to surface missing metrics. Code, data and a reproducibility checklist accompany this paper to facilitate replication.

Related Topics

Concepts

No concepts available.

Metadata

Type: preprint
Language: en
Landing Page: https://doi.org/10.21203/rs.3.rs-7497109/v1
PDF: https://www.researchsquare.com/article/rs-7497109/latest.pdf
OA Status: gold
Related Works: 10
OpenAlex ID: https://openalex.org/W4414142579

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4414142579

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.21203/rs.3.rs-7497109/v1

Digital Object Identifier
Title: Adaptive Monitoring and Real-World Evaluation of Agentic AI Systems

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2025

Year of publication
Publication date: 2025-09-12

Full publication date if available
Authors: Manish Shukla

List of authors in order
Landing page: https://doi.org/10.21203/rs.3.rs-7497109/v1

Publisher landing page
PDF URL: https://www.researchsquare.com/article/rs-7497109/latest.pdf

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: gold

Open access status per OpenAlex
OA URL: https://www.researchsquare.com/article/rs-7497109/latest.pdf

Direct OA link when available
Cited by: 0

Total citation count in OpenAlex
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4414142579
doi	https://doi.org/10.21203/rs.3.rs-7497109/v1
ids.doi	https://doi.org/10.21203/rs.3.rs-7497109/v1
ids.openalex	https://openalex.org/W4414142579
fwci	0.0
type	preprint
title	Adaptive Monitoring and Real-World Evaluation of Agentic AI Systems
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T11512
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9894000291824341
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Anomaly Detection Techniques and Applications
topics[1].id	https://openalex.org/T12761
topics[1].field.id	https://openalex.org/fields/17
topics[1].field.display_name	Computer Science
topics[1].score	0.9581000208854675
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/1702
topics[1].subfield.display_name	Artificial Intelligence
topics[1].display_name	Data Stream Mining Techniques
topics[2].id	https://openalex.org/T10400
topics[2].field.id	https://openalex.org/fields/17
topics[2].field.display_name	Computer Science
topics[2].score	0.9174000024795532
topics[2].domain.id	https://openalex.org/domains/3
topics[2].domain.display_name	Physical Sciences
topics[2].subfield.id	https://openalex.org/subfields/1705
topics[2].subfield.display_name	Computer Networks and Communications
topics[2].display_name	Network Security and Intrusion Detection
is_xpac	False
apc_list
apc_paid
language	en
locations[0].id	doi:10.21203/rs.3.rs-7497109/v1
locations[0].is_oa	True
locations[0].source
locations[0].license	cc-by
locations[0].pdf_url	https://www.researchsquare.com/article/rs-7497109/latest.pdf
locations[0].version	acceptedVersion
locations[0].raw_type	posted-content
locations[0].license_id	https://openalex.org/licenses/cc-by
locations[0].is_accepted	True
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	https://doi.org/10.21203/rs.3.rs-7497109/v1
indexed_in	crossref
authorships[0].author.id	https://openalex.org/A5101737653
authorships[0].author.orcid	https://orcid.org/0000-0003-4867-3530
authorships[0].author.display_name	Manish Shukla
authorships[0].affiliations[0].raw_affiliation_string	Independent Researcher
authorships[0].author_position	first
authorships[0].raw_author_name	Manish A. Shukla
authorships[0].is_corresponding	True
authorships[0].raw_affiliation_strings	Independent Researcher
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://www.researchsquare.com/article/rs-7497109/latest.pdf
open_access.oa_status	gold
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Adaptive Monitoring and Real-World Evaluation of Agentic AI Systems
has_fulltext	True
is_retracted	False
updated_date	2025-11-06T03:46:38.306776
primary_topic.id	https://openalex.org/T11512
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9894000291824341
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Anomaly Detection Techniques and Applications
related_works	https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W4391913857, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052
cited_by_count	0
locations_count	1
best_oa_location.id	doi:10.21203/rs.3.rs-7497109/v1
best_oa_location.is_oa	True
best_oa_location.source
best_oa_location.license	cc-by
best_oa_location.pdf_url	https://www.researchsquare.com/article/rs-7497109/latest.pdf
best_oa_location.version	acceptedVersion
best_oa_location.raw_type	posted-content
best_oa_location.license_id	https://openalex.org/licenses/cc-by
best_oa_location.is_accepted	True
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	https://doi.org/10.21203/rs.3.rs-7497109/v1
primary_location.id	doi:10.21203/rs.3.rs-7497109/v1
primary_location.is_oa	True
primary_location.source
primary_location.license	cc-by
primary_location.pdf_url	https://www.researchsquare.com/article/rs-7497109/latest.pdf
primary_location.version	acceptedVersion
primary_location.raw_type	posted-content
primary_location.license_id	https://openalex.org/licenses/cc-by
primary_location.is_accepted	True
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	https://doi.org/10.21203/rs.3.rs-7497109/v1
publication_date	2025-09-12
publication_year	2025
referenced_works_count	0
abstract_inverted_index.a	34, 80, 165, 183
abstract_inverted_index.s	143, 146
abstract_inverted_index.84	84
abstract_inverted_index.We	163
abstract_inverted_index.an	52, 105
abstract_inverted_index.as	42
abstract_inverted_index.of	83
abstract_inverted_index.on	147
abstract_inverted_index.or	55, 99
abstract_inverted_index.to	72, 144, 157, 176, 189
abstract_inverted_index.we	65, 103, 131, 172
abstract_inverted_index.30%	96
abstract_inverted_index.5.6	145
abstract_inverted_index.83%	90
abstract_inverted_index.Our	29
abstract_inverted_index.and	16, 37, 45, 69, 121, 134, 151, 168, 171, 182
abstract_inverted_index.are	20
abstract_inverted_index.but	48
abstract_inverted_index.did	49
abstract_inverted_index.not	50
abstract_inverted_index.the	127
abstract_inverted_index.via	126
abstract_inverted_index.—	5, 19
abstract_inverted_index.(AI)	4
abstract_inverted_index.0.9%	158
abstract_inverted_index.12.3	142
abstract_inverted_index.4.5%	156
abstract_inverted_index.AMDM	137
abstract_inverted_index.This	58
abstract_inverted_index.case	174
abstract_inverted_index.cuts	138
abstract_inverted_index.data	181
abstract_inverted_index.from	23, 86, 141, 155
abstract_inverted_index.gap.	63
abstract_inverted_index.goal	43, 149
abstract_inverted_index.harm	46
abstract_inverted_index.into	26
abstract_inverted_index.only	95
abstract_inverted_index.show	73
abstract_inverted_index.such	41
abstract_inverted_index.that	8, 62, 74, 89, 111
abstract_inverted_index.this	187
abstract_inverted_index.with	13, 160
abstract_inverted_index.Code,	180
abstract_inverted_index.drift	44, 150
abstract_inverted_index.fills	61
abstract_inverted_index.found	88
abstract_inverted_index.joint	123
abstract_inverted_index.large	10
abstract_inverted_index.paper	32, 188
abstract_inverted_index.rates	154
abstract_inverted_index.still	77
abstract_inverted_index.table	167
abstract_inverted_index.tools	15
abstract_inverted_index.while	94
abstract_inverted_index.(AMDM)	109
abstract_inverted_index.First,	64
abstract_inverted_index.ROC/PR	169
abstract_inverted_index.Third,	130
abstract_inverted_index.models	12
abstract_inverted_index.papers	85
abstract_inverted_index.recent	67
abstract_inverted_index.report	91
abstract_inverted_index.review	82
abstract_inverted_index.sequel	60
abstract_inverted_index.static	161
abstract_inverted_index.Agentic	1
abstract_inverted_index.Second,	102
abstract_inverted_index.anomaly	124
abstract_inverted_index.applies	115
abstract_inverted_index.combine	9
abstract_inverted_index.conduct	132
abstract_inverted_index.curves,	170
abstract_inverted_index.earlier	30
abstract_inverted_index.latency	140
abstract_inverted_index.metrics	40, 76, 93
abstract_inverted_index.missing	178
abstract_inverted_index.present	164
abstract_inverted_index.provide	51
abstract_inverted_index.rapidly	21
abstract_inverted_index.reduces	152
abstract_inverted_index.revisit	66
abstract_inverted_index.studies	175
abstract_inverted_index.surface	177
abstract_inverted_index.systems	7
abstract_inverted_index.Adaptive	106
abstract_inverted_index.axes[2].	101
abstract_inverted_index.compared	159
abstract_inverted_index.consider	97
abstract_inverted_index.domains.	28
abstract_inverted_index.dominate	78
abstract_inverted_index.economic	100
abstract_inverted_index.external	14
abstract_inverted_index.language	11
abstract_inverted_index.metrics,	114
abstract_inverted_index.metrics.	179
abstract_inverted_index.per-axis	116
abstract_inverted_index.performs	122
abstract_inverted_index.planning	18
abstract_inverted_index.proposed	38
abstract_inverted_index.research	24
abstract_inverted_index.weighted	118
abstract_inverted_index.accompany	186
abstract_inverted_index.algorithm	110
abstract_inverted_index.checklist	185
abstract_inverted_index.detection	125
abstract_inverted_index.distance.	129
abstract_inverted_index.empirical	56
abstract_inverted_index.evidence.	57
abstract_inverted_index.five-axis	35
abstract_inverted_index.formalise	104
abstract_inverted_index.framework	36
abstract_inverted_index.reanalyse	173
abstract_inverted_index.reduction	47
abstract_inverted_index.simulated	148
abstract_inverted_index.technical	75
abstract_inverted_index.Monitoring	108
abstract_inverted_index.artificial	2
abstract_inverted_index.autonomous	17
abstract_inverted_index.benchmarks	68
abstract_inverted_index.capability	92
abstract_inverted_index.comparison	166
abstract_inverted_index.facilitate	190
abstract_inverted_index.industrial	70
abstract_inverted_index.introduced	33
abstract_inverted_index.normalises	112
abstract_inverted_index.real-world	135
abstract_inverted_index.systematic	81
abstract_inverted_index.thresholds	120
abstract_inverted_index.2023–2025	87
abstract_inverted_index.Mahalanobis	128
abstract_inverted_index.algorithmic	53
abstract_inverted_index.deployments	71
abstract_inverted_index.high-stakes	27
abstract_inverted_index.multi-agent	6
abstract_inverted_index.preliminary	39
abstract_inverted_index.simulations	133
abstract_inverted_index.thresholds.	162
abstract_inverted_index.“Basic”	31
abstract_inverted_index.evaluations:	79
abstract_inverted_index.experiments.	136
abstract_inverted_index.intelligence	3
abstract_inverted_index.laboratories	25
abstract_inverted_index.replication.	191
abstract_inverted_index.exponentially	117
abstract_inverted_index.heterogeneous	113
abstract_inverted_index.human-centred	98
abstract_inverted_index.instantiation	54
abstract_inverted_index.transitioning	22
abstract_inverted_index.false-positive	153
abstract_inverted_index.moving-average	119
abstract_inverted_index.“Advanced”	59
abstract_inverted_index.reproducibility	184
abstract_inverted_index.Multi-Dimensional	107
abstract_inverted_index.anomaly-detection	139
abstract_inverted_index.<title>Abstract</title>	0
cited_by_percentile_year
corresponding_author_ids	https://openalex.org/A5101737653
countries_distinct_count	0
institutions_distinct_count	1
citation_normalized_percentile.value	0.18661588
citation_normalized_percentile.is_in_top_1_percent	False
citation_normalized_percentile.is_in_top_10_percent	True