Comparative Analysis of Loss Functions in TD3 forAutonomous Parking Article Swipe

View

Ka Heng Chan , Aida Mustapha , Mohammed Ahmed Jubair ·

YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.30880/jscdm.2024.05.01.001

Autonomous parking is a revolutionary technology that has transformed the automotive industry with the rise of deep reinforcement learning, in particular, the Twin-Delayed Deep Deterministic Policy Gradient Algorithm (TD3). Nonetheless, the robustness of TD3 remains a significant challenge due to bias in Q-value estimates when determining how good an Action, A, taken at a particular state, S. To investigate this gap, this paper analyzes different loss functions in TD3 to better approximate the true Q-value, which is necessary for optimal decision making. Three loss functions are evaluated; Mean Squared Error (MSE), Mean Absolute Error (MAE) and Huber Loss via a simulation experiment for autonomous parking. The results showed that TD3 with Huber Loss has the highest convergence speed with the fastest Actor and Critic loss convergence. The Huber Loss function is found to be more robust and efficient than either loss function such MSE or MAE used in isolation, making it a suitable replacement for existing loss functions in the TD3 algorithm. In the future, TD3 with Huber Loss will be used as the base model to solve overestimation problem in TD3 when the estimated Q-values that represent the expected rewards of taking an action in a particular state, are higher than their true values.

Related Topics

Transportation Engineering

Engineering

Concepts

Transport engineering Engineering

Metadata

Type: article
Language: en
Landing Page: https://doi.org/10.30880/jscdm.2024.05.01.001
OA Status: diamond
Cited By: 1
Related Works: 10
OpenAlex ID: https://openalex.org/W4403507353

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4403507353

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.30880/jscdm.2024.05.01.001

Digital Object Identifier
Title: Comparative Analysis of Loss Functions in TD3 forAutonomous Parking

Work title
Type: article

OpenAlex work type
Language: en

Primary language
Publication year: 2024

Year of publication
Publication date: 2024-06-21

Full publication date if available
Authors: Ka Heng Chan, Aida Mustapha, Mohammed Ahmed Jubair

List of authors in order
Landing page: https://doi.org/10.30880/jscdm.2024.05.01.001

Publisher landing page
Open access: Yes

Whether a free full text is available
OA status: diamond

Open access status per OpenAlex
OA URL: https://doi.org/10.30880/jscdm.2024.05.01.001

Direct OA link when available
Concepts: Transport engineering, Engineering

Top concepts (fields/topics) attached by OpenAlex
Cited by: 1

Total citation count in OpenAlex
Citations by year (recent): 2025: 1

Per-year citation counts (last 5 years)
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4403507353
doi	https://doi.org/10.30880/jscdm.2024.05.01.001
ids.doi	https://doi.org/10.30880/jscdm.2024.05.01.001
ids.openalex	https://openalex.org/W4403507353
fwci	0.53997443
type	article
title	Comparative Analysis of Loss Functions in TD3 forAutonomous Parking
biblio.issue	1
biblio.volume	5
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T12546
topics[0].field.id	https://openalex.org/fields/22
topics[0].field.display_name	Engineering
topics[0].score	0.9799000024795532
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/2215
topics[0].subfield.display_name	Building and Construction
topics[0].display_name	Smart Parking Systems Research
topics[1].id	https://openalex.org/T10761
topics[1].field.id	https://openalex.org/fields/22
topics[1].field.display_name	Engineering
topics[1].score	0.9549000263214111
topics[1].domain.id	https://openalex.org/domains/3
topics[1].domain.display_name	Physical Sciences
topics[1].subfield.id	https://openalex.org/subfields/2208
topics[1].subfield.display_name	Electrical and Electronic Engineering
topics[1].display_name	Vehicular Ad Hoc Networks (VANETs)
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C22212356
concepts[0].level	1
concepts[0].score	0.31690865755081177
concepts[0].wikidata	https://www.wikidata.org/wiki/Q775325
concepts[0].display_name	Transport engineering
concepts[1].id	https://openalex.org/C127413603
concepts[1].level	0
concepts[1].score	0.2183929979801178
concepts[1].wikidata	https://www.wikidata.org/wiki/Q11023
concepts[1].display_name	Engineering
keywords[0].id	https://openalex.org/keywords/transport-engineering
keywords[0].score	0.31690865755081177
keywords[0].display_name	Transport engineering
keywords[1].id	https://openalex.org/keywords/engineering
keywords[1].score	0.2183929979801178
keywords[1].display_name	Engineering
language	en
locations[0].id	doi:10.30880/jscdm.2024.05.01.001
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4210211867
locations[0].source.issn	2716-621X
locations[0].source.type	journal
locations[0].source.is_oa	True
locations[0].source.issn_l	2716-621X
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	Journal of Soft Computing and Data Mining
locations[0].source.host_organization	https://openalex.org/P4310322246
locations[0].source.host_organization_name	Penerbit UTHM
locations[0].source.host_organization_lineage	https://openalex.org/P4310322246
locations[0].source.host_organization_lineage_names	Penerbit UTHM
locations[0].license
locations[0].pdf_url
locations[0].version	publishedVersion
locations[0].raw_type	journal-article
locations[0].license_id
locations[0].is_accepted	True
locations[0].is_published	True
locations[0].raw_source_name	Journal of Soft Computing and Data Mining
locations[0].landing_page_url	https://doi.org/10.30880/jscdm.2024.05.01.001
indexed_in	crossref
authorships[0].author.id	https://openalex.org/A5062983141
authorships[0].author.orcid
authorships[0].author.display_name	Ka Heng Chan
authorships[0].author_position	first
authorships[0].raw_author_name	Ka Heng Chan
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5032045434
authorships[1].author.orcid	https://orcid.org/0000-0002-9077-4995
authorships[1].author.display_name	Aida Mustapha
authorships[1].author_position	middle
authorships[1].raw_author_name	Aida Mustapha
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5107930857
authorships[2].author.orcid
authorships[2].author.display_name	Mohammed Ahmed Jubair
authorships[2].author_position	last
authorships[2].raw_author_name	Mohammed Ahmed Jubair
authorships[2].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://doi.org/10.30880/jscdm.2024.05.01.001
open_access.oa_status	diamond
open_access.any_repository_has_fulltext	False
created_date	2024-10-18T00:00:00
display_name	Comparative Analysis of Loss Functions in TD3 forAutonomous Parking
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T03:46:38.306776
primary_topic.id	https://openalex.org/T12546
primary_topic.field.id	https://openalex.org/fields/22
primary_topic.field.display_name	Engineering
primary_topic.score	0.9799000024795532
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/2215
primary_topic.subfield.display_name	Building and Construction
primary_topic.display_name	Smart Parking Systems Research
related_works	https://openalex.org/W590383186, https://openalex.org/W1595912296, https://openalex.org/W567987265, https://openalex.org/W1596963112, https://openalex.org/W2362891875, https://openalex.org/W2290266725, https://openalex.org/W2406535223, https://openalex.org/W574715732, https://openalex.org/W11458981, https://openalex.org/W2742581101
cited_by_count	1
counts_by_year[0].year	2025
counts_by_year[0].cited_by_count	1
locations_count	1
best_oa_location.id	doi:10.30880/jscdm.2024.05.01.001
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4210211867
best_oa_location.source.issn	2716-621X
best_oa_location.source.type	journal
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l	2716-621X
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	Journal of Soft Computing and Data Mining
best_oa_location.source.host_organization	https://openalex.org/P4310322246
best_oa_location.source.host_organization_name	Penerbit UTHM
best_oa_location.source.host_organization_lineage	https://openalex.org/P4310322246
best_oa_location.source.host_organization_lineage_names	Penerbit UTHM
best_oa_location.license
best_oa_location.pdf_url
best_oa_location.version	publishedVersion
best_oa_location.raw_type	journal-article
best_oa_location.license_id
best_oa_location.is_accepted	True
best_oa_location.is_published	True
best_oa_location.raw_source_name	Journal of Soft Computing and Data Mining
best_oa_location.landing_page_url	https://doi.org/10.30880/jscdm.2024.05.01.001
primary_location.id	doi:10.30880/jscdm.2024.05.01.001
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4210211867
primary_location.source.issn	2716-621X
primary_location.source.type	journal
primary_location.source.is_oa	True
primary_location.source.issn_l	2716-621X
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	Journal of Soft Computing and Data Mining
primary_location.source.host_organization	https://openalex.org/P4310322246
primary_location.source.host_organization_name	Penerbit UTHM
primary_location.source.host_organization_lineage	https://openalex.org/P4310322246
primary_location.source.host_organization_lineage_names	Penerbit UTHM
primary_location.license
primary_location.pdf_url
primary_location.version	publishedVersion
primary_location.raw_type	journal-article
primary_location.license_id
primary_location.is_accepted	True
primary_location.is_published	True
primary_location.raw_source_name	Journal of Soft Computing and Data Mining
primary_location.landing_page_url	https://doi.org/10.30880/jscdm.2024.05.01.001
publication_date	2024-06-21
publication_year	2024
referenced_works_count	0
abstract_inverted_index.a	3, 35, 53, 99, 151, 196
abstract_inverted_index.A,	50
abstract_inverted_index.In	162
abstract_inverted_index.S.	56
abstract_inverted_index.To	57
abstract_inverted_index.an	48, 193
abstract_inverted_index.as	172
abstract_inverted_index.at	52
abstract_inverted_index.be	133, 170
abstract_inverted_index.in	19, 41, 67, 147, 158, 180, 195
abstract_inverted_index.is	2, 76, 130
abstract_inverted_index.it	150
abstract_inverted_index.of	15, 32, 191
abstract_inverted_index.or	144
abstract_inverted_index.to	39, 69, 132, 176
abstract_inverted_index.MAE	145
abstract_inverted_index.MSE	143
abstract_inverted_index.TD3	33, 68, 109, 160, 165, 181
abstract_inverted_index.The	105, 126
abstract_inverted_index.and	95, 122, 136
abstract_inverted_index.are	85, 199
abstract_inverted_index.due	38
abstract_inverted_index.for	78, 102, 154
abstract_inverted_index.has	7, 113
abstract_inverted_index.how	46
abstract_inverted_index.the	9, 13, 21, 30, 72, 114, 119, 159, 163, 173, 183, 188
abstract_inverted_index.via	98
abstract_inverted_index.Deep	23
abstract_inverted_index.Loss	97, 112, 128, 168
abstract_inverted_index.Mean	87, 91
abstract_inverted_index.base	174
abstract_inverted_index.bias	40
abstract_inverted_index.deep	16
abstract_inverted_index.gap,	60
abstract_inverted_index.good	47
abstract_inverted_index.loss	65, 83, 124, 140, 156
abstract_inverted_index.more	134
abstract_inverted_index.rise	14
abstract_inverted_index.such	142
abstract_inverted_index.than	138, 201
abstract_inverted_index.that	6, 108, 186
abstract_inverted_index.this	59, 61
abstract_inverted_index.true	73, 203
abstract_inverted_index.used	146, 171
abstract_inverted_index.when	44, 182
abstract_inverted_index.will	169
abstract_inverted_index.with	12, 110, 118, 166
abstract_inverted_index.(MAE)	94
abstract_inverted_index.Actor	121
abstract_inverted_index.Error	89, 93
abstract_inverted_index.Huber	96, 111, 127, 167
abstract_inverted_index.Three	82
abstract_inverted_index.found	131
abstract_inverted_index.model	175
abstract_inverted_index.paper	62
abstract_inverted_index.solve	177
abstract_inverted_index.speed	117
abstract_inverted_index.taken	51
abstract_inverted_index.their	202
abstract_inverted_index.which	75
abstract_inverted_index.(MSE),	90
abstract_inverted_index.(TD3).	28
abstract_inverted_index.Critic	123
abstract_inverted_index.Policy	25
abstract_inverted_index.action	194
abstract_inverted_index.better	70
abstract_inverted_index.either	139
abstract_inverted_index.higher	200
abstract_inverted_index.making	149
abstract_inverted_index.robust	135
abstract_inverted_index.showed	107
abstract_inverted_index.state,	55, 198
abstract_inverted_index.taking	192
abstract_inverted_index.Action,	49
abstract_inverted_index.Q-value	42
abstract_inverted_index.Squared	88
abstract_inverted_index.fastest	120
abstract_inverted_index.future,	164
abstract_inverted_index.highest	115
abstract_inverted_index.making.	81
abstract_inverted_index.optimal	79
abstract_inverted_index.parking	1
abstract_inverted_index.problem	179
abstract_inverted_index.remains	34
abstract_inverted_index.results	106
abstract_inverted_index.rewards	190
abstract_inverted_index.values.	204
abstract_inverted_index.Absolute	92
abstract_inverted_index.Gradient	26
abstract_inverted_index.Q-value,	74
abstract_inverted_index.Q-values	185
abstract_inverted_index.analyzes	63
abstract_inverted_index.decision	80
abstract_inverted_index.existing	155
abstract_inverted_index.expected	189
abstract_inverted_index.function	129, 141
abstract_inverted_index.industry	11
abstract_inverted_index.parking.	104
abstract_inverted_index.suitable	152
abstract_inverted_index.Algorithm	27
abstract_inverted_index.challenge	37
abstract_inverted_index.different	64
abstract_inverted_index.efficient	137
abstract_inverted_index.estimated	184
abstract_inverted_index.estimates	43
abstract_inverted_index.functions	66, 84, 157
abstract_inverted_index.learning,	18
abstract_inverted_index.necessary	77
abstract_inverted_index.represent	187
abstract_inverted_index.Autonomous	0
abstract_inverted_index.algorithm.	161
abstract_inverted_index.automotive	10
abstract_inverted_index.autonomous	103
abstract_inverted_index.evaluated;	86
abstract_inverted_index.experiment	101
abstract_inverted_index.isolation,	148
abstract_inverted_index.particular	54, 197
abstract_inverted_index.robustness	31
abstract_inverted_index.simulation	100
abstract_inverted_index.technology	5
abstract_inverted_index.approximate	71
abstract_inverted_index.convergence	116
abstract_inverted_index.determining	45
abstract_inverted_index.investigate	58
abstract_inverted_index.particular,	20
abstract_inverted_index.replacement	153
abstract_inverted_index.significant	36
abstract_inverted_index.transformed	8
abstract_inverted_index.Nonetheless,	29
abstract_inverted_index.Twin-Delayed	22
abstract_inverted_index.convergence.	125
abstract_inverted_index.Deterministic	24
abstract_inverted_index.reinforcement	17
abstract_inverted_index.revolutionary	4
abstract_inverted_index.overestimation	178
cited_by_percentile_year.max	95
cited_by_percentile_year.min	91
countries_distinct_count	0
institutions_distinct_count	3
citation_normalized_percentile.value	0.62310511
citation_normalized_percentile.is_in_top_1_percent	False
citation_normalized_percentile.is_in_top_10_percent	False