Discrete-Time Approximations of Controlled Diffusions with Infinite Horizon Discounted and Average Cost Article Swipe

PDF

YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2502.05596

We present discrete-time approximation of optimal control policies for infinite horizon discounted/ergodic control problems for controlled diffusions in $\Rd$\,. In particular, our objective is to show near optimality of optimal policies designed from the approximating discrete-time controlled Markov chain model, for the discounted/ergodic optimal control problems, in the true controlled diffusion model (as the sampling period approaches zero). To this end, we first construct suitable discrete-time controlled Markov chain models for which one can compute optimal policies and optimal values via several methods (such as value iteration, convex analytic method, reinforcement learning etc.). Then using a weak convergence technique, we show that the optimal policy designed for the discrete-time Markov chain model is near-optimal for the controlled diffusion model as the discrete-time model approaches the continuous-time model. This provides a practical approach for finding near-optimal control policies for controlled diffusions. Our conditions complement existing results in the literature, which have been arrived at via either probabilistic or PDE based methods.

Related Topics

Concepts

Discrete time and continuous time Approximations of π Horizon Mathematics Time horizon Applied mathematics Mathematical economics Economics Mathematical optimization Econometrics Statistics Geometry

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2502.05596
PDF: https://arxiv.org/pdf/2502.05596
OA Status: green
Related Works: 10
OpenAlex ID: https://openalex.org/W4407385158

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4407385158

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2502.05596

Digital Object Identifier
Title: Discrete-Time Approximations of Controlled Diffusions with Infinite Horizon Discounted and Average Cost

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2025

Year of publication
Publication date: 2025-02-08

Full publication date if available
Authors: Somnath Pradhan, Serdar Yüksel

List of authors in order
Landing page: https://arxiv.org/abs/2502.05596

Publisher landing page
PDF URL: https://arxiv.org/pdf/2502.05596

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2502.05596

Direct OA link when available
Concepts: Discrete time and continuous time, Approximations of π, Horizon, Mathematics, Time horizon, Applied mathematics, Mathematical economics, Economics, Mathematical optimization, Econometrics, Statistics, Geometry

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4407385158
doi	https://doi.org/10.48550/arxiv.2502.05596
ids.doi	https://doi.org/10.48550/arxiv.2502.05596
ids.openalex	https://openalex.org/W4407385158
fwci
type	preprint
title	Discrete-Time Approximations of Controlled Diffusions with Infinite Horizon Discounted and Average Cost
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T12727
topics[0].field.id	https://openalex.org/fields/26
topics[0].field.display_name	Mathematics
topics[0].score	0.9377999901771545
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/2612
topics[0].subfield.display_name	Numerical Analysis
topics[0].display_name	Differential Equations and Numerical Methods
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C55689738
concepts[0].level	2
concepts[0].score	0.6669381856918335
concepts[0].wikidata	https://www.wikidata.org/wiki/Q15963867
concepts[0].display_name	Discrete time and continuous time
concepts[1].id	https://openalex.org/C193386753
concepts[1].level	2
concepts[1].score	0.5114966630935669
concepts[1].wikidata	https://www.wikidata.org/wiki/Q1130396
concepts[1].display_name	Approximations of π
concepts[2].id	https://openalex.org/C159176650
concepts[2].level	2
concepts[2].score	0.5112051963806152
concepts[2].wikidata	https://www.wikidata.org/wiki/Q43261
concepts[2].display_name	Horizon
concepts[3].id	https://openalex.org/C33923547
concepts[3].level	0
concepts[3].score	0.5034777522087097
concepts[3].wikidata	https://www.wikidata.org/wiki/Q395
concepts[3].display_name	Mathematics
concepts[4].id	https://openalex.org/C28761237
concepts[4].level	2
concepts[4].score	0.4850429892539978
concepts[4].wikidata	https://www.wikidata.org/wiki/Q7805321
concepts[4].display_name	Time horizon
concepts[5].id	https://openalex.org/C28826006
concepts[5].level	1
concepts[5].score	0.48271986842155457
concepts[5].wikidata	https://www.wikidata.org/wiki/Q33521
concepts[5].display_name	Applied mathematics
concepts[6].id	https://openalex.org/C144237770
concepts[6].level	1
concepts[6].score	0.4009351134300232
concepts[6].wikidata	https://www.wikidata.org/wiki/Q747534
concepts[6].display_name	Mathematical economics
concepts[7].id	https://openalex.org/C162324750
concepts[7].level	0
concepts[7].score	0.38371822237968445
concepts[7].wikidata	https://www.wikidata.org/wiki/Q8134
concepts[7].display_name	Economics
concepts[8].id	https://openalex.org/C126255220
concepts[8].level	1
concepts[8].score	0.3371858596801758
concepts[8].wikidata	https://www.wikidata.org/wiki/Q141495
concepts[8].display_name	Mathematical optimization
concepts[9].id	https://openalex.org/C149782125
concepts[9].level	1
concepts[9].score	0.33119940757751465
concepts[9].wikidata	https://www.wikidata.org/wiki/Q160039
concepts[9].display_name	Econometrics
concepts[10].id	https://openalex.org/C105795698
concepts[10].level	1
concepts[10].score	0.20742958784103394
concepts[10].wikidata	https://www.wikidata.org/wiki/Q12483
concepts[10].display_name	Statistics
concepts[11].id	https://openalex.org/C2524010
concepts[11].level	1
concepts[11].score	0.05939525365829468
concepts[11].wikidata	https://www.wikidata.org/wiki/Q8087
concepts[11].display_name	Geometry
keywords[0].id	https://openalex.org/keywords/discrete-time-and-continuous-time
keywords[0].score	0.6669381856918335
keywords[0].display_name	Discrete time and continuous time
keywords[1].id	https://openalex.org/keywords/approximations-of-π
keywords[1].score	0.5114966630935669
keywords[1].display_name	Approximations of π
keywords[2].id	https://openalex.org/keywords/horizon
keywords[2].score	0.5112051963806152
keywords[2].display_name	Horizon
keywords[3].id	https://openalex.org/keywords/mathematics
keywords[3].score	0.5034777522087097
keywords[3].display_name	Mathematics
keywords[4].id	https://openalex.org/keywords/time-horizon
keywords[4].score	0.4850429892539978
keywords[4].display_name	Time horizon
keywords[5].id	https://openalex.org/keywords/applied-mathematics
keywords[5].score	0.48271986842155457
keywords[5].display_name	Applied mathematics
keywords[6].id	https://openalex.org/keywords/mathematical-economics
keywords[6].score	0.4009351134300232
keywords[6].display_name	Mathematical economics
keywords[7].id	https://openalex.org/keywords/economics
keywords[7].score	0.38371822237968445
keywords[7].display_name	Economics
keywords[8].id	https://openalex.org/keywords/mathematical-optimization
keywords[8].score	0.3371858596801758
keywords[8].display_name	Mathematical optimization
keywords[9].id	https://openalex.org/keywords/econometrics
keywords[9].score	0.33119940757751465
keywords[9].display_name	Econometrics
keywords[10].id	https://openalex.org/keywords/statistics
keywords[10].score	0.20742958784103394
keywords[10].display_name	Statistics
keywords[11].id	https://openalex.org/keywords/geometry
keywords[11].score	0.05939525365829468
keywords[11].display_name	Geometry
language	en
locations[0].id	pmh:oai:arXiv.org:2502.05596
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2502.05596
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2502.05596
locations[1].id	doi:10.48550/arxiv.2502.05596
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2502.05596
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5034450946
authorships[0].author.orcid	https://orcid.org/0000-0002-1470-8240
authorships[0].author.display_name	Somnath Pradhan
authorships[0].author_position	first
authorships[0].raw_author_name	Pradhan, Somnath
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5005401257
authorships[1].author.orcid	https://orcid.org/0000-0001-6099-5001
authorships[1].author.display_name	Serdar Yüksel
authorships[1].author_position	last
authorships[1].raw_author_name	Yuksel, Serdar
authorships[1].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2502.05596
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Discrete-Time Approximations of Controlled Diffusions with Infinite Horizon Discounted and Average Cost
has_fulltext	False
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T12727
primary_topic.field.id	https://openalex.org/fields/26
primary_topic.field.display_name	Mathematics
primary_topic.score	0.9377999901771545
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/2612
primary_topic.subfield.display_name	Numerical Analysis
primary_topic.display_name	Differential Equations and Numerical Methods
related_works	https://openalex.org/W2906267174, https://openalex.org/W134501823, https://openalex.org/W2295626654, https://openalex.org/W2013239059, https://openalex.org/W4286622840, https://openalex.org/W2536121351, https://openalex.org/W2809608782, https://openalex.org/W2333245620, https://openalex.org/W1889576908, https://openalex.org/W3147633063
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2502.05596
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2502.05596
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2502.05596
primary_location.id	pmh:oai:arXiv.org:2502.05596
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2502.05596
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2502.05596
publication_date	2025-02-08
publication_year	2025
referenced_works_count	0
abstract_inverted_index.a	95, 129
abstract_inverted_index.In	19
abstract_inverted_index.To	58
abstract_inverted_index.We	0
abstract_inverted_index.as	84, 119
abstract_inverted_index.at	152
abstract_inverted_index.in	17, 46, 145
abstract_inverted_index.is	23, 112
abstract_inverted_index.of	4, 28
abstract_inverted_index.or	156
abstract_inverted_index.to	24
abstract_inverted_index.we	61, 99
abstract_inverted_index.(as	52
abstract_inverted_index.Our	140
abstract_inverted_index.PDE	157
abstract_inverted_index.and	77
abstract_inverted_index.can	73
abstract_inverted_index.for	8, 14, 40, 70, 106, 114, 132, 137
abstract_inverted_index.one	72
abstract_inverted_index.our	21
abstract_inverted_index.the	33, 41, 47, 53, 102, 107, 115, 120, 124, 146
abstract_inverted_index.via	80, 153
abstract_inverted_index.Then	93
abstract_inverted_index.This	127
abstract_inverted_index.been	150
abstract_inverted_index.end,	60
abstract_inverted_index.from	32
abstract_inverted_index.have	149
abstract_inverted_index.near	26
abstract_inverted_index.show	25, 100
abstract_inverted_index.that	101
abstract_inverted_index.this	59
abstract_inverted_index.true	48
abstract_inverted_index.weak	96
abstract_inverted_index.(such	83
abstract_inverted_index.based	158
abstract_inverted_index.chain	38, 68, 110
abstract_inverted_index.first	62
abstract_inverted_index.model	51, 111, 118, 122
abstract_inverted_index.using	94
abstract_inverted_index.value	85
abstract_inverted_index.which	71, 148
abstract_inverted_index.Markov	37, 67, 109
abstract_inverted_index.convex	87
abstract_inverted_index.either	154
abstract_inverted_index.etc.).	92
abstract_inverted_index.model,	39
abstract_inverted_index.model.	126
abstract_inverted_index.models	69
abstract_inverted_index.period	55
abstract_inverted_index.policy	104
abstract_inverted_index.values	79
abstract_inverted_index.zero).	57
abstract_inverted_index.arrived	151
abstract_inverted_index.compute	74
abstract_inverted_index.control	6, 12, 44, 135
abstract_inverted_index.finding	133
abstract_inverted_index.horizon	10
abstract_inverted_index.method,	89
abstract_inverted_index.methods	82
abstract_inverted_index.optimal	5, 29, 43, 75, 78, 103
abstract_inverted_index.present	1
abstract_inverted_index.results	144
abstract_inverted_index.several	81
abstract_inverted_index.$\Rd$\,.	18
abstract_inverted_index.analytic	88
abstract_inverted_index.approach	131
abstract_inverted_index.designed	31, 105
abstract_inverted_index.existing	143
abstract_inverted_index.infinite	9
abstract_inverted_index.learning	91
abstract_inverted_index.methods.	159
abstract_inverted_index.policies	7, 30, 76, 136
abstract_inverted_index.problems	13
abstract_inverted_index.provides	128
abstract_inverted_index.sampling	54
abstract_inverted_index.suitable	64
abstract_inverted_index.construct	63
abstract_inverted_index.diffusion	50, 117
abstract_inverted_index.objective	22
abstract_inverted_index.practical	130
abstract_inverted_index.problems,	45
abstract_inverted_index.approaches	56, 123
abstract_inverted_index.complement	142
abstract_inverted_index.conditions	141
abstract_inverted_index.controlled	15, 36, 49, 66, 116, 138
abstract_inverted_index.diffusions	16
abstract_inverted_index.iteration,	86
abstract_inverted_index.optimality	27
abstract_inverted_index.technique,	98
abstract_inverted_index.convergence	97
abstract_inverted_index.diffusions.	139
abstract_inverted_index.literature,	147
abstract_inverted_index.particular,	20
abstract_inverted_index.near-optimal	113, 134
abstract_inverted_index.approximating	34
abstract_inverted_index.approximation	3
abstract_inverted_index.discrete-time	2, 35, 65, 108, 121
abstract_inverted_index.probabilistic	155
abstract_inverted_index.reinforcement	90
abstract_inverted_index.continuous-time	125
abstract_inverted_index.discounted/ergodic	11, 42
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	2
citation_normalized_percentile