Partially Observed Optimal Stochastic Control: Regularity, Optimality, Approximations, and Learning Article Swipe
Ali̇ Devran Kara
,
Serdar Yüksel
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2412.06735
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2412.06735
In this review/tutorial article, we present recent progress on optimal control of partially observed Markov Decision Processes (POMDPs). We first present regularity and continuity conditions for POMDPs and their belief-MDP reductions, where these constitute weak Feller and Wasserstein regularity and controlled filter stability. These are then utilized to arrive at existence results on optimal policies for both discounted and average cost problems, and regularity of value functions. Then, we study rigorous approximation results involving quantization based finite model approximations as well as finite window approximations under controlled filter stability. Finally, we present several recent reinforcement learning theoretic results which rigorously establish convergence to near optimality under both criteria.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2412.06735
- https://arxiv.org/pdf/2412.06735
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4405254397
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4405254397Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2412.06735Digital Object Identifier
- Title
-
Partially Observed Optimal Stochastic Control: Regularity, Optimality, Approximations, and LearningWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-12-09Full publication date if available
- Authors
-
Ali̇ Devran Kara, Serdar YükselList of authors in order
- Landing page
-
https://arxiv.org/abs/2412.06735Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2412.06735Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2412.06735Direct OA link when available
- Concepts
-
Stochastic control, Approximations of π, Control (management), Optimal control, Mathematics, Mathematical optimization, Computer science, Applied mathematics, Mathematical economics, Artificial intelligenceTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4405254397 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2412.06735 |
| ids.doi | https://doi.org/10.48550/arxiv.2412.06735 |
| ids.openalex | https://openalex.org/W4405254397 |
| fwci | |
| type | preprint |
| title | Partially Observed Optimal Stochastic Control: Regularity, Optimality, Approximations, and Learning |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10791 |
| topics[0].field.id | https://openalex.org/fields/22 |
| topics[0].field.display_name | Engineering |
| topics[0].score | 0.9409999847412109 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/2207 |
| topics[0].subfield.display_name | Control and Systems Engineering |
| topics[0].display_name | Advanced Control Systems Optimization |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C170131372 |
| concepts[0].level | 3 |
| concepts[0].score | 0.5708996057510376 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q7617811 |
| concepts[0].display_name | Stochastic control |
| concepts[1].id | https://openalex.org/C193386753 |
| concepts[1].level | 2 |
| concepts[1].score | 0.5585671067237854 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q1130396 |
| concepts[1].display_name | Approximations of π |
| concepts[2].id | https://openalex.org/C2775924081 |
| concepts[2].level | 2 |
| concepts[2].score | 0.5364668369293213 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q55608371 |
| concepts[2].display_name | Control (management) |
| concepts[3].id | https://openalex.org/C91575142 |
| concepts[3].level | 2 |
| concepts[3].score | 0.5121243000030518 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q1971426 |
| concepts[3].display_name | Optimal control |
| concepts[4].id | https://openalex.org/C33923547 |
| concepts[4].level | 0 |
| concepts[4].score | 0.43805521726608276 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[4].display_name | Mathematics |
| concepts[5].id | https://openalex.org/C126255220 |
| concepts[5].level | 1 |
| concepts[5].score | 0.4237516522407532 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q141495 |
| concepts[5].display_name | Mathematical optimization |
| concepts[6].id | https://openalex.org/C41008148 |
| concepts[6].level | 0 |
| concepts[6].score | 0.4023389220237732 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[6].display_name | Computer science |
| concepts[7].id | https://openalex.org/C28826006 |
| concepts[7].level | 1 |
| concepts[7].score | 0.38352450728416443 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q33521 |
| concepts[7].display_name | Applied mathematics |
| concepts[8].id | https://openalex.org/C144237770 |
| concepts[8].level | 1 |
| concepts[8].score | 0.3776731491088867 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q747534 |
| concepts[8].display_name | Mathematical economics |
| concepts[9].id | https://openalex.org/C154945302 |
| concepts[9].level | 1 |
| concepts[9].score | 0.2496749460697174 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[9].display_name | Artificial intelligence |
| keywords[0].id | https://openalex.org/keywords/stochastic-control |
| keywords[0].score | 0.5708996057510376 |
| keywords[0].display_name | Stochastic control |
| keywords[1].id | https://openalex.org/keywords/approximations-of-π |
| keywords[1].score | 0.5585671067237854 |
| keywords[1].display_name | Approximations of π |
| keywords[2].id | https://openalex.org/keywords/control |
| keywords[2].score | 0.5364668369293213 |
| keywords[2].display_name | Control (management) |
| keywords[3].id | https://openalex.org/keywords/optimal-control |
| keywords[3].score | 0.5121243000030518 |
| keywords[3].display_name | Optimal control |
| keywords[4].id | https://openalex.org/keywords/mathematics |
| keywords[4].score | 0.43805521726608276 |
| keywords[4].display_name | Mathematics |
| keywords[5].id | https://openalex.org/keywords/mathematical-optimization |
| keywords[5].score | 0.4237516522407532 |
| keywords[5].display_name | Mathematical optimization |
| keywords[6].id | https://openalex.org/keywords/computer-science |
| keywords[6].score | 0.4023389220237732 |
| keywords[6].display_name | Computer science |
| keywords[7].id | https://openalex.org/keywords/applied-mathematics |
| keywords[7].score | 0.38352450728416443 |
| keywords[7].display_name | Applied mathematics |
| keywords[8].id | https://openalex.org/keywords/mathematical-economics |
| keywords[8].score | 0.3776731491088867 |
| keywords[8].display_name | Mathematical economics |
| keywords[9].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[9].score | 0.2496749460697174 |
| keywords[9].display_name | Artificial intelligence |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2412.06735 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | cc-by |
| locations[0].pdf_url | https://arxiv.org/pdf/2412.06735 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | https://openalex.org/licenses/cc-by |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2412.06735 |
| locations[1].id | doi:10.48550/arxiv.2412.06735 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | cc-by |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | https://openalex.org/licenses/cc-by |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2412.06735 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5101664274 |
| authorships[0].author.orcid | https://orcid.org/0000-0001-8119-6620 |
| authorships[0].author.display_name | Ali̇ Devran Kara |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Kara, Ali Devran |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5005401257 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-6099-5001 |
| authorships[1].author.display_name | Serdar Yüksel |
| authorships[1].author_position | last |
| authorships[1].raw_author_name | Yuksel, Serdar |
| authorships[1].is_corresponding | False |
| has_content.pdf | True |
| has_content.grobid_xml | True |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2412.06735 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2024-12-12T00:00:00 |
| display_name | Partially Observed Optimal Stochastic Control: Regularity, Optimality, Approximations, and Learning |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10791 |
| primary_topic.field.id | https://openalex.org/fields/22 |
| primary_topic.field.display_name | Engineering |
| primary_topic.score | 0.9409999847412109 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/2207 |
| primary_topic.subfield.display_name | Control and Systems Engineering |
| primary_topic.display_name | Advanced Control Systems Optimization |
| related_works | https://openalex.org/W2086351220, https://openalex.org/W2943897807, https://openalex.org/W4366198066, https://openalex.org/W3120484221, https://openalex.org/W2037349991, https://openalex.org/W3047748938, https://openalex.org/W2358522863, https://openalex.org/W4386034604, https://openalex.org/W3099285423, https://openalex.org/W278441094 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2412.06735 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | cc-by |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2412.06735 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | https://openalex.org/licenses/cc-by |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2412.06735 |
| primary_location.id | pmh:oai:arXiv.org:2412.06735 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | cc-by |
| primary_location.pdf_url | https://arxiv.org/pdf/2412.06735 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | https://openalex.org/licenses/cc-by |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2412.06735 |
| publication_date | 2024-12-09 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.In | 0 |
| abstract_inverted_index.We | 18 |
| abstract_inverted_index.as | 79, 81 |
| abstract_inverted_index.at | 49 |
| abstract_inverted_index.of | 11, 64 |
| abstract_inverted_index.on | 8, 52 |
| abstract_inverted_index.to | 47, 102 |
| abstract_inverted_index.we | 4, 68, 90 |
| abstract_inverted_index.and | 22, 27, 36, 39, 58, 62 |
| abstract_inverted_index.are | 44 |
| abstract_inverted_index.for | 25, 55 |
| abstract_inverted_index.both | 56, 106 |
| abstract_inverted_index.cost | 60 |
| abstract_inverted_index.near | 103 |
| abstract_inverted_index.then | 45 |
| abstract_inverted_index.this | 1 |
| abstract_inverted_index.weak | 34 |
| abstract_inverted_index.well | 80 |
| abstract_inverted_index.Then, | 67 |
| abstract_inverted_index.These | 43 |
| abstract_inverted_index.based | 75 |
| abstract_inverted_index.first | 19 |
| abstract_inverted_index.model | 77 |
| abstract_inverted_index.study | 69 |
| abstract_inverted_index.their | 28 |
| abstract_inverted_index.these | 32 |
| abstract_inverted_index.under | 85, 105 |
| abstract_inverted_index.value | 65 |
| abstract_inverted_index.where | 31 |
| abstract_inverted_index.which | 98 |
| abstract_inverted_index.Feller | 35 |
| abstract_inverted_index.Markov | 14 |
| abstract_inverted_index.POMDPs | 26 |
| abstract_inverted_index.arrive | 48 |
| abstract_inverted_index.filter | 41, 87 |
| abstract_inverted_index.finite | 76, 82 |
| abstract_inverted_index.recent | 6, 93 |
| abstract_inverted_index.window | 83 |
| abstract_inverted_index.average | 59 |
| abstract_inverted_index.control | 10 |
| abstract_inverted_index.optimal | 9, 53 |
| abstract_inverted_index.present | 5, 20, 91 |
| abstract_inverted_index.results | 51, 72, 97 |
| abstract_inverted_index.several | 92 |
| abstract_inverted_index.Decision | 15 |
| abstract_inverted_index.Finally, | 89 |
| abstract_inverted_index.article, | 3 |
| abstract_inverted_index.learning | 95 |
| abstract_inverted_index.observed | 13 |
| abstract_inverted_index.policies | 54 |
| abstract_inverted_index.progress | 7 |
| abstract_inverted_index.rigorous | 70 |
| abstract_inverted_index.utilized | 46 |
| abstract_inverted_index.(POMDPs). | 17 |
| abstract_inverted_index.Processes | 16 |
| abstract_inverted_index.criteria. | 107 |
| abstract_inverted_index.establish | 100 |
| abstract_inverted_index.existence | 50 |
| abstract_inverted_index.involving | 73 |
| abstract_inverted_index.partially | 12 |
| abstract_inverted_index.problems, | 61 |
| abstract_inverted_index.theoretic | 96 |
| abstract_inverted_index.belief-MDP | 29 |
| abstract_inverted_index.conditions | 24 |
| abstract_inverted_index.constitute | 33 |
| abstract_inverted_index.continuity | 23 |
| abstract_inverted_index.controlled | 40, 86 |
| abstract_inverted_index.discounted | 57 |
| abstract_inverted_index.functions. | 66 |
| abstract_inverted_index.optimality | 104 |
| abstract_inverted_index.regularity | 21, 38, 63 |
| abstract_inverted_index.rigorously | 99 |
| abstract_inverted_index.stability. | 42, 88 |
| abstract_inverted_index.Wasserstein | 37 |
| abstract_inverted_index.convergence | 101 |
| abstract_inverted_index.reductions, | 30 |
| abstract_inverted_index.quantization | 74 |
| abstract_inverted_index.approximation | 71 |
| abstract_inverted_index.reinforcement | 94 |
| abstract_inverted_index.approximations | 78, 84 |
| abstract_inverted_index.review/tutorial | 2 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 2 |
| citation_normalized_percentile |