Average Cost Optimality of Partially Observed MDPS: Contraction of Non-linear Filters, Optimal Solutions and Approximations Article Swipe
Yunus Emre Demirci
,
Ali̇ Devran Kara
,
Serdar Yüksel
·
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2312.14111
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2312.14111
The average cost optimality is known to be a challenging problem for partially observable stochastic control, with few results available beyond the finite state, action, and measurement setup, for which somewhat restrictive conditions are available. In this paper, we present explicit and easily testable conditions for the existence of solutions to the average cost optimality equation where the state space is compact. In particular, we present a new contraction based analysis, which is new to the literature to our knowledge, building on recent regularity results for non-linear filters. Beyond establishing existence, we also present several implications of our analysis that are new to the literature: (i) robustness to incorrect priors (ii) near optimality of policies based on quantized approximations, (iii) near optimality of policies with finite memory, and (iv) convergence in Q-learning. In addition to our main theorem, each of these represents a novel contribution for average cost criteria.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2312.14111
- https://arxiv.org/pdf/2312.14111
- OA Status
- green
- Cited By
- 1
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4390137211
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4390137211Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2312.14111Digital Object Identifier
- Title
-
Average Cost Optimality of Partially Observed MDPS: Contraction of Non-linear Filters, Optimal Solutions and ApproximationsWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2023Year of publication
- Publication date
-
2023-12-21Full publication date if available
- Authors
-
Yunus Emre Demirci, Ali̇ Devran Kara, Serdar YükselList of authors in order
- Landing page
-
https://arxiv.org/abs/2312.14111Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2312.14111Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2312.14111Direct OA link when available
- Concepts
-
Contraction (grammar), Observable, Mathematics, Mathematical optimization, Action (physics), State space, Applied mathematics, Stochastic control, Optimal control, Computer science, Statistics, Physics, Internal medicine, Quantum mechanics, MedicineTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
1Total citation count in OpenAlex
- Citations by year (recent)
-
2024: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4390137211 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2312.14111 |
| ids.doi | https://doi.org/10.48550/arxiv.2312.14111 |
| ids.openalex | https://openalex.org/W4390137211 |
| fwci | |
| type | preprint |
| title | Average Cost Optimality of Partially Observed MDPS: Contraction of Non-linear Filters, Optimal Solutions and Approximations |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10791 |
| topics[0].field.id | https://openalex.org/fields/22 |
| topics[0].field.display_name | Engineering |
| topics[0].score | 0.8896999955177307 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/2207 |
| topics[0].subfield.display_name | Control and Systems Engineering |
| topics[0].display_name | Advanced Control Systems Optimization |
| topics[1].id | https://openalex.org/T10067 |
| topics[1].field.id | https://openalex.org/fields/20 |
| topics[1].field.display_name | Economics, Econometrics and Finance |
| topics[1].score | 0.8446999788284302 |
| topics[1].domain.id | https://openalex.org/domains/2 |
| topics[1].domain.display_name | Social Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/2003 |
| topics[1].subfield.display_name | Finance |
| topics[1].display_name | Stochastic processes and financial applications |
| topics[2].id | https://openalex.org/T11520 |
| topics[2].field.id | https://openalex.org/fields/31 |
| topics[2].field.display_name | Physics and Astronomy |
| topics[2].score | 0.7914000153541565 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/3109 |
| topics[2].subfield.display_name | Statistical and Nonlinear Physics |
| topics[2].display_name | Advanced Thermodynamics and Statistical Mechanics |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C163415756 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7589434385299683 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q126473 |
| concepts[0].display_name | Contraction (grammar) |
| concepts[1].id | https://openalex.org/C32848918 |
| concepts[1].level | 2 |
| concepts[1].score | 0.7031389474868774 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q845789 |
| concepts[1].display_name | Observable |
| concepts[2].id | https://openalex.org/C33923547 |
| concepts[2].level | 0 |
| concepts[2].score | 0.6041346192359924 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[2].display_name | Mathematics |
| concepts[3].id | https://openalex.org/C126255220 |
| concepts[3].level | 1 |
| concepts[3].score | 0.5391391515731812 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q141495 |
| concepts[3].display_name | Mathematical optimization |
| concepts[4].id | https://openalex.org/C2780791683 |
| concepts[4].level | 2 |
| concepts[4].score | 0.5270894765853882 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q846785 |
| concepts[4].display_name | Action (physics) |
| concepts[5].id | https://openalex.org/C72434380 |
| concepts[5].level | 2 |
| concepts[5].score | 0.4514998495578766 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q230930 |
| concepts[5].display_name | State space |
| concepts[6].id | https://openalex.org/C28826006 |
| concepts[6].level | 1 |
| concepts[6].score | 0.4313633441925049 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q33521 |
| concepts[6].display_name | Applied mathematics |
| concepts[7].id | https://openalex.org/C170131372 |
| concepts[7].level | 3 |
| concepts[7].score | 0.413818359375 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q7617811 |
| concepts[7].display_name | Stochastic control |
| concepts[8].id | https://openalex.org/C91575142 |
| concepts[8].level | 2 |
| concepts[8].score | 0.3711678087711334 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q1971426 |
| concepts[8].display_name | Optimal control |
| concepts[9].id | https://openalex.org/C41008148 |
| concepts[9].level | 0 |
| concepts[9].score | 0.33827775716781616 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[9].display_name | Computer science |
| concepts[10].id | https://openalex.org/C105795698 |
| concepts[10].level | 1 |
| concepts[10].score | 0.06590569019317627 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q12483 |
| concepts[10].display_name | Statistics |
| concepts[11].id | https://openalex.org/C121332964 |
| concepts[11].level | 0 |
| concepts[11].score | 0.0 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q413 |
| concepts[11].display_name | Physics |
| concepts[12].id | https://openalex.org/C126322002 |
| concepts[12].level | 1 |
| concepts[12].score | 0.0 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q11180 |
| concepts[12].display_name | Internal medicine |
| concepts[13].id | https://openalex.org/C62520636 |
| concepts[13].level | 1 |
| concepts[13].score | 0.0 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q944 |
| concepts[13].display_name | Quantum mechanics |
| concepts[14].id | https://openalex.org/C71924100 |
| concepts[14].level | 0 |
| concepts[14].score | 0.0 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q11190 |
| concepts[14].display_name | Medicine |
| keywords[0].id | https://openalex.org/keywords/contraction |
| keywords[0].score | 0.7589434385299683 |
| keywords[0].display_name | Contraction (grammar) |
| keywords[1].id | https://openalex.org/keywords/observable |
| keywords[1].score | 0.7031389474868774 |
| keywords[1].display_name | Observable |
| keywords[2].id | https://openalex.org/keywords/mathematics |
| keywords[2].score | 0.6041346192359924 |
| keywords[2].display_name | Mathematics |
| keywords[3].id | https://openalex.org/keywords/mathematical-optimization |
| keywords[3].score | 0.5391391515731812 |
| keywords[3].display_name | Mathematical optimization |
| keywords[4].id | https://openalex.org/keywords/action |
| keywords[4].score | 0.5270894765853882 |
| keywords[4].display_name | Action (physics) |
| keywords[5].id | https://openalex.org/keywords/state-space |
| keywords[5].score | 0.4514998495578766 |
| keywords[5].display_name | State space |
| keywords[6].id | https://openalex.org/keywords/applied-mathematics |
| keywords[6].score | 0.4313633441925049 |
| keywords[6].display_name | Applied mathematics |
| keywords[7].id | https://openalex.org/keywords/stochastic-control |
| keywords[7].score | 0.413818359375 |
| keywords[7].display_name | Stochastic control |
| keywords[8].id | https://openalex.org/keywords/optimal-control |
| keywords[8].score | 0.3711678087711334 |
| keywords[8].display_name | Optimal control |
| keywords[9].id | https://openalex.org/keywords/computer-science |
| keywords[9].score | 0.33827775716781616 |
| keywords[9].display_name | Computer science |
| keywords[10].id | https://openalex.org/keywords/statistics |
| keywords[10].score | 0.06590569019317627 |
| keywords[10].display_name | Statistics |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2312.14111 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2312.14111 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2312.14111 |
| locations[1].id | doi:10.48550/arxiv.2312.14111 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2312.14111 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5070577846 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Yunus Emre Demirci |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Demirci, Yunus Emre |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5101664274 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-8119-6620 |
| authorships[1].author.display_name | Ali̇ Devran Kara |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Kara, Ali Devran |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5005401257 |
| authorships[2].author.orcid | https://orcid.org/0000-0001-6099-5001 |
| authorships[2].author.display_name | Serdar Yüksel |
| authorships[2].author_position | last |
| authorships[2].raw_author_name | Yüksel, Serdar |
| authorships[2].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2312.14111 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2023-12-23T00:00:00 |
| display_name | Average Cost Optimality of Partially Observed MDPS: Contraction of Non-linear Filters, Optimal Solutions and Approximations |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10791 |
| primary_topic.field.id | https://openalex.org/fields/22 |
| primary_topic.field.display_name | Engineering |
| primary_topic.score | 0.8896999955177307 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/2207 |
| primary_topic.subfield.display_name | Control and Systems Engineering |
| primary_topic.display_name | Advanced Control Systems Optimization |
| related_works | https://openalex.org/W1976452401, https://openalex.org/W2943897807, https://openalex.org/W4366198066, https://openalex.org/W3120484221, https://openalex.org/W3047748938, https://openalex.org/W2358522863, https://openalex.org/W4386034604, https://openalex.org/W278441094, https://openalex.org/W3099285423, https://openalex.org/W4381248241 |
| cited_by_count | 1 |
| counts_by_year[0].year | 2024 |
| counts_by_year[0].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2312.14111 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2312.14111 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2312.14111 |
| primary_location.id | pmh:oai:arXiv.org:2312.14111 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2312.14111 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2312.14111 |
| publication_date | 2023-12-21 |
| publication_year | 2023 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 8, 66, 142 |
| abstract_inverted_index.In | 35, 62, 132 |
| abstract_inverted_index.be | 7 |
| abstract_inverted_index.in | 130 |
| abstract_inverted_index.is | 4, 60, 72 |
| abstract_inverted_index.of | 48, 96, 113, 122, 139 |
| abstract_inverted_index.on | 81, 116 |
| abstract_inverted_index.to | 6, 50, 74, 77, 102, 107, 134 |
| abstract_inverted_index.we | 38, 64, 91 |
| abstract_inverted_index.(i) | 105 |
| abstract_inverted_index.The | 0 |
| abstract_inverted_index.and | 25, 41, 127 |
| abstract_inverted_index.are | 33, 100 |
| abstract_inverted_index.few | 17 |
| abstract_inverted_index.for | 11, 28, 45, 85, 145 |
| abstract_inverted_index.new | 67, 73, 101 |
| abstract_inverted_index.our | 78, 97, 135 |
| abstract_inverted_index.the | 21, 46, 51, 57, 75, 103 |
| abstract_inverted_index.(ii) | 110 |
| abstract_inverted_index.(iv) | 128 |
| abstract_inverted_index.also | 92 |
| abstract_inverted_index.cost | 2, 53, 147 |
| abstract_inverted_index.each | 138 |
| abstract_inverted_index.main | 136 |
| abstract_inverted_index.near | 111, 120 |
| abstract_inverted_index.that | 99 |
| abstract_inverted_index.this | 36 |
| abstract_inverted_index.with | 16, 124 |
| abstract_inverted_index.(iii) | 119 |
| abstract_inverted_index.based | 69, 115 |
| abstract_inverted_index.known | 5 |
| abstract_inverted_index.novel | 143 |
| abstract_inverted_index.space | 59 |
| abstract_inverted_index.state | 58 |
| abstract_inverted_index.these | 140 |
| abstract_inverted_index.where | 56 |
| abstract_inverted_index.which | 29, 71 |
| abstract_inverted_index.Beyond | 88 |
| abstract_inverted_index.beyond | 20 |
| abstract_inverted_index.easily | 42 |
| abstract_inverted_index.finite | 22, 125 |
| abstract_inverted_index.paper, | 37 |
| abstract_inverted_index.priors | 109 |
| abstract_inverted_index.recent | 82 |
| abstract_inverted_index.setup, | 27 |
| abstract_inverted_index.state, | 23 |
| abstract_inverted_index.action, | 24 |
| abstract_inverted_index.average | 1, 52, 146 |
| abstract_inverted_index.memory, | 126 |
| abstract_inverted_index.present | 39, 65, 93 |
| abstract_inverted_index.problem | 10 |
| abstract_inverted_index.results | 18, 84 |
| abstract_inverted_index.several | 94 |
| abstract_inverted_index.addition | 133 |
| abstract_inverted_index.analysis | 98 |
| abstract_inverted_index.building | 80 |
| abstract_inverted_index.compact. | 61 |
| abstract_inverted_index.control, | 15 |
| abstract_inverted_index.equation | 55 |
| abstract_inverted_index.explicit | 40 |
| abstract_inverted_index.filters. | 87 |
| abstract_inverted_index.policies | 114, 123 |
| abstract_inverted_index.somewhat | 30 |
| abstract_inverted_index.testable | 43 |
| abstract_inverted_index.theorem, | 137 |
| abstract_inverted_index.analysis, | 70 |
| abstract_inverted_index.available | 19 |
| abstract_inverted_index.criteria. | 148 |
| abstract_inverted_index.existence | 47 |
| abstract_inverted_index.incorrect | 108 |
| abstract_inverted_index.partially | 12 |
| abstract_inverted_index.quantized | 117 |
| abstract_inverted_index.solutions | 49 |
| abstract_inverted_index.available. | 34 |
| abstract_inverted_index.conditions | 32, 44 |
| abstract_inverted_index.existence, | 90 |
| abstract_inverted_index.knowledge, | 79 |
| abstract_inverted_index.literature | 76 |
| abstract_inverted_index.non-linear | 86 |
| abstract_inverted_index.observable | 13 |
| abstract_inverted_index.optimality | 3, 54, 112, 121 |
| abstract_inverted_index.regularity | 83 |
| abstract_inverted_index.represents | 141 |
| abstract_inverted_index.robustness | 106 |
| abstract_inverted_index.stochastic | 14 |
| abstract_inverted_index.Q-learning. | 131 |
| abstract_inverted_index.challenging | 9 |
| abstract_inverted_index.contraction | 68 |
| abstract_inverted_index.convergence | 129 |
| abstract_inverted_index.literature: | 104 |
| abstract_inverted_index.measurement | 26 |
| abstract_inverted_index.particular, | 63 |
| abstract_inverted_index.restrictive | 31 |
| abstract_inverted_index.contribution | 144 |
| abstract_inverted_index.establishing | 89 |
| abstract_inverted_index.implications | 95 |
| abstract_inverted_index.approximations, | 118 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 3 |
| citation_normalized_percentile |