Quantum Markov Decision Processes: General Theory, Approximations, and Classes of Policies Article Swipe

PDF

Naci Saldı , Sina Sanjari , Serdar Yüksel ·

YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2402.14649

In this paper, the aim is to develop a quantum counterpart to classical Markov decision processes (MDPs). Firstly, we provide a very general formulation of quantum MDPs with state and action spaces in the quantum domain, quantum transitions, and cost functions. Once we formulate the quantum MDP (q-MDP), our focus shifts to establishing the verification theorem that proves the sufficiency of Markovian quantum control policies and provides a dynamic programming principle. Subsequently, a comparison is drawn between our q-MDP model and previously established quantum MDP models (referred to as QOMDPs) found in the literature. Furthermore, approximations of q-MDPs are obtained via finite-action models, which can be formulated as QOMDPs. Finally, classes of open-loop and classical-state-preserving closed-loop policies for q-MDPs are introduced, along with structural results for these policies. In summary, we present a novel quantum MDP model aiming to introduce a new framework, algorithms, and future research avenues. We hope that our approach will pave the way for a new research direction in discrete-time quantum control.

Related Topics

Concepts

Markov decision process Markov process Decision theory Markov chain Mathematical economics Approximations of π Mathematics Applied mathematics Statistical physics Computer science Physics Statistics

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2402.14649
PDF: https://arxiv.org/pdf/2402.14649
OA Status: green
Related Works: 10
OpenAlex ID: https://openalex.org/W4392120993

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4392120993

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2402.14649

Digital Object Identifier
Title: Quantum Markov Decision Processes: General Theory, Approximations, and Classes of Policies

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2024

Year of publication
Publication date: 2024-02-22

Full publication date if available
Authors: Naci Saldı, Sina Sanjari, Serdar Yüksel

List of authors in order
Landing page: https://arxiv.org/abs/2402.14649

Publisher landing page
PDF URL: https://arxiv.org/pdf/2402.14649

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2402.14649

Direct OA link when available
Concepts: Markov decision process, Markov process, Decision theory, Markov chain, Mathematical economics, Approximations of π, Mathematics, Applied mathematics, Statistical physics, Computer science, Physics, Statistics

Top concepts (fields/topics) attached by OpenAlex
Cited by: 0

Total citation count in OpenAlex
Related works (count): 10

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W4392120993
doi	https://doi.org/10.48550/arxiv.2402.14649
ids.doi	https://doi.org/10.48550/arxiv.2402.14649
ids.openalex	https://openalex.org/W4392120993
fwci
type	preprint
title	Quantum Markov Decision Processes: General Theory, Approximations, and Classes of Policies
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T10622
topics[0].field.id	https://openalex.org/fields/31
topics[0].field.display_name	Physics and Astronomy
topics[0].score	0.31779998540878296
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/3107
topics[0].subfield.display_name	Atomic and Molecular Physics, and Optics
topics[0].display_name	Quantum Mechanics and Applications
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C106189395
concepts[0].level	3
concepts[0].score	0.6089679002761841
concepts[0].wikidata	https://www.wikidata.org/wiki/Q176789
concepts[0].display_name	Markov decision process
concepts[1].id	https://openalex.org/C159886148
concepts[1].level	2
concepts[1].score	0.531269907951355
concepts[1].wikidata	https://www.wikidata.org/wiki/Q176645
concepts[1].display_name	Markov process
concepts[2].id	https://openalex.org/C28901747
concepts[2].level	2
concepts[2].score	0.5309094190597534
concepts[2].wikidata	https://www.wikidata.org/wiki/Q177571
concepts[2].display_name	Decision theory
concepts[3].id	https://openalex.org/C98763669
concepts[3].level	2
concepts[3].score	0.49886274337768555
concepts[3].wikidata	https://www.wikidata.org/wiki/Q176645
concepts[3].display_name	Markov chain
concepts[4].id	https://openalex.org/C144237770
concepts[4].level	1
concepts[4].score	0.4911041259765625
concepts[4].wikidata	https://www.wikidata.org/wiki/Q747534
concepts[4].display_name	Mathematical economics
concepts[5].id	https://openalex.org/C193386753
concepts[5].level	2
concepts[5].score	0.42754748463630676
concepts[5].wikidata	https://www.wikidata.org/wiki/Q1130396
concepts[5].display_name	Approximations of π
concepts[6].id	https://openalex.org/C33923547
concepts[6].level	0
concepts[6].score	0.4063985347747803
concepts[6].wikidata	https://www.wikidata.org/wiki/Q395
concepts[6].display_name	Mathematics
concepts[7].id	https://openalex.org/C28826006
concepts[7].level	1
concepts[7].score	0.3811037242412567
concepts[7].wikidata	https://www.wikidata.org/wiki/Q33521
concepts[7].display_name	Applied mathematics
concepts[8].id	https://openalex.org/C121864883
concepts[8].level	1
concepts[8].score	0.3695583641529083
concepts[8].wikidata	https://www.wikidata.org/wiki/Q677916
concepts[8].display_name	Statistical physics
concepts[9].id	https://openalex.org/C41008148
concepts[9].level	0
concepts[9].score	0.369132936000824
concepts[9].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[9].display_name	Computer science
concepts[10].id	https://openalex.org/C121332964
concepts[10].level	0
concepts[10].score	0.13694912195205688
concepts[10].wikidata	https://www.wikidata.org/wiki/Q413
concepts[10].display_name	Physics
concepts[11].id	https://openalex.org/C105795698
concepts[11].level	1
concepts[11].score	0.10936683416366577
concepts[11].wikidata	https://www.wikidata.org/wiki/Q12483
concepts[11].display_name	Statistics
keywords[0].id	https://openalex.org/keywords/markov-decision-process
keywords[0].score	0.6089679002761841
keywords[0].display_name	Markov decision process
keywords[1].id	https://openalex.org/keywords/markov-process
keywords[1].score	0.531269907951355
keywords[1].display_name	Markov process
keywords[2].id	https://openalex.org/keywords/decision-theory
keywords[2].score	0.5309094190597534
keywords[2].display_name	Decision theory
keywords[3].id	https://openalex.org/keywords/markov-chain
keywords[3].score	0.49886274337768555
keywords[3].display_name	Markov chain
keywords[4].id	https://openalex.org/keywords/mathematical-economics
keywords[4].score	0.4911041259765625
keywords[4].display_name	Mathematical economics
keywords[5].id	https://openalex.org/keywords/approximations-of-π
keywords[5].score	0.42754748463630676
keywords[5].display_name	Approximations of π
keywords[6].id	https://openalex.org/keywords/mathematics
keywords[6].score	0.4063985347747803
keywords[6].display_name	Mathematics
keywords[7].id	https://openalex.org/keywords/applied-mathematics
keywords[7].score	0.3811037242412567
keywords[7].display_name	Applied mathematics
keywords[8].id	https://openalex.org/keywords/statistical-physics
keywords[8].score	0.3695583641529083
keywords[8].display_name	Statistical physics
keywords[9].id	https://openalex.org/keywords/computer-science
keywords[9].score	0.369132936000824
keywords[9].display_name	Computer science
keywords[10].id	https://openalex.org/keywords/physics
keywords[10].score	0.13694912195205688
keywords[10].display_name	Physics
keywords[11].id	https://openalex.org/keywords/statistics
keywords[11].score	0.10936683416366577
keywords[11].display_name	Statistics
language	en
locations[0].id	pmh:oai:arXiv.org:2402.14649
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2402.14649
locations[0].version	submittedVersion
locations[0].raw_type
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2402.14649
locations[1].id	doi:10.48550/arxiv.2402.14649
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2402.14649
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5029240602
authorships[0].author.orcid	https://orcid.org/0000-0002-2677-7366
authorships[0].author.display_name	Naci Saldı
authorships[0].author_position	first
authorships[0].raw_author_name	Saldi, Naci
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5001852382
authorships[1].author.orcid	https://orcid.org/0000-0002-6409-7994
authorships[1].author.display_name	Sina Sanjari
authorships[1].author_position	middle
authorships[1].raw_author_name	Sanjari, Sina
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5005401257
authorships[2].author.orcid	https://orcid.org/0000-0001-6099-5001
authorships[2].author.display_name	Serdar Yüksel
authorships[2].author_position	last
authorships[2].raw_author_name	Yuksel, Serdar
authorships[2].is_corresponding	False
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2402.14649
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2024-02-24T00:00:00
display_name	Quantum Markov Decision Processes: General Theory, Approximations, and Classes of Policies
has_fulltext	True
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T10622
primary_topic.field.id	https://openalex.org/fields/31
primary_topic.field.display_name	Physics and Astronomy
primary_topic.score	0.31779998540878296
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/3107
primary_topic.subfield.display_name	Atomic and Molecular Physics, and Optics
primary_topic.display_name	Quantum Mechanics and Applications
related_works	https://openalex.org/W1660242800, https://openalex.org/W2379651310, https://openalex.org/W2113019827, https://openalex.org/W1541249122, https://openalex.org/W2413828414, https://openalex.org/W2367222340, https://openalex.org/W187740018, https://openalex.org/W2162286586, https://openalex.org/W4255368532, https://openalex.org/W1605817664
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2402.14649
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2402.14649
best_oa_location.version	submittedVersion
best_oa_location.raw_type
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2402.14649
primary_location.id	pmh:oai:arXiv.org:2402.14649
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2402.14649
primary_location.version	submittedVersion
primary_location.raw_type
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2402.14649
publication_date	2024-02-22
publication_year	2024
referenced_works_count	0
abstract_inverted_index.a	8, 20, 67, 72, 132, 140, 158
abstract_inverted_index.In	0, 128
abstract_inverted_index.We	148
abstract_inverted_index.as	88, 107
abstract_inverted_index.be	105
abstract_inverted_index.in	32, 91, 162
abstract_inverted_index.is	5, 74
abstract_inverted_index.of	24, 60, 96, 111
abstract_inverted_index.to	6, 11, 51, 87, 138
abstract_inverted_index.we	18, 42, 130
abstract_inverted_index.MDP	46, 84, 135
abstract_inverted_index.aim	4
abstract_inverted_index.and	29, 38, 65, 80, 113, 144
abstract_inverted_index.are	98, 119
abstract_inverted_index.can	104
abstract_inverted_index.for	117, 125, 157
abstract_inverted_index.new	141, 159
abstract_inverted_index.our	48, 77, 151
abstract_inverted_index.the	3, 33, 44, 53, 58, 92, 155
abstract_inverted_index.via	100
abstract_inverted_index.way	156
abstract_inverted_index.MDPs	26
abstract_inverted_index.Once	41
abstract_inverted_index.cost	39
abstract_inverted_index.hope	149
abstract_inverted_index.pave	154
abstract_inverted_index.that	56, 150
abstract_inverted_index.this	1
abstract_inverted_index.very	21
abstract_inverted_index.will	153
abstract_inverted_index.with	27, 122
abstract_inverted_index.along	121
abstract_inverted_index.drawn	75
abstract_inverted_index.focus	49
abstract_inverted_index.found	90
abstract_inverted_index.model	79, 136
abstract_inverted_index.novel	133
abstract_inverted_index.q-MDP	78
abstract_inverted_index.state	28
abstract_inverted_index.these	126
abstract_inverted_index.which	103
abstract_inverted_index.Markov	13
abstract_inverted_index.action	30
abstract_inverted_index.aiming	137
abstract_inverted_index.future	145
abstract_inverted_index.models	85
abstract_inverted_index.paper,	2
abstract_inverted_index.proves	57
abstract_inverted_index.q-MDPs	97, 118
abstract_inverted_index.shifts	50
abstract_inverted_index.spaces	31
abstract_inverted_index.(MDPs).	16
abstract_inverted_index.QOMDPs)	89
abstract_inverted_index.QOMDPs.	108
abstract_inverted_index.between	76
abstract_inverted_index.classes	110
abstract_inverted_index.control	63
abstract_inverted_index.develop	7
abstract_inverted_index.domain,	35
abstract_inverted_index.dynamic	68
abstract_inverted_index.general	22
abstract_inverted_index.models,	102
abstract_inverted_index.present	131
abstract_inverted_index.provide	19
abstract_inverted_index.quantum	9, 25, 34, 36, 45, 62, 83, 134, 164
abstract_inverted_index.results	124
abstract_inverted_index.theorem	55
abstract_inverted_index.(q-MDP),	47
abstract_inverted_index.Finally,	109
abstract_inverted_index.Firstly,	17
abstract_inverted_index.approach	152
abstract_inverted_index.avenues.	147
abstract_inverted_index.control.	165
abstract_inverted_index.decision	14
abstract_inverted_index.obtained	99
abstract_inverted_index.policies	64, 116
abstract_inverted_index.provides	66
abstract_inverted_index.research	146, 160
abstract_inverted_index.summary,	129
abstract_inverted_index.(referred	86
abstract_inverted_index.Markovian	61
abstract_inverted_index.classical	12
abstract_inverted_index.direction	161
abstract_inverted_index.formulate	43
abstract_inverted_index.introduce	139
abstract_inverted_index.open-loop	112
abstract_inverted_index.policies.	127
abstract_inverted_index.processes	15
abstract_inverted_index.comparison	73
abstract_inverted_index.formulated	106
abstract_inverted_index.framework,	142
abstract_inverted_index.functions.	40
abstract_inverted_index.previously	81
abstract_inverted_index.principle.	70
abstract_inverted_index.structural	123
abstract_inverted_index.algorithms,	143
abstract_inverted_index.closed-loop	115
abstract_inverted_index.counterpart	10
abstract_inverted_index.established	82
abstract_inverted_index.formulation	23
abstract_inverted_index.introduced,	120
abstract_inverted_index.literature.	93
abstract_inverted_index.programming	69
abstract_inverted_index.sufficiency	59
abstract_inverted_index.Furthermore,	94
abstract_inverted_index.establishing	52
abstract_inverted_index.transitions,	37
abstract_inverted_index.verification	54
abstract_inverted_index.Subsequently,	71
abstract_inverted_index.discrete-time	163
abstract_inverted_index.finite-action	101
abstract_inverted_index.approximations	95
abstract_inverted_index.classical-state-preserving	114
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	3
citation_normalized_percentile