Value alignment: a formal approach Article Swipe

PDF

Carles Sierra , Nardine Osman , Pablo Noriega , Jordi Sabater-Mir , Antoni Perelló ·

YOU? · · 2021 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2110.09240

principles that should govern autonomous AI systems. It essentially states that a system's goals and behaviour should be aligned with human values. But how to ensure value alignment? In this paper we first provide a formal model to represent values through preferences and ways to compute value aggregations; i.e. preferences with respect to a group of agents and/or preferences with respect to sets of values. Value alignment is then defined, and computed, for a given norm with respect to a given value through the increase/decrease that it results in the preferences of future states of the world. We focus on norms as it is norms that govern behaviour, and as such, the alignment of a given system with a given value will be dictated by the norms the system follows.

Related Topics

Concepts

Value (mathematics) Value systems Computer science Norm (philosophy) Focus (optics) Mathematical economics Mathematics Epistemology Sociology Social science Machine learning Philosophy Optics Physics

Metadata

Type: article
Language: en
Landing Page: http://arxiv.org/abs/2110.09240
PDF: https://arxiv.org/pdf/2110.09240
OA Status: green
Cited By: 2
Related Works: 20
OpenAlex ID: https://openalex.org/W3009961753

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W3009961753

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2110.09240

Digital Object Identifier
Title: Value alignment: a formal approach

Work title
Type: article

OpenAlex work type
Language: en

Primary language
Publication year: 2021

Year of publication
Publication date: 2021-10-18

Full publication date if available
Authors: Carles Sierra, Nardine Osman, Pablo Noriega, Jordi Sabater-Mir, Antoni Perelló

List of authors in order
Landing page: https://arxiv.org/abs/2110.09240

Publisher landing page
PDF URL: https://arxiv.org/pdf/2110.09240

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2110.09240

Direct OA link when available
Concepts: Value (mathematics), Value systems, Computer science, Norm (philosophy), Focus (optics), Mathematical economics, Mathematics, Epistemology, Sociology, Social science, Machine learning, Philosophy, Optics, Physics

Top concepts (fields/topics) attached by OpenAlex
Cited by: 2

Total citation count in OpenAlex
Citations by year (recent): 2021: 1, 2020: 1

Per-year citation counts (last 5 years)
Related works (count): 20

Other works algorithmically related by OpenAlex

Full payload

id	https://openalex.org/W3009961753
doi	https://doi.org/10.48550/arxiv.2110.09240
ids.doi	https://doi.org/10.48550/arxiv.2110.09240
ids.mag	3009961753
ids.openalex	https://openalex.org/W3009961753
fwci	0.14110358
type	article
title	Value alignment: a formal approach
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
topics[0].id	https://openalex.org/T11010
topics[0].field.id	https://openalex.org/fields/17
topics[0].field.display_name	Computer Science
topics[0].score	0.9793999791145325
topics[0].domain.id	https://openalex.org/domains/3
topics[0].domain.display_name	Physical Sciences
topics[0].subfield.id	https://openalex.org/subfields/1702
topics[0].subfield.display_name	Artificial Intelligence
topics[0].display_name	Logic, Reasoning, and Knowledge
is_xpac	False
apc_list
apc_paid
concepts[0].id	https://openalex.org/C2776291640
concepts[0].level	2
concepts[0].score	0.6615750789642334
concepts[0].wikidata	https://www.wikidata.org/wiki/Q2912517
concepts[0].display_name	Value (mathematics)
concepts[1].id	https://openalex.org/C3019094977
concepts[1].level	2
concepts[1].score	0.49455639719963074
concepts[1].wikidata	https://www.wikidata.org/wiki/Q194112
concepts[1].display_name	Value systems
concepts[2].id	https://openalex.org/C41008148
concepts[2].level	0
concepts[2].score	0.4699699282646179
concepts[2].wikidata	https://www.wikidata.org/wiki/Q21198
concepts[2].display_name	Computer science
concepts[3].id	https://openalex.org/C191795146
concepts[3].level	2
concepts[3].score	0.46153318881988525
concepts[3].wikidata	https://www.wikidata.org/wiki/Q3878446
concepts[3].display_name	Norm (philosophy)
concepts[4].id	https://openalex.org/C192209626
concepts[4].level	2
concepts[4].score	0.4474150240421295
concepts[4].wikidata	https://www.wikidata.org/wiki/Q190909
concepts[4].display_name	Focus (optics)
concepts[5].id	https://openalex.org/C144237770
concepts[5].level	1
concepts[5].score	0.4075620472431183
concepts[5].wikidata	https://www.wikidata.org/wiki/Q747534
concepts[5].display_name	Mathematical economics
concepts[6].id	https://openalex.org/C33923547
concepts[6].level	0
concepts[6].score	0.35034871101379395
concepts[6].wikidata	https://www.wikidata.org/wiki/Q395
concepts[6].display_name	Mathematics
concepts[7].id	https://openalex.org/C111472728
concepts[7].level	1
concepts[7].score	0.290545254945755
concepts[7].wikidata	https://www.wikidata.org/wiki/Q9471
concepts[7].display_name	Epistemology
concepts[8].id	https://openalex.org/C144024400
concepts[8].level	0
concepts[8].score	0.21361565589904785
concepts[8].wikidata	https://www.wikidata.org/wiki/Q21201
concepts[8].display_name	Sociology
concepts[9].id	https://openalex.org/C36289849
concepts[9].level	1
concepts[9].score	0.08566996455192566
concepts[9].wikidata	https://www.wikidata.org/wiki/Q34749
concepts[9].display_name	Social science
concepts[10].id	https://openalex.org/C119857082
concepts[10].level	1
concepts[10].score	0.08154928684234619
concepts[10].wikidata	https://www.wikidata.org/wiki/Q2539
concepts[10].display_name	Machine learning
concepts[11].id	https://openalex.org/C138885662
concepts[11].level	0
concepts[11].score	0.07636094093322754
concepts[11].wikidata	https://www.wikidata.org/wiki/Q5891
concepts[11].display_name	Philosophy
concepts[12].id	https://openalex.org/C120665830
concepts[12].level	1
concepts[12].score	0.0
concepts[12].wikidata	https://www.wikidata.org/wiki/Q14620
concepts[12].display_name	Optics
concepts[13].id	https://openalex.org/C121332964
concepts[13].level	0
concepts[13].score	0.0
concepts[13].wikidata	https://www.wikidata.org/wiki/Q413
concepts[13].display_name	Physics
keywords[0].id	https://openalex.org/keywords/value
keywords[0].score	0.6615750789642334
keywords[0].display_name	Value (mathematics)
keywords[1].id	https://openalex.org/keywords/value-systems
keywords[1].score	0.49455639719963074
keywords[1].display_name	Value systems
keywords[2].id	https://openalex.org/keywords/computer-science
keywords[2].score	0.4699699282646179
keywords[2].display_name	Computer science
keywords[3].id	https://openalex.org/keywords/norm
keywords[3].score	0.46153318881988525
keywords[3].display_name	Norm (philosophy)
keywords[4].id	https://openalex.org/keywords/focus
keywords[4].score	0.4474150240421295
keywords[4].display_name	Focus (optics)
keywords[5].id	https://openalex.org/keywords/mathematical-economics
keywords[5].score	0.4075620472431183
keywords[5].display_name	Mathematical economics
keywords[6].id	https://openalex.org/keywords/mathematics
keywords[6].score	0.35034871101379395
keywords[6].display_name	Mathematics
keywords[7].id	https://openalex.org/keywords/epistemology
keywords[7].score	0.290545254945755
keywords[7].display_name	Epistemology
keywords[8].id	https://openalex.org/keywords/sociology
keywords[8].score	0.21361565589904785
keywords[8].display_name	Sociology
keywords[9].id	https://openalex.org/keywords/social-science
keywords[9].score	0.08566996455192566
keywords[9].display_name	Social science
keywords[10].id	https://openalex.org/keywords/machine-learning
keywords[10].score	0.08154928684234619
keywords[10].display_name	Machine learning
keywords[11].id	https://openalex.org/keywords/philosophy
keywords[11].score	0.07636094093322754
keywords[11].display_name	Philosophy
language	en
locations[0].id	pmh:oai:arXiv.org:2110.09240
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license
locations[0].pdf_url	https://arxiv.org/pdf/2110.09240
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2110.09240
locations[1].id	pmh:oai:digital.csic.es:10261/238919
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400616
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	False
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	DIGITAL.CSIC (Spanish National Research Council (CSIC))
locations[1].source.host_organization	https://openalex.org/I134820265
locations[1].source.host_organization_name	Consejo Superior de Investigaciones Científicas
locations[1].source.host_organization_lineage	https://openalex.org/I134820265
locations[1].license
locations[1].pdf_url	http://hdl.handle.net/10261/238919
locations[1].version	submittedVersion
locations[1].raw_type	comunicación de congreso
locations[1].license_id
locations[1].is_accepted	False
locations[1].is_published	False
locations[1].raw_source_name
locations[1].landing_page_url	http://hdl.handle.net/10261/238919
locations[2].id	doi:10.48550/arxiv.2110.09240
locations[2].is_oa	True
locations[2].source.id	https://openalex.org/S4306400194
locations[2].source.issn
locations[2].source.type	repository
locations[2].source.is_oa	True
locations[2].source.issn_l
locations[2].source.is_core	False
locations[2].source.is_in_doaj	False
locations[2].source.display_name	arXiv (Cornell University)
locations[2].source.host_organization	https://openalex.org/I205783295
locations[2].source.host_organization_name	Cornell University
locations[2].source.host_organization_lineage	https://openalex.org/I205783295
locations[2].license
locations[2].pdf_url
locations[2].version
locations[2].raw_type	article-journal
locations[2].license_id
locations[2].is_accepted	False
locations[2].is_published
locations[2].raw_source_name
locations[2].landing_page_url	https://doi.org/10.48550/arxiv.2110.09240
locations[3].id	mag:3009961753
locations[3].is_oa	False
locations[3].source
locations[3].license
locations[3].pdf_url
locations[3].version
locations[3].raw_type
locations[3].license_id
locations[3].is_accepted	False
locations[3].is_published
locations[3].raw_source_name
locations[3].landing_page_url	https://digital.csic.es/bitstream/10261/238919/1/Value%20alignment%20A%20formal%20approach.pdf
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5076261469
authorships[0].author.orcid	https://orcid.org/0000-0003-0839-6233
authorships[0].author.display_name	Carles Sierra
authorships[0].author_position	first
authorships[0].raw_author_name	Carles Sierra
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5030063242
authorships[1].author.orcid	https://orcid.org/0000-0002-2766-3475
authorships[1].author.display_name	Nardine Osman
authorships[1].author_position	middle
authorships[1].raw_author_name	Nardine Osman
authorships[1].is_corresponding	False
authorships[2].author.id	https://openalex.org/A5004237789
authorships[2].author.orcid	https://orcid.org/0000-0003-1317-2541
authorships[2].author.display_name	Pablo Noriega
authorships[2].author_position	middle
authorships[2].raw_author_name	Pablo Noriega
authorships[2].is_corresponding	False
authorships[3].author.id	https://openalex.org/A5003686253
authorships[3].author.orcid	https://orcid.org/0000-0001-6982-3572
authorships[3].author.display_name	Jordi Sabater-Mir
authorships[3].author_position	middle
authorships[3].raw_author_name	Jordi Sabater-Mir
authorships[3].is_corresponding	False
authorships[4].author.id	https://openalex.org/A5067977778
authorships[4].author.orcid
authorships[4].author.display_name	Antoni Perelló
authorships[4].author_position	last
authorships[4].raw_author_name	Antoni Perelló
authorships[4].is_corresponding	False
has_content.pdf	False
has_content.grobid_xml	False
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2110.09240
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	Value alignment: a formal approach
has_fulltext	True
is_retracted	False
updated_date	2025-11-06T06:51:31.235846
primary_topic.id	https://openalex.org/T11010
primary_topic.field.id	https://openalex.org/fields/17
primary_topic.field.display_name	Computer Science
primary_topic.score	0.9793999791145325
primary_topic.domain.id	https://openalex.org/domains/3
primary_topic.domain.display_name	Physical Sciences
primary_topic.subfield.id	https://openalex.org/subfields/1702
primary_topic.subfield.display_name	Artificial Intelligence
primary_topic.display_name	Logic, Reasoning, and Knowledge
related_works	https://openalex.org/W3205554113, https://openalex.org/W2791436589, https://openalex.org/W2775957762, https://openalex.org/W2750338308, https://openalex.org/W2304620995, https://openalex.org/W161135201, https://openalex.org/W3121336784, https://openalex.org/W2126909684, https://openalex.org/W1537363486, https://openalex.org/W2266833148, https://openalex.org/W360044869, https://openalex.org/W2597203370, https://openalex.org/W1798379796, https://openalex.org/W3185778395, https://openalex.org/W2949792401, https://openalex.org/W2598975055, https://openalex.org/W1853258856, https://openalex.org/W1906784591, https://openalex.org/W2908213509, https://openalex.org/W2163146702
cited_by_count	2
counts_by_year[0].year	2021
counts_by_year[0].cited_by_count	1
counts_by_year[1].year	2020
counts_by_year[1].cited_by_count	1
locations_count	4
best_oa_location.id	pmh:oai:arXiv.org:2110.09240
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license
best_oa_location.pdf_url	https://arxiv.org/pdf/2110.09240
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2110.09240
primary_location.id	pmh:oai:arXiv.org:2110.09240
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license
primary_location.pdf_url	https://arxiv.org/pdf/2110.09240
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2110.09240
publication_date	2021-10-18
publication_year	2021
referenced_works_count	0
abstract_inverted_index.a	11, 34, 53, 73, 79, 114, 118
abstract_inverted_index.AI	5
abstract_inverted_index.In	28
abstract_inverted_index.It	7
abstract_inverted_index.We	97
abstract_inverted_index.as	101, 109
abstract_inverted_index.be	17, 122
abstract_inverted_index.by	124
abstract_inverted_index.in	88
abstract_inverted_index.is	67, 103
abstract_inverted_index.it	86, 102
abstract_inverted_index.of	55, 63, 91, 94, 113
abstract_inverted_index.on	99
abstract_inverted_index.to	24, 37, 44, 52, 61, 78
abstract_inverted_index.we	31
abstract_inverted_index.But	22
abstract_inverted_index.and	14, 42, 70, 108
abstract_inverted_index.for	72
abstract_inverted_index.how	23
abstract_inverted_index.the	83, 89, 95, 111, 125, 127
abstract_inverted_index.i.e.	48
abstract_inverted_index.norm	75
abstract_inverted_index.sets	62
abstract_inverted_index.that	1, 10, 85, 105
abstract_inverted_index.then	68
abstract_inverted_index.this	29
abstract_inverted_index.ways	43
abstract_inverted_index.will	121
abstract_inverted_index.with	19, 50, 59, 76, 117
abstract_inverted_index.Value	65
abstract_inverted_index.first	32
abstract_inverted_index.focus	98
abstract_inverted_index.given	74, 80, 115, 119
abstract_inverted_index.goals	13
abstract_inverted_index.group	54
abstract_inverted_index.human	20
abstract_inverted_index.model	36
abstract_inverted_index.norms	100, 104, 126
abstract_inverted_index.paper	30
abstract_inverted_index.such,	110
abstract_inverted_index.value	26, 46, 81, 120
abstract_inverted_index.agents	56
abstract_inverted_index.and/or	57
abstract_inverted_index.ensure	25
abstract_inverted_index.formal	35
abstract_inverted_index.future	92
abstract_inverted_index.govern	3, 106
abstract_inverted_index.should	2, 16
abstract_inverted_index.states	9, 93
abstract_inverted_index.system	116, 128
abstract_inverted_index.values	39
abstract_inverted_index.world.	96
abstract_inverted_index.aligned	18
abstract_inverted_index.compute	45
abstract_inverted_index.provide	33
abstract_inverted_index.respect	51, 60, 77
abstract_inverted_index.results	87
abstract_inverted_index.through	40, 82
abstract_inverted_index.values.	21, 64
abstract_inverted_index.defined,	69
abstract_inverted_index.dictated	123
abstract_inverted_index.follows.	129
abstract_inverted_index.system's	12
abstract_inverted_index.systems.	6
abstract_inverted_index.alignment	66, 112
abstract_inverted_index.behaviour	15
abstract_inverted_index.computed,	71
abstract_inverted_index.represent	38
abstract_inverted_index.alignment?	27
abstract_inverted_index.autonomous	4
abstract_inverted_index.behaviour,	107
abstract_inverted_index.principles	0
abstract_inverted_index.essentially	8
abstract_inverted_index.preferences	41, 49, 58, 90
abstract_inverted_index.aggregations;	47
abstract_inverted_index.increase/decrease	84
cited_by_percentile_year.max	94
cited_by_percentile_year.min	89
countries_distinct_count	0
institutions_distinct_count	5
sustainable_development_goals[0].id	https://metadata.un.org/sdg/10
sustainable_development_goals[0].score	0.5
sustainable_development_goals[0].display_name	Reduced inequalities
citation_normalized_percentile.value	0.50597512
citation_normalized_percentile.is_in_top_1_percent	False
citation_normalized_percentile.is_in_top_10_percent	False