Filters








17,935 Hits in 5.5 sec

AI safety via debate [article]

Geoffrey Irving, Paul Christiano, Dario Amodei
2018 arXiv   pre-print
To help address this concern, we propose training agents via self play on a zero sum debate game.  ...  In practice, whether debate works involves empirical questions about humans and the tasks we want AIs to perform, plus theoretical questions about the meaning of AI alignment.  ...  We had many useful discussions at an AI strategy retreat run by the Future of Humanity Institute in January 2018, in particular with David Manley.  ... 
arXiv:1805.00899v2 fatcat:agtqi7kaj5ff7g45cxcjenv2oi

An overview of 11 proposals for building safe advanced AI [article]

Evan Hubinger
2020 arXiv   pre-print
This paper analyzes and compares 11 different proposals for building safe advanced AI under the current machine learning paradigm, including major contenders such as iterated amplification, AI safety via  ...  debate, and recursive reward modeling.  ...  AI safety via debate with transparency tools There are many different forms of AI safety via debate, [3] but the approach we'll be considering here is as follows: 1.  ... 
arXiv:2012.07532v1 fatcat:mfcsnozm5rec7jksxvizhvz4pu

AI Risk Skepticism [article]

Roman V. Yampolskiy
2021 arXiv   pre-print
We start by classifying different types of AI Risk skepticism and analyze their root causes.  ...  In this work, we survey skepticism regarding AI risk and show parallels with other types of scientific skepticism.  ...  In addition, author would like to acknowledge his own bias, as an AI safety researcher I would benefit from flourishing of the field of AI safety.  ... 
arXiv:2105.02704v3 fatcat:ktmbfccwojby7grkjraamrpbni

Axes for Sociotechnical Inquiry in AI Research

Sarah Dean, Thomas Krendl Gilbert, Nathan Lambert, Tom Zick
2021 IEEE Transactions on Technology and Society  
The development of artificial intelligence (AI) technologies has far exceeded the investigation of their relationship with society.  ...  To date, subfields of AI research develop primarily individual views on their relationship with sociotechnics, while tools for external investigation, comparison, and cross-pollination are lacking.  ...  For an AI Safety practitioner, failure in this regard involves pervasive risk to society via numerous dangerous drones without well modeled intents.  ... 
doi:10.1109/tts.2021.3074097 fatcat:pyqsbrfxxfg5xldocfrver6tyu

AI Safety and Reproducibility: Establishing Robust Foundations for the Neuropsychology of Human Values [chapter]

Gopal P. Sarma, Nick J. Hay, Adam Safron
2018 Lecture Notes in Computer Science  
Our aim is to ensure that research underpinning the value alignment problem of artificial intelligence has been sufficiently validated to play a role in the design of AI systems.  ...  In the same survey, 48% responded that greater emphasis should be placed on minimizing the societal risks of AI, an emerging area of study known as "AI safety" [1] .  ...  A significant tension that has arisen in the AI safety community is between those researchers concerned with near-term safety concerns and those more oriented towards longer-term, superintelligence-related  ... 
doi:10.1007/978-3-319-99229-7_45 fatcat:6hrpwpoonzhxriu3shtxhhhbzy

Demanding and Designing Aligned Cognitive Architectures [article]

Koen Holtman
2021 arXiv   pre-print
With AI systems becoming more powerful and pervasive, there is increasing debate about keeping their actions aligned with the broader goals and needs of humanity.  ...  This multi-disciplinary and multi-stakeholder debate must resolve many issues, here we examine three of them.  ...  We feel that GOFAI is too often overlooked in the current debate about AI safety and alignment.  ... 
arXiv:2112.10190v1 fatcat:kvynws2cybebzefukwcmmvyiga

The European Commission report on ethics of connected and automated vehicles and the future of ethics of transportation

Filippo Santoni de Sio
2021 Ethics and Information Technology  
The recent debate on the ethics of Connected and Automated Vehicles (CAVs) presents a paradox and an opportunity.  ...  Responsible Innovation, road justice, road safety, freedom, human control, privacy, data fairness, responsibility.  ...  This problem is usually addressed from a technical point of view, via the development of more "explainable AI", which is also the focus of Recommendation 14 of the report.  ... 
doi:10.1007/s10676-021-09609-8 fatcat:3ynsgpycgfgfbhkeb52nqrnqiq

A model of pathways to artificial superintelligence catastrophe for risk and decision analysis

Anthony M. Barrett, Seth D. Baum
2016 Journal of experimental and theoretical artificial intelligence (Print)  
This paper presents a graphical model of major pathways to ASI catastrophe, focusing on ASI created via recursive self-improvement.  ...  The model uses the established risk and decision analysis modeling paradigms of fault trees and influence diagrams in order to depict combinations of events and conditions that could lead to AI catastrophe  ...  There is some debate about whether it could ever be physically possible for ASI to be created via recursive self-improvement. There is also debate on whether humans are capable of creating such ASI.  ... 
doi:10.1080/0952813x.2016.1186228 fatcat:fmgxig7o5zbc7dvbwqbyk5hojy

Artificial Intelligence Safety Engineering: Why Machine Ethics Is a Wrong Approach [chapter]

Roman V. Yampolskiy
2013 Studies in Applied Philosophy, Epistemology and Rational Ethics  
Instead we propose a new science of safety engineering for intelligent artificial agents.  ...  Unfortunately the perceived abundance of research in intelligent machine safety is misleading.  ...  Grand Challenge As the grand challenge of AI safety engineering, we propose the problem of developing safety mechanisms for self-improving systems [23] .  ... 
doi:10.1007/978-3-642-31674-6_29 dblp:series/sapere/Yampolskiy13 fatcat:wwfqmwv34rh7tawdbdeswdqzmm

Safely Crowd-Sourcing Critical Mass for a Self-improving Human-Level Learner/"Seed AI" [chapter]

Mark R. Waser
2013 Advances in Intelligent Systems and Computing  
We believe that a moral, self-improving learner ("seed AI") can be created today via a safe "sousveillance" crowd-sourcing process and propose a plan by which this can be done.  ...  Artificial Intelligence (AI), the "science and engineering of intelligent machines", still has yet to create even a simple "Advice Taker" (McCarthy 1959) .  ...  We will draw on the human example while remembering that many aspects and details of the human implementation of learning are clearly contra-indicated for efficiency or safety reasons and many common debates  ... 
doi:10.1007/978-3-642-34274-5_58 fatcat:aqewc4r2v5hwrhyzwcqtcy42a4

Unethical Research: How to Create a Malevolent Artificial Intelligence [article]

Federico Pistono, Roman V. Yampolskiy
2016 arXiv   pre-print
In the blooming domain of AI Safety Engineering, hundreds of papers have been published on different proposals geared at the creation of a safe machine, yet nothing, to our knowledge, has been published  ...  Availability of such information would be of great value particularly to computer scientists, mathematicians, and others who have an interest in AI safety, and who are attempting to avoid the spontaneous  ...  Acknowledgements Roman Yampolskiy expresses appreciation to Elon Musk and FLI for partially funding his work via project grant: "Evaluation of Safe Development Pathways for Artificial Superintelligence  ... 
arXiv:1605.02817v2 fatcat:56xkqavhung5zlqxncw2jfbn2y

Autonomous Vehicles and Embedded Artificial Intelligence: The Challenges of Framing Machine Driving Decisions

Martin Cunneen, Martin Mullins, Finbarr Murphy
2019 Applied Artificial Intelligence  
This paper interrogates the significant shortcomings in the current framing of the debate, both in terms of safety discussions and in consideration of AI as a moral actor, and offers a number of ways forward  ...  Due to the potential safety benefits, autonomous vehicles are often presented as significant risk mitigation technologies.  ...  Such is the complexity and amplification of debates around the societal impact of AVs that there is a tendency to pare back the debate for more general consumption.  ... 
doi:10.1080/08839514.2019.1600301 fatcat:nnft43q7cvhmno52t3k3f274wy

Transdisciplinary AI Observatory—Retrospective Analyses and Future-Oriented Contradistinctions

Nadisha-Marie Aliman, Leon Kester, Roman Yampolskiy
2021 Philosophies  
Building on these AI observatory tools, we present near-term transdisciplinary guidelines for AI safety.  ...  In the last years, artificial intelligence (AI) safety gained international recognition in the light of heterogeneous safety-critical and ethical issues that risk overshadowing the broad beneficial impacts  ...  We suggest that for a safety-relevant sense-making, AI governance may even need to stimulate debates and exchanges on the very epistemological grounding of science-before for example, future texts written  ... 
doi:10.3390/philosophies6010006 fatcat:mx4ex4hdgnfrfjdwyjmvewinga

Transdisciplinary AI Observatory – Retrospective Analyses and Future-Oriented Contradistinctions [article]

Nadisha-Marie Aliman, Leon Kester, Roman Yampolskiy
2020 arXiv   pre-print
In the last years, AI safety gained international recognition in the light of heterogeneous safety-critical and ethical issues that risk overshadowing the broad beneficial impacts of AI.  ...  Building on these AI observatory tools, we present near-term transdisciplinary guidelines for AI safety.  ...  We suggest that for a safety-relevant sense-making, AI governance may even need to stimulate debates and exchanges on the very epistemological grounding of science -before e.g. future texts written by  ... 
arXiv:2012.02592v2 fatcat:q77hwo6xtfcczfwosotfpnvukq

Artificial intelligence and robotisation in the EU - should we change OHS law?

Maciej Jarota
2021 Journal of Occupational Medicine and Toxicology  
, and debate.  ...  In order to structure European Union legislation on health and safety at work, these changes could be defined in a single piece of legislation covering robotics and AI after detailed analysis, dialogue  ...  safety.  ... 
doi:10.1186/s12995-021-00301-7 pmid:33952297 fatcat:itqgzi2mvjc2jchozxixhbxnoy
« Previous Showing results 1 — 15 out of 17,935 results