AI ALIGNMENT FORUM
AF

Wikitags

Development phase unpredictable

Edited by Eliezer Yudkowsky last updated 28th Apr 2015

Several proposed problems in advanced safety are alleged to be difficult because they depend on some property of a mature agent that is allegedly hard to predict in advance at the time we are designing, teaching, or testing the agent. We say that such properties are allegedly 'development phase unpredictable'. For example, the Unforeseen Maximums problem arises when we can't search a rich solution space as widely as an advanced agent, making it development-phase unpredictable which real-world strategy or outcome state will maximize some formal utility function.

Parents:
AI alignment
Children:
Unforeseen maximum
1
1
Discussion0
Discussion0