That might work in a tiny world model with only two possible hypotheses. In a high-dimensional world model with exponentially many hypotheses, the weight on happy humans would be exponentially small.

[-]Gordon Seidoh Worley4y60

This doesn't really seem like solving symbol grounding, partially or not, so much as an argument that it's a non-problem for the purposes of value alignment.

Moderation Log

More from Stuart_Armstrong

Curated and popular this week

2Comments

Take the following AI, trained on videos of happy humans:

Since we know about AI wireheading, we know that there are at least two ways the AI could interpret its reward function^[1]: either we want it to make more happy humans (or more humans happy); call this . Or we want it to make more videos of happy humans; call this $R_{2}$ .

We would want the AI to learn to maximise $R_{1}$ , of course. But even without that, if it generates $R_{1}$ as a candidate and applies a suitable diminishing return to all its reward functions, then we will have a positive outcome - the AI may fill the universe with videos of happy humans, but it will also act to make us happy.

Thus solving value extrapolation will solve symbol grounding, at least in part.

This is a massive over-simplification of what would be needed to define "happy" or anything similar. ↩︎

AI ALIGNMENT FORUM
AF

AI ALIGNMENT FORUM
AF

12

Value extrapolation partially resolves symbol grounding

12