AI ALIGNMENT FORUM
AF

All of migueltorrescosta's Comments + Replies

The Blackwell order as a formalization of knowledge
migueltorrescosta00

Would this be a concrete example of the above:

We have two states S=0, S=1 as inputs, channel k1 given by the identity matrix, i.e. it gives us all information about the original, and k2 which loses all information about the initial states (i.e. it always returns S=1 as the output, regardless of the input ). Then k1 strictly dominates k2, however if we preprocess the inputs by mapping them both to S=1, then both channels convey no information, and as such there is no strict domination anymore. Is this so?

More generally, any k1>k2 can lose the strict domination property by a pregarbling where all information is destroyed, rendering both channels useless.

Have I missed anything?

Reply
1Alex Flint
Yes I believe everything you have said here is consistent with the way the Blackwell order is defined.
In Logical Time, All Games are Iterated Games
migueltorrescosta00

Thank you for your post abramdemski!

I failed to understand why you can't arrive at a solution for the Single-Shot game via Iterated Play without memory of the previous game. In order to clarify my ideas let me define two concepts first:

Iterated Play with memory: We repeatedly play the game knowing the results of the previous games.

Iterated Play without memory: We repeatedly play the game, while having no memory of the previous play.

The distinction is important: With memory we can at any time search all previous games and act accordingly, allowing for ... (read more)

Reply
1Abram Demski
If you have no memory, how can you learn? I recognize that you can draw a formal distinction, allowing learning without allowing the strategies being learned to depend on the previous games. But, you are still allowing the agent itself to depend on the previous games, which means that "learning" methods wich bake in more strategy will perform better. For example, a learning method could learn to always go straight in a game of chicken by checking to see whether going straight causes the other player to learn to swerve. IE, it doesn't seem like a principled distinction. Furthermore, I don't see the motivation for trying to do well in a single-shot game via iterated play. What kind of situation is it trying to model? This is discussed extensively in the paper I mentioned in the post, "If multi-agent learning is the answer, what is the question?"