AI ALIGNMENT FORUM
AF

Acausal Trade
Personal Blog

1

Acausal trade: full decision algorithms

by Stuart_Armstrong
15th May 2017
2 min read
1

1

Acausal Trade
Personal Blog
Acausal trade: full decision algorithms
7Johannes Treutlein
New Comment
1 comment, sorted by
top scoring
Click to highlight new comments since: Today at 6:27 AM
[-]Johannes Treutlein2y70

Fixed links to all the posts in the sequence:

  1. Acausal trade: Introduction
  2. Acausal trade: double decrease
  3. Acausal trade: universal utility, or selling non-existence insurance too late
  4. Acausal trade: full decision algorithms
  5. Acausal trade: trade barriers
  6. Acausal trade: different utilities, different trades
  7. Acausal trade: being unusual
  8. Acausal trade: conclusion: theory vs practice
Reply
Moderation Log
More from Stuart_Armstrong
View more
Curated and popular this week
1Comments

The description I gave here about how to balance and negotiate between different utility functions, is a bit incomplete, as this post will show. Here I'll give more details on possible decision algorithms the agents could run.

Here, two agents A1 and A2, who maximise u1 and u2, exist with probabilities q1 and q2, not necessarily independent. The joint probability that both agents exist is r12.

Waiting souls

In this perspective, we imagine that each utility function is represented by a disembodied entity, that negotiate the terms of acausal trade before the universe begins and before any agents exist or not. This is the Rawlsian veil of ignorance, that I said we'd ignore in the original post, with justifications presented here.

Now, however, we need to consider it, as a gateway to the case we want. How would the agents balance each other's utilities?

One possibility is that the agents assign equal weight to both utilities. In that case they will be both maximising u1+u2. But this poses a continuity problem as the probability of any agent declines towards 0. So the best option seems to be to have the agents agree to maximise q1u1+q2u2.

Then, in the situation presented in the previous post, both agents would increase the other utility until the marginal cost of doing so increased to either q2/q1 (for agent A1) or q1/q2 (for agent A2).

There is no "double decrease" in this situation.

Existing souls

Here we restrict ourselves to agents that actually exist. So, if for instance r12=0 (the two agents cannot both exist) then agent A1, should they exist, will have no compunction about not maximising u2 in any way.

One way of modelling this is to go back to the "waiting souls" formalism, but replace ui with u′i=uiIi where Ii is the indicator variable on whether agent Ai existed at any point in the universe. Thus all utilities depend on the existence of the agent that prefers them, in order to be maximised by anyone.

There is not longer a continuity issue with u′1+u′2 when the probabilities qi tend to zero, since low qi mean that changes to u′i become smaller and smaller in expectation.

So, when maximising u′1+u′2, the agent A1 will consider that increases to u′2 have an effect that is r12/q1 times as large as increases to u2 (while increases to u1 and u′1 are identical from its perspective, since the agent exists). Thus it will increase u2 until the marginal cost of further increases is r12/q1; similarly, A2 will increase u1 until the marginal cost of further increases is r12/q2.

Setting q1=q2=q and r12=q2 reproduces the situation of this post. This acausal trade is subject to double decrease.

Alternatively, maximising q1u′1+q2u′2 means agent A1 will increase u2 till the marginal cost of doing so is r12q2/(q1)2 (and conversely for A2 to r12q1/(q2)2). This is also subject to a double decrease, and improves the relative position of those agents most likely to exist.

Advantage only

Some agents may decide to join an acausal trade network if there something to gain for them - an actual gain once they look at the agents or potential agents in the network. This will exacerbate any double decrease, because agents who would have previously been willing to maximise some mix of u1 and u2, where maximising that mix would have been against their utility, will no longer be willing to trade.

These agents therefore treat the "no trade" position as a default disagreement point.

Other options

Of course, there are many ways of reaching a trade deal, and they will give quite different results -- especially when agents that use different types of fairness criteria attempt to reach a deal. In general, any extra difficulty will decrease the size of the trading network.