Summary:

This post outlines how a view we call subjective naturalism^[1] poses challenges to classical Savage-style decision theory. Subjective naturalism requires (i) richness (the ability to represent all propositions the agent can entertain, including self-referential ones) and (ii) austerity (excluding events the agent deems impossible). It is one way of making precise certain requirements of embedded agency. We then present the Jeffrey–Bolker (JB) framework, which better accommodates an agent’s self-model and avoids forcing her to consider things she takes to be impossible.^[2]

1. Subjective Naturalism: Richness & Austerity

A naturalistic perspective treats an agent as part of the physical world—just another system subject to the same laws. Among other constraints, we think this means:

1. Richness: The model must include all the propositions the agent can meaningfully consider, including those about herself. If the agent can form a proposition “I will do X”, then that belongs in the space of propositions over which she has beliefs and (where appropriate) desirabilities.
2. Austerity: The model should only include events the agent thinks are genuinely possible. If she is certain something cannot happen, the theory shouldn’t force her to rank or measure preferences for that scenario. (Formally, we can regard zero-probability events as excluded from the relevant algebra.)

A decision-theoretic framework that meets both conditions is subjectively naturalist: it reflects the agent’s own worldview fully (richness) but doesn’t outstrip that worldview (austerity).

1.1. Framework vs. Action-Guiding Rule

In the literature, “decision theory” can refer to (at least) two different kinds of things:

1. A conceptual or mathematical framework for representing an agent’s beliefs, desires, and preferences. These frameworks show how one might encode uncertainty, evaluate outcomes, or measure utility—but they don’t necessarily dictate a unique rule for which choice to make in any given scenario.
2. A decision rule or algorithmic procedure for taking an action—e.g., “choose the act that maximizes expected utility according to these probabilities,” or “choose so as to maximize causal expected utility.”

When people say “vNM decision theory”, “Savage’s decision theory”, or “Jeffrey–Bolker,” they sometimes shift back and forth between framework-level discussion (how to model an agent’s preferences and degrees of belief) and rule-level discussion (which choice is rational to make, given that model).

• Savage’s framework is often used together with a rule: “pick the act whose expected utility is highest, where the agent’s beliefs are defined over external states only.”^[3]
• Jeffrey–Bolker is likewise a framework: it posits a single probability–desirability space for all propositions (including ones about the agent). But within it, one can still adopt different decision rules. For instance, Evidential Decision Theory (EDT) says “choose the proposition A that maximizes desirability”;^[4] Causal Decision Theory (CDT) modifies how probabilities are updated under hypothetical ‘interventions’ on A, and recommends an agent maximize this kind of causal expected utility.^[5]

Recognizing this framework vs. decision rule distinction helps clarify how a single formalism (like Jeffrey–Bolker) can encode multiple theories of choice. We can thus separate the mathematical modeling of beliefs and utilities (“framework”) from the question of which choice is prescribed (“rule”). Here we are focusing on the choice of framework.

2. Savage’s Framework

Leonard Savage’s classic theory in The Foundations of Statistics (1954) organizes decision problems via:

• A set of states $S$, fully describing matters outside the agent’s control.
• A set of consequences $C$.
• A set of acts $A\subseteq C^S$, where each act is a function $f:S\to C$.

The agent’s preference ordering ⪰ is defined over all possible acts $f$. Under certain axioms—particularly the Sure-Thing Principle—Savage proves there exists a unique probability measure $P$ on $S$^[6] and a bounded utility function $u$ on $C$ such that:

2.1. The Rectangular Field and Its Problems

A crucial step is the Rectangular Field Assumption: the agent’s preference ordering must extend over every function from $S$ to $C$. This often means considering acts like “if it rains, then a nuclear war occurs; if it does not rain, then aliens will attack,” even if the agent herself thinks that’s physically absurd.

With this in hand, we can see that from a subjectively naturalist standpoint Savage doesn't do well:

• Richness failure: The agent does not assign probabilities to her own acts. Acts and states are disjoint sets, so “I choose act $f$” is never a proposition in $S$. It is a very dualistic framework in the sense that propositions about the world live in a separate domain from propositions about the agent's acts, which are themselves off-limits from her credences.
• Austerity failure: By covering all mappings $f$ from $S$ to $C$ Savage’s framework forces the agent to rank acts she judges downright impossible (e.g. “control whether or not there is a nuclear attack by flipping a coin”).

Thus, while Savage's theory is very useful for some purposes, it violates both conditions of subjective naturalism. 

3. The Jeffrey–Bolker Framework

Richard Jeffrey (The Logic of Decision, 1965) and Ethan Bolker (1967) introduced a different formal approach that addresses these worries.

3.1. Basic Setup

Instead of dividing the world into “states” and “acts,” JB theory starts with a Boolean algebra $A$.^[7] Each element $A\in A$ is a proposition the agent can meaningfully entertain. That includes not just “It rains” but also “I will pick up the pen,” “I will have credence $x$ in $Y$ at time $t$” etc. Some of the core components are:

1. A strictly positive probability measure $P$ defined over $A$.
2. A desirability (or utility) function $v$ (a signed measure) is also defined over $A$.^[8]
3. The agent has a preference ordering $⪰$ defined on $A$.^[9] Certain axioms—Averaging, Impartiality, and Continuity—ensure that $⪰$ is representable by expected utility:

         with $A⪰B$ iff $U\left(A\right)\ge U\left(B\right)$.

3.2. A Key Axiom (Informally)
 

Here we consider the key axiom for Jeffrey-Bolker, just as an example so that people can get a flavour for the framework.^[10] 

• Averaging: If $A$ and $B$ are disjoint, then
  • if $A\succ B$ then $A\succ A\vee B\succ B$; and
  • if $A⪰B$ and $B⪰A$ (written $A\approx B$) then $A\approx A\vee B\approx B$.^[11]

When averaging, plus another axiom (Impartiality^[12]), and some structual/continuity conditions hold, a representation theorem (due to Bolker) shows that preference is captured by a unique–up-to–transformation probability $P$ and a signed measure $v$, giving an expected utility structure.

3.3. Richness in Jeffrey-Bolker

In JB, the agent can have a proposition “I choose $X$” right in $A$. That means the agent’s beliefs about herself—probabilities about her actions or mental states—fit seamlessly into her overall probability space. No artificial separation between “states” and “acts.”

Hence, richness is greatly improved: all relevant propositions live together in $A$.

3.4. Austerity in Jeffrey-Bolker

Because $A$ is just a Boolean algebra closed under logical operations, the agent isn’t forced to include bizarre “causal” connections she rules out as physically impossible. Bolker puts it bluntly:

“The ‘Bolker objection’ (which could just as well have been named the Jeffrey objection) says that it is unreasonable to ask a decision maker to express preferences about events or lotteries he feels cannot occur.”

(Bolker 1974, p. 80)

And Jeffrey notes:

“I take it to be the principal virtue of the present theory, that it makes no use of the notion of a gamble or of any other causal notion.”

(Jeffrey 1965, p. 157)

Thus, in JB theory, you can avoid the bizarre “If it rains, nuclear war” situation simply by never admitting that object^[13] into the algebra. The algebra only includes propositions that the agent views as possible.

In this way, austerity is satisfied. The framework tracks the agent’s sense of what is possible and excludes everything else.

4. Comparison & Key Advantages

Here’s how Jeffrey–Bolker addresses typical critiques of Savage:^[14]

• No “Rectangular Field”: JB only requires closure under logical operations, not all possible functions from states to consequences.
• Agent Not Dualistic: Acts, states, consequences, and self-beliefs all appear in the same Boolean structure, allowing for probabilities assigned to propositions about the agent herself.

Hence, from the viewpoint of subjective naturalism, JB theory neatly combines:

1. Richness: The agent can represent her own actions in the same probability space that describes the rest of the world.
2. Austerity: The agent excludes events that are truly impossible or have zero probability from her perspective—no forced ranking of propositions or situations that are inconsistent with the agent's perspective.

5. Concluding Remarks

• Savage’s theory remains foundational in economics and beyond, but it segments the world into states vs. acts, fails to give probabilities to the latter, and forces an unnatural breadth of acts into the agent's judgements.
• Jeffrey–Bolker places all propositions (including those about one’s own actions) into a single algebra, with a single probability measure and a single desirability (utility) function. No more bizarre acts or situations that the agent deems impossible.

In short, JB helps us take an embedded, subjectively naturalist view of the agent—one that is both richer and more austere in a mathematically coherent way. 

To be clear, we are not claiming that JB solves all problems of embedded or naturalized agency.^[15] But we think it is a useful starting point, for the reasons above. 

1. ^{^}

   Following Daniel’s terminology.

2. ^{^}

   There are already discussions of these different frameworks on LessWrong. For example, Abram's discussion here. This post is meant to complement such existing posts, and give our take on some of the conceptual differences between different decision theory frameworks.

3. ^{^}

   Savage's framework is similar to vNM, but superior in the sense that you don't assume the agent's degrees of belief obey the probability axioms, or even that she has degrees of belief in the first place. Rather, just as how in vNM we derive an agent's utility function from her preferences over gambles, in Savage we derive an agent's utilities and probabilities from her preferences over acts.

4. ^{^}

   EDT is often associated with the Jeffrey-Bolker framework since it is what Jeffrey initially wrote down in his framework, but the framework itself admits of different decision rules. 

5. ^{^}

   Usually this is done with an imaging operator.

6. ^{^}

   Really, we have a $\sigma$-algebra on $S$ over which the probability measure is defined, and we can get integration, not just summation for expected value. To keep things more readable we'll stick with the states instead of the algebra over states, and we'll often write things down in sums instead of integration.

7. ^{^}

   This algebra is also complete and atomless:

   • An algebra is complete if every subset of the algebra has both a supremum and an infimum, relative to implication.
   • An algebra is atomless if for each member of the algebra $A$ there is some other member of the algebra $B\ne A$ other than the bottom element such that $B$ implies $A$.

   We also think that the atomlessness of the Jeffrey-Bolker algebra has a very naturalistic flavour, as partially spelled out by Kolmogorov, but we leave a more thorough discussion of this feature for a future post. 

8. ^{^}

   Technically it is not defined over the bottom element of the algebra.

9. ^{^}

   Again, minus the bottom element of the algebra.

10. ^{^}

    You can find a brief introduction to the full axioms here. We also really like this paper by Broome (1990).

11. ^{^}

    Averaging ensures that the disjunction of two propositions lies between the two propositions. For example, if you prefer visiting the Museum of Jurassic Technology to visiting the Getty Museum, then the prospect of visiting either the one or the other should be dispreferred to surely visiting the Museum of Jurassic Technology, and preferred to surely visiting the Getty Museum.

12. ^{^}

    A technical condition that effectively pinpoints when two disjoint propositions $A$ and $B$ are equiprobable, by checking how adding a third proposition $C$ to each side does (or doesn’t) alter the preference.

13. ^{^}

    In decision theory, we often call the objects of preference "prospects". Thus we can think of the point here as noting that in JB, all prospects are propositions, whereas this isn't the case in something like Savage. 

14. ^{^}

    There are others, a bit beyond the scope of this post: 

    • Partition Invariance: JB calculations for expected desirability do not depend on how you slice up propositions into partitions (unlike some nuances in Savage’s approach).
    • Measure-Theoretic Simplicity: By using a complete, atomless Boolean algebra, JB’s approach aligns with standard measure-algebra ideas in probability theory—something Kolmogorov himself advocated for avoiding the pitfalls of measure-zero events.
15. ^{^}

    For example, Daniel thinks that we need to do more than what we usually do in JB to represent how an agent comes to view herself as having control over something.