Richard Ngo challenged me to set a time box and write down as many of the most important features of my formal epistemology as I can in one sitting. Here goes.
Where probability distributions fail...
...to express beliefs
There is no probability distribution over that says "". That is a support condition about probability distributions, namely, ⨾⨾. Some s satisfy this condition and some do not; but there is no that expresses the range belief itself.
There is no probability distribution over that says " and are independent". This is an equation condition about probability distributions, namely, . Some s satisfy this equation and some do not; there is no that expresses the independence belief itself.
There is no probability distribution over that can express a conditional probability distribution , even though this is just as essential a part of a Bayesian reasoner's epistemic state as her prior. A conditional probability distribution is a family of probability distributions indexed by the condition variable, not a probability distribution. There is no single distribution that expresses it.
...to make safety tradeoffs
Suppose there is an unfair coin, which you know to be unfair (but not exactly how much or in which direction), and you must choose between three options: bet on Heads, bet on Tails, or bet instead on a fair d20 coming up at least 12. Exercise for the reader: Every probability distribution you could possibly believe makes choosing the d20 irrational.
The offer doesn't even need to be coming from Omega for the d20 to be a permissible choice; if the offer does come from Omega (who determined which way the unfair coin is unfair by predicting your betting choice), the d20 is arguably the uniquely rational choice, making the Bayesian view, that beliefs are probability distributions, clearly wrong.
Maximum entropy does not save you!
Beliefs, according to davidad
Definition 1. Given a state space , we define probability space as the space of all conceivable probability distributions on .
Definition 2. A belief about () is a functional
which is lower semicontinuous (meaning that ).
We interpret as the level of inconsistency between the conceivable probability distribution and one's belief .
Following Lawvere, we order by ("backwards"), and pointwise, so that all conceivable beliefs about form a thin category of beliefs . is a commutative monoidal category with .
Slogan: When one has multiple beliefs at the same time, one's overall belief is simply the sum of its parts.
All other known notions of belief fit in nicely
Bayesian beliefs
Definition 3. If one has a prior (big if), then one's prior belief is
Bayesian updating
Definition 4. If one has a belief over a hypothesis space, and the actual state of reality is the product of hypothesis space and data space , then one's prior belief about is the inverse-image functor
Definition 5. An observation is a closed subset .
Example 6. If one observes that , this is the subset .
Definition 7. The belief that corresponds to an observation is
Definition 8. If one has a conditional distribution, then one's conditional belief is
Example 9. If one has a likelihood function, then the previous definition applies with and :
Definition 10. A Bayesian reasoner with prior and likelihood function has belief state
Theorem 11. The two components of a Bayesian reasoner's initial belief state, only one of which is a probability distribution, simply sum when lifted into the belief space, forming the joint belief:
However, this would be meaningless unless we could recover the Bayesian update by also summing the observations in the belief space.
Theorem 12. Whenever the Bayesian posterior is well-defined, then
with a constant denoting the level of incompatibility of the observation with the Bayesian reasoner's initial belief state, namely the prior predictive surprisal, .
Notice that the orthodox Bayesian update's "normalization" to a full-mass posterior distribution silently subtracts away the constant — the amount of incompatibility between the observed data and the statistical model as a whole — which is Deborah Mayo's criticism of Bayesian epistemology in a nutshell.
Infra-Bayesian beliefs (Kosoy and Appel)
Definition 13 (Kosoy and Appel). A homogenous ultracontribution is a nonempty topologically-closed convex down-closed subset of subprobability space, .
Theorem 14. Homogenous ultracontributions (ordered by ) form a full subcategory of . Specifically, given , the corresponding belief is
and given a belief , the corresponding subset of subprobability space is
such that . Furthermore, is a member of iff is convex in probability space, which every is.
Definition 15 (Kosoy and Appel). A homogenous ultradistribution is a homogenous ultracontribution whose intersection with the full-mass face is nonempty ().
Of course, is also a full subcategory of , since is a full subcategory of .
MWER (Halpern and Leung)
Theorem 16. Homogenous ultradistributions are exactly isomorphic to the belief states of Halpern and Leung's MWER framework (Minimax Weighted Expected Regret).
Theorem 17. The updating process of simply adding, which is equivalent to that of Theorem 12, is also equivalent to Halpern and Leung's prescribed belief-updating procedure on the domain of their belief states.
Note: via the transform, this updating process is also consonant with the famous Multiplicative Weight Update family of algorithms (although I am not yet confident about whether e.g. AdaBoost is literally a special case of it).
Probabilistic dependency graphs (Richardson and Halpern)
Richardson and Halpern's PDGs, a common generalization of Bayes nets and factor graphs, take their semantics in functionals which are always lower semicontinuous (though Richardson does not prove this), and therefore every PDG denotes a belief in my sense. Further, the beliefs denoted by PDGs are convex, and thus immediately satisfy the criterion to be transformed into homogenous ultracontributions (since is convex, and composites of convex functions are convex). (However, typical PDGs do not denote homogenous ultradistributions.)
Credal sets (Cozman)
Definition 18 (Cozman). A credal set is a nonempty closed convex set of probability distributions, .
Theorem 19. Credal sets about (ordered by ) form a full subcategory of . Specifically, given , the corresponding belief is
and given a belief , the corresponding credal set is
Previsions (Goubault-Larrecq)
Definition 20 (Goubault-Larrecq). A gamble about is a Borel function . A prevision about is a functional such that and . An upper prevision is a prevision which is sub-additive: . A continuous upper prevision is an upper prevision which is Scott-continuous (for every directed family with least upper bound , ).
Definition 21. A coherent continuous upper prevision is a continuous upper prevision satisfying .
Theorem 22. Coherent continuous upper previsions about form a full subcategory of . Specifically, given the coherent continuous upper prevision , the corresponding belief is
and given a belief , the corresponding upper prevision is
The monad (Mio, Sarkis, and Vignudelli)
Definition 23 (Mio, Sarkis, and Vignudelli). The monad is defined on sets as the set of non-empty finitely-generated down-closed convex sets of subprobability distributions .
Mio, Sarkis, and Vignudelli prove that this monad is presented by the equational theory of semilattices equipped with finite probabilistic choice and , making in a strong sense the smallest semantic universe that can simultaneously interpret finite nondeterminism, finite probability, and partiality (and partiality is, in turn, needed to interpret either inconsistent beliefs or nonterminating probabilistic programs).
Theorem 24. ordered by inclusion is a full subcategory of , assuming is a Polish space. Specifically, this condition implies that finitely-generated convex sets are topologically closed, which makes a full subcategory of .
Richard Ngo challenged me to set a time box and write down as many of the most important features of my formal epistemology as I can in one sitting. Here goes.
Where probability distributions fail...
...to express beliefs
...to make safety tradeoffs
Beliefs, according to davidad
Definition 1. Given a state space
Definition 2. A belief about
which is lower semicontinuous (meaning that
We interpret
Slogan: When one has multiple beliefs at the same time, one's overall belief is simply the sum of its parts.
All other known notions of belief fit in nicely
Bayesian beliefs
Definition 3. If one has a prior
Bayesian updating
Definition 4. If one has a belief
Definition 5. An observation is a closed subset
Example 6. If one observes that
Definition 7. The belief that corresponds to an observation is
Definition 8. If one has a conditional distribution
Example 9. If one has a likelihood function
Definition 10. A Bayesian reasoner with prior
Theorem 11. The two components of a Bayesian reasoner's initial belief state, only one of which is a probability distribution, simply sum when lifted into the belief space, forming the joint belief:
However, this would be meaningless unless we could recover the Bayesian update by also summing the observations in the belief space.
Theorem 12. Whenever the Bayesian posterior
with
Notice that the orthodox Bayesian update's "normalization" to a full-mass posterior distribution silently subtracts away the constant
Infra-Bayesian beliefs (Kosoy and Appel)
Definition 13 (Kosoy and Appel). A homogenous ultracontribution
Theorem 14. Homogenous ultracontributions
and given a belief
such that
Definition 15 (Kosoy and Appel). A homogenous ultradistribution
Of course,
MWER (Halpern and Leung)
Theorem 16. Homogenous ultradistributions are exactly isomorphic to the belief states of Halpern and Leung's MWER framework (Minimax Weighted Expected Regret).
Theorem 17. The updating process of simply adding
Note: via the
Probabilistic dependency graphs (Richardson and Halpern)
Richardson and Halpern's PDGs, a common generalization of Bayes nets and factor graphs, take their semantics in functionals
Credal sets (Cozman)
Definition 18 (Cozman). A credal set is a nonempty closed convex set
Theorem 19. Credal sets about
and given a belief
Previsions (Goubault-Larrecq)
Definition 20 (Goubault-Larrecq). A gamble about
Definition 21. A coherent continuous upper prevision is a continuous upper prevision satisfying
Theorem 22. Coherent continuous upper previsions about
and given a belief
The
Definition 23 (Mio, Sarkis, and Vignudelli). The monad
Mio, Sarkis, and Vignudelli prove that this monad is presented by the equational theory of semilattices equipped with finite probabilistic choice and
Theorem 24.