Epistemic Status

Originally written in a 47-minute sprint without any sort of proof checking or review^[1]. I've edited it hence, but it's still just a rough sketch.

This should be understood as a brief high-level summary of the idea. I would hopefully refine the thesis of this piece and present it in a clearer and more coherent form at some later point.

Preamble

I claim that there's a single criterion for the validity of a logical counterfactual. That logical counterfactuals are valid if and only if they meet this criterion, and that counterfactuals that satisfy this criterion suffice for reflective reasoning in e.g. logical decision theories.

I'll state the criterion, provide motivations/justification for the criterion, and explain how it might be used in practice.

The Epistemic Criterion

A logical counterfactual is valid with respect to a given agent if and only if the counterfactual is consistent with that agent's epistemic state.

I.e., given the agent's current knowledge/belief pool, the agent considers it possible that the counterfactual is true, or the agent does not explicitly know^[2] the counterfactual to be false.

Motivations for the Epistemic Criterion

Desiderata for Logical Counterfactuals

Any compelling account of logical counterfactuals should satisfy two basic properties to be useful for any kind of reasoning.

Local Counterfactual Surgery
- Changes to the truth value of a logical proposition should have "local changes". It should affect only propositions that are in some intuitive sense directly dependent on the counterfactual, and not arbitrary other propositions.
Explosion Resistant
- It shouldn't be vulnerable to the principle of explosion. We don't want a situation where considering a logical counterfactual which has a false truth value in fact allows us to derive arbitrary false propositions.

The two desiderata are similar, but distinct. Locality isn't entailed by explosion resistance. Locality is not about not deriving false propositions, but the effect of a counterfactual being "confined" in some sense. Affecting only some beliefs, and not others.

Simple Model of Logical Systems for Counterfactual Surgery

One could naively model a logical system as a directed graph^[3]:

Each node is a valid sentence of the language of the system
The earliest children of the root node are the axioms of the system
- To make the graph a tree^[4]
- You can imagine the root node being .
  - We can hypothesise this as the axioms of the system following directly from unconditional truth.
There's an edge from one node $U$ to another node $V$ if and only if $V$ can be validly inferred from $U$ given the inference rules of the system

This is not the only way to model or reason about logical systems for counterfactuals — it wasn't the model I used when I first had the idea for the epistemic criterion — but I think graphical models serve as a natural intuition pump for thinking about logical counterfactuals (especially given their success with causal counterfactuals).

Our above model of counterfactual reasoning fails both desiderata.

Non-Locality

Unlike causal counterfactuals, logical counterfactuals do not admit local changes. Performing Pearlian esque counterfactual surgery on a single proposition in the graph would propagate to arbitrary many nodes in the graph and alter their truth values as well.

Setting $2 + 2 = 5$ in a model of Peano arithmetic would change the results of basically all addition statements (as all such sentences can be rewritten to compose with " $2 + 2$ "^[5]). Maintaining consistency of our new graph would require changing the axioms of the system itself to admit that small change^[6], and at that point you're no longer working with the same system.

Explosion Vulnerability

If one chose not to change the axioms of the system for propositions that were inconsistent with said axioms, they would fall prey to the principle of explosion; from a contradiction, you can derive anything.

Subsection Conclusions

Thus, in its current form, our graphical model of logical systems does not admit useful counterfactual surgery.

I claim that the epistemic criterion addresses both shortcomings.

Using the Epistemic Criterion

We could apply the epistemic criterion to our previous model of the system.

The map is not the territory — our model of a domain is not the domain itself — and this applies even to formal systems. We can apply the same graph formalism as before to the logical system, but instead of the graph modelling the entire logical system, it models only the agent's explicit knowledge^[7] of the logical system^[8].

For any statement that the agent doesn't explicitly know, they could have logical hypotheses (hypotheses about the truth values of valid sentences in the given logical system), and connect edges in their graph accordingly (special edges could be used for hypothetical statements, and the descendants of hypothetical statements in order not to contaminate ground truth [non hypothetical explicit knowledge]).

These hypothetical changes admit local propagation of semantic content. If I hypothesise that $3871$ is prime, I can confidently hypothesise that $11613 (3871 * 3)$ is not prime. And this kind of inference is independent of whether $3871$ is actually prime or not.

If the agent does try to factorise $3871$ and find that it is not prime, it could sever the edge connecting that node to its explicit knowledge. (This automatically propagates throughout all its descendants^[9], without adversely affecting the rest of the knowledge graph).

Modelling an agent's explicit knowledge graph of a logical system allows local counterfactual surgery. The inferences from a particular logical hypothesis would be intuitively confined to it in a certain sense. The reason "2 + 2 = 5" propagates arbitrarily far through the graph is because I know it to be false and thus its implications on other logical statements. For statements whose truth value I don't know, I wouldn't know what its implications are for arbitrary other statements^[10] (but will know direct implications on some particular statements: $[\forall x \in N (P (3871) ⟹ \neg P (3871 * x))]$ [where $P (x)$ is the proposition: " $x$ is prime".])

The counterfactual surgery thus facilitated also isn't vulnerable to the principle of explosion as the agent cannot derive arbitrary false conclusions as they do not know any counterfactual they consider to be false. Once the counterfactual is known to be false, it becomes invalid and its connection to ground truth is severed.

Thus, the epistemic criterion admits local, explosion resistant counterfactual surgery.

Such knowledge graphs aren't the only way to model logical counterfactuals, and the epistemic criterion isn't at all dependent on this particular presentation. They are merely meant to serve as an intuitive presentation of the concept^[11].

Application to Decision Theory

A particular kind of counterfactual seems especially pertinent for decision theoretic approaches. I call these "output hypothesis". Basically, counterfactuals of the form:
$┌ f (x) = y ┐$

Where the output of a given function $(f)$ on some input $(x)$ isn't explicitly known.

Decision algorithms trying to compute the agent's decision function can consider various hypotheses for the output of the function while the computation is ongoing. Until the final decision is made, the agent doesn't yet know, what the output of the function its decision algorithm embodies is.

Once the agent knows its decision all hypotheses for the output of its decision function other than its actual output become invalid.

^{^}
I have several 3k - 6K posts in my drafts that I've accumulated over the past few months. I don't seem to be able to complete any of the essays I'm working on, so I just want to write something and publish to break the pattern.
^{^}
Explicit knowledge is possessing an intensional description of an object that is "essentially isomorphic" to the extension.
In the context of computation, explicit knowledge of data is possessing the number (in some constant predefined numeral system) associated with that piece of data. (Recognise that every string can be encoded by some number, so all data [that can be represented as binary strings] can be associated with a number.)
It can also be interpreted as an intensional description of a piece of data that can be turned into an extensional representation of said data at trivial/negligible computational cost.
^{^}
Note that due to how formal systems work, the graph need not be acyclic.
^{^}
If it turns out to be acyclic; I don't actually know if such graphical models of logical systems would be acyclic. I suspect they are cyclic, but I haven't actually tried investigating it.
^{^}
And thus all arithmetic operations, as all numbers can be written as some composition of a sum or difference with " $2 + 2$ ".
^{^}
Unless the statement you were performing counterfactual surgery on was independent of the axioms.
^{^}
The agents that we concern ourselves with are not logically omniscient. Any logical facts that are not explicitly represented in the agent's knowledge graph (e.g. they require non-negligible/non-trivial computational cost to evaluate in real time) are not part of the graph, until (and unless) the required computation is actually performed.
^{^}
If needed, the graph could be made weighted, where the weights correspond to probability assignments. The weight of the edge $(U, V)$ would be the probability you assign to the proposition: " $V$ is a valid inference from $U$ " given the inference rules of the logical system.
^{^}
Their conditional truth was inferred from the primality of $3871$ ; the inferences may remain valid given the truth of the premise, but given that the agent knows the premise to be false, it no longer considers inferences from it to be true (hence severed from the agent's explicit knowledge graph).
That said, any nodes that could be inferred from other sources may remain in the graph.
^{^}
I don't know whether the logical hypothesis " $3871$ is prime" implies "2 + 2 = 5" and I can't trivially check without performing calculations that would reveal the truth value of the proposition to me and thus invalidate the counterfactual
^{^}
I thought they would make a good intuition pump given their success as models for causal counterfactuals.