This post is partially in response to Critch's boundaries sequence. My best guess is that he would agree with most of it in theory, but disagree with some of it in practice due to tradeoffs with other considerations in defining the concepts.

Boundaries and Frames

Imagine a world consisting of some atomic objects $R=\{r_0,r_1,\dots \}$. For example, you can think of the $r_i$ as physical atoms or cells in the game of life. Each object comes along with a collection of states it can be in $r_i=\{s_i^0,s_i^1,\dots \}$. 

(I am not committing now on whether or not we are thinking about our model as timeless. Maybe $r_i$ should be thought about as a cell in the game of life that passes through time, or maybe it should be thought about as a (cell, time) pair.)

There are two sets we naturally want to associate with our world. First, we have $R=\{r_0,r_1,\dots \}$, which I will call the object space. Second, we have $S={\Pi }_{r\in R}r$, which is the set of all ways to assign state to each object in $R$. I will call this the state space. Note that partitions of $R$ correspond to factorizations of $S$. 

If I want to point at (for example) an agent in $R$, I might tell you what atoms are inside that agent, and thus express $R$ in the form $R=a\bigsqcup e$, where $a$ is the set of all atoms that are in the agent and $e$ is the set of all atoms that are outside of the agent (and thus in the environment).

The agent then has its own state space $A={\Pi }_{r\in a}r$, while the environment has its own state space, $E={\prod }_{r\in e}r$. Now, to point at this agent in $S$, I can express $S$ in the form $S=A\times E$. 

Instead of specifying a list of objects in the agent, I specify the state space of the agent. Instead of thinking of the environment as the result of subtracting the agent object out of the world, I think of the environment as the result of quotienting out the agent's state space from the world's state space.

Instead of defining a Cartesian Boundary, I am defining a Cartesian Frame.

Playing on this naming scheme, in general (when not necessarily talking about agents), I will use the word boundary when talking about partitioning the object space into a disjoint inside and outside, and I will use the word frame when talking about factoring the state space into an independent inside and outside. (This naming might end up only being used within the scope of this post. I'm not sure.)

Frames are More General/Basic

I usually prefer to think in terms of frames (but not always!). While every boundary can be recast as a frame, the converse is not also true. If we view the state space as primary, then by imagining the world as a collection of objects, we are essentially factoring the state space. The frames that correspond to boundaries are basically those that carve along the joints given by the factorization into basic objects.

If the world is given to you pre-factored as a collection of objects, then it makes sense to partition those objects into larger objects and draw boundaries around them. However, I think for interesting problems, this is rarely the case. Further, even if the world is pre-factorized, that factorization could be wrong!

Also, the act of drawing a boundary feels similar to me to the act of factoring the state space into atomic objects. Both are carving out interesting features of state space. The initial factorization into atomic objects is identifying microscopic objects, while the boundaries are identifying macroscopic objects. Thus, starting from a pre-factored world feels especially bad when thinking about identifying boundaries, since it is starting with half the problem already done. (Note: I think this argument is weaker than it sounds. The factorization into objects is a different type than the drawing of the boundaries.)

Getting Past the Physical Frame

The main practical reason I think we need to work with frames is that we are simply not given a pre-factored world. If you look at the physical world, it feels like we have objects with which we can define boundaries. These atoms are the United States. These atoms are Canada. These atoms are neither. These atoms are on the boundary and are ambiguous/both/neither.

However when you look at math it becomes messier. (When you look at physics more deeply, it also becomes messy, but that is not my main point.) And I think to navigate the future, we are going to have to look at math. 

When defining the boundaries on a physical organism, we can use the factorization we get from the physical world. However when defining the boundaries of a gene or a meme or an algorithm, (or even a physical organism that is being predicted by its environment) we don't have that luxury.

I predict that as time goes, informational organisms will become more and more powerful relative to physical organisms, and we will thus want to be able to model them. Or, put another way, frames that factor the world into informational organisms will be more predictive/productive than frames that factor the world into physical organisms.

(This also further motivates frames over boundaries, as we might want to reason about multiple different frames simultaneously with no underlying factorization that multiplicatively-refines both of them. (This was a major update for me in the last year. Finite factored sets was largely about discovering the one true underlying factorization, and unfortunately I now think things won't be that simple.) )

Note: this can be thought of as the motivation for a bunch of agent foundations questions. Logical uncertainty and decision theory are about taking concepts that we understand for physical organisms, and generalizing them to apply to informational organisms.

Note: This is not a disagreement with Critch's pressure towards thinking about living organisms. A meme is a central example of an organism that is living, but not physical.

Thick Boundaries

In my previous work, and thus far in this post, I have been focusing on what I will call thin boundaries, but my model of Critch largely wants to talk about thick boundaries (for good reason). I will explain when I mean first in terms of boundaries, and then make it in terms of frames.

A boundary partitions the world into an inside and an outside. If the boundary is thin, then everything is either inside or outside, there is no ambiguous middle ground. This is the simplest kind of boundary, because the boundary isn't actually there! However, we are going to have to get past this over-simplification.

Above, I said: "I will use the word boundary when talking about partitioning the object space into a disjoint inside and outside." 

Now, I want to instead say we are approximately partitioning the object space into an inside and outside, which would be disjoint if not for the thick boundary. So the thick boundary is the intersection of the inside with the outside. 

Here is a nice visualization/example:

Imagine closed subset $a\subseteq R^2$ as the inside or agent, and the closure of the complement of $a$ will be the outside or environment, $e$. The thick boundary is the intersection $a\cap e$.

I like thinking of the thick boundary as both inside and outside, rather than neither/ambiguous, but you could define the inside/outside as the interior of my inside/outside instead.

Thick Frames

Now, lets see what happens to this concept when thinking about frames.  

Above, I said: "I will use the word frame when talking about factoring the state space into an independent inside and outside."

Now, I want to instead say that we are approximately factoring the state space into an inside and outside that would be independent if not for the thick frame. By this, I mean that the inside and outside are conditionally orthogonal given the thick frame. (recall that we are working in the state space, so "inside," "outside," and "thick frame" are properties of the world rather than physical objects, so we are the right type conditional orthogonality.)

I use the word "orthogonal" rather than "independent" because I don't want to restrict myself to a probability-based ontology, but you can just think of it as independent if you want. I am speaking loosely, and I am talking about the concepts from Finite Factored Sets, but only roughly.

Going back to our visualization, imagine that the state space $S$ of the world is actually the set of continuous functions $R^2\to R$, and our $a$ and $e$ are subsets of the domain as defined above. We can think of the state space $A$ of the agent as the set of continuous functions $a\to R$, and similarly think of the state space $E$ of the environment as the set of continuous functions $e\to R$. Finally, let $b=a\cap e$ be the boundary, and let B be the set of continuous functions  $b\to E$.

Note that $S\ne A\times E$, since the agent and environment must agree on the boundary. However we do have $S={\sum }_{f\in B}\left(A{|}_f\right)\times \left(E{|}_f\right)$, where $A{|}_f$ is the set of all function in A that restrict to $f$ on $b$, and similarly for $E{|}_f.$ (The shape of this formula should be strongly reminding you of conditional orthogonality/independence.)

The state of the boundary screens off the state of the inside from the state of the outside. When the frame is thin, the boundary is empty, and so only has one possible state, so the inside and outside are already independent. Here "independent" is referring to what is possible, not referring to probability. 

Implications

Boundaries are Crucial for Decision Theory

I think that this ontology illustrates how important boundaries are for decision theory. You can think of the main problem of decision theory as figuring out how to think of the world as a function of your actions. Factoring the world as Agent and Environment is exactly doing this. By currying, we can see environment as a function from the agent to the world. 

When thinking of boundaries as frames, now we see the definition of a boundary instead as the the definition of a factorization into agent and environment, which then gives us all our counterfactuals. 

Boundaries are Related to Time

I am not a big fan of reasoning about agents/minds/anything as embedded in a pre-existing time. (I have avoided systems embedded in linear time in basically all my thoughts since logical induction in 2016.) However I claim that this is especially important for thinking about boundaries. Boundaries are about breaking the world up into orthogonal properties, (and maintaining their orthogonality) and I basically think of orthogonality and time as the same concept (orthogonal:disjoint::before:subset).

Unfortunately this is not very workable as advice, I am mostly just making the prediction that the textbook from the future that would make us change our ontology on time would also make us change our ontology on boundaries in a similar way. But in the meantime, if you can use simple notions of time to say something interesting about boundaries, that is better than saying nothing. (Also, I tend to manically say everything is the same as everything else, so advice like this should maybe be taken with a grain of salt.)

For example, since we are thinking of the thick frame as that which screens of an agent from its environment, we might want to think of the history of the development of the agent as part of its boundary. 

Maintaining Boundaries is about Maintaining Free Will and Privacy

A physical boundary can be thought of as keeping physical objects in or out. This has the consequence of keeping the stuff on the inside independent from the stuff on the inside. If we take the physical objects out of our ontology, all that is left is the independence. Instead of keeping physical stuff inside, we are keeping information inside, and thus maintaining the inside's privacy. Instead of keeping physical stuff outside, we are stopping the outside features from controlling the inside, and thus maintaining the inside's free will.