This is a brief summary post of a recent paper titled "The Concept of Agent in Biology: Motivations and Meanings" by Samir Okasha (2023). 

In both Biology and AI Alignment, the concept of agency comes up a lot and appears relevant to properly understanding the respecting "territory" - while also being used in confusing, and sometimes confused, ways. 

Many more person-hours have been invested in progress in Biology compared to AI alignment, so it seems apt to keep an eyes out for any insights (as well as mistakes) from the discourse in Biology that might help improve the quality of discourse around the notion of 'agency' in Alignment.

I decided to write this summary post in order to make the information in the paper more accessible to the Alignment discourse. The paper itself is already a succinct (5 page!) summary of decades of discussions about the appropriate use of the notion of 'agency' in Philosophy of Biology. I can recommend to read it fully, though I have tried to capture all relevant ideas/concepts in this post.

Okasha himself is a well-respected Philosopher of Biology who has a range of substantive prior work on this and related topics (e.g.  Agents and Goals in Evolution (2018),  Evolution and the Levels of Selection (2006)).

0. Overview

"Biological agency" is a topic that has recently been gaining attention within Philosophy of Biology. In this paper, Okasha sets out to discuss "what exactly we mean by the term 'agent'?". 

To bring some clarity to the discourse, he:

1. Distinguishes two motivations for why scholars typically invoke notions of 'biological agency', and then goes on to 
2. Differentiate three distinct concepts of agency.

1. Two motivations

Motivation 1: 'Agency' as a way to point at what is distinctive about living organisms relative to other biological entities at different hierarchical levels (e.g. organs, tissues, species or ecosystems)

• Biological organisms seems to possess certain 'agent-like attributes' (e.g. decision-making, learning about the environment, performing actions) that other biological entities, at different hierarchical scales, do not. 
• This is sometimes motivated by scholars wanting to counterbalance or oppose "gene-centric or reductionistic views of evolution and development". 
  • "The worry that the gene-centric paradigm either 'leaves out the organism' altogether, or wrongly depicts all organismic activity as subservient to gene action, is of course familiar his was a well-known critique of Richard Dawkins’s early views and still resonates today."  (p.2)
• "The 'organism-as-agent' idea is seen by some authors as a potential corrective to gene-centrism, emphasizing as it does the power of organisms to make choices, overcome challenges, modify their environment, and shape their own fate. " (p.2)
  • The 'organism-as-agent' notion finds some empirical and theoretical support. E.g. "Sultan et al. (2021) point to a number of empirical phenomena, including phenotypic plasticity, the evolution of novelties, and niche construction, which, they claim, are easily accommodated by an agential perspective but less easily by rival perspectives on evolution and development." (p.2)

Motivation 2: Treating biological entities as if they were agents is useful when it comes to understanding how evolution has shaped their phenotype or behavior 

• 'As-if Agency' is invoked as a "heuristically valuable" way of treating biological entities for the "purpose of scientific understanding". 
  • This 'as if' treatment remains silent about whether or not "the organisms in question really do have the attributes of agency" (p. 2)
  • "Biological entities" here refers typically to individual organism, but can also be other entities like genes or groups. However, it is not typically meant to apply to treating the evolutionary process itself as an agent. (For more detail on this, see "type 1 vs type 2 agential thinking" in Okasha (2018))
• Why would this be heuristically valuable?
  • Explanations of adaptive phenomena, such as functions/functional behaviour which result from evolutionary processes, "can often be naturally couched in the intentional idiom".
    • See Dennett (1983, 2013, 2017), c.f. intentional stance). Other scholars who have defended this view include: Dawkins (1976), Okasha (2018)

Summary:  

Motivation 1 ('organisms-as-agents thesis') "says that organisms really do exhibit some or even all of the attributes of agency". Motivation 2 ('organism-as-agents heuristic') "says that it can be heuristically useful to treat organisms as if they were agents for certain intellectual purposes". 

2. Four concepts of agency

Concept 1: Minimal concept

• "The minimal concept of agency is simply that of doing something, or behaving." (p. 3)
  • Meant to highlight the distinction between things an entity does vs things that happen to it (Dretske, 1988)
  • The difference is taken to relate to whether the proximate cause of the behaviour is “internal” or “external”; e.g. 
    • "Consider the difference between a captive rat moving its paw towards a lever to get food and a scientist manipulating the rat’s paw"
• According to this definition, many entities (organisms and non-organisms) will count as agents. (E.g. "cells divide, hearts beat, mitochondria make ATP, bacteria swim, and insect colonies swarm.", p. 3)
• Examples of biological entities that are not agents according to this minimal definition include species, because the immediate cause of anything that happens to a species is external (e.g. consider the notion of "a species going extinct")

Concept 2: Intelligent Agent concept (from AI) 

• "[A]n intelligent agent is defined as any entity that senses its environment and changes its behavior in response (Russell and Norvig, 1995)" (p. 3)
•  Flexibility (or Adaptivity) is the key feature of this notion of agency. It can come in differentially sophisticated forms, from simple 'stimulus-response' to 'learning of complex behaviour over long time horizons'
  • "The simplest intelligent agent is a 'reflex agent' whose behavior depends only on its current percept; it thus implements a set of stimulus-response conditionals. More sophisticated intelligent agents can learn from experience; have a goal which they are trying to achieve; and in some cases can engage in search and forward planning in order to achieve their goal." (p. 3)
• The behaviour of such an agent is goal-directed (without stipulating "that the agent must have a mental representation of the goal")
• Examples: simple control systems such as thermostats, software agents, and robots;
  • In biology: "Virtually all organisms, from microbes to animals, will qualify as intelligent agents in this sense." (p. 3)

Concept 3: Rational Agent concept (from Economics)  

• "[A] rational agent is defined as one whose choices maximize their utility" Kreps 1988)
  • This is meant as a descriptive statement about the behaviour of a 'rational agent' rather than about their psychology (i.e. maximization is used in the 'as if' sense).
  • The simplest case of this boils down to "rational pursuit of a goal [and thus]consistency of choice" (according to the axioms of rational choice, c.f. von Neumann-Morgenstern utility theorems)
• Since the rational actor model is meant to describe behavior (rather than psychology) it's applicable to the analysis of non-human behaviour too. What counts as examples of rational agency in the biological sphere is not entirely clear, among others because it's not entirely clear what should count as "choice".
  • For one, "many organisms with nervous systems, even simple ones, are capable of making within-lifetime choices " (e.g. "butterflies choose what plant to oviposit on, birds choose who to mate with, and primates choose who to groom"). Therefor, they are "potentially rational agent[s]." 
    • "[R]esearchers have studied whether the choice behavior of rats, pigeons, and even insects satisfies the axioms of the rational actor model, such as transitivity of choice (Kagel et al. 1995; McFarland 2016)."

Concept 4: Intentional Agent concept (from Philosophy) 

• "[T]o be an agent is not just to behave but to act, where this means that the behavior is suitably caused by the agent’s psychological states—beliefs, desires, and intentions" (p. 3)
  • As such, this notion of agency is intimately related to notions of practical reasoning and means-to-end rationality (e.g. working out what course of action will lead to the desired outcome)
• Which entities posses intentional agency is debated; "Some authors believe that
  intentional agency is exclusive to Homo sapiens, others that it is found widely among vertebrates.) (p. 3)
  • However, most people would agree most biological entities do not count as intentional agents (e.g. bacteria, species, ecosystems)

3. Some final notes

This part is not longer exclusively pulling from the paper.

Reflection 1: Meta on 'motivations' (for using the notion of agency)

Okasha illustrate that there are two very distinct motivations for which philosophers of science (and others) may invoke the notion of agency. It seems useful to see just how different the motivations are, and that they imply different things about what looks like an appropriate concept/definition of agency. For example, for the 'organisms-as-agents thesis' (motivation 1), the 'intelligent agents' concepts seems most relevant. For the 'organisms-as-agents heuristic', the concepts of 'rational' or 'intentional agency' are more fitting. 

As such, in Alignment discourse too, it can be useful to make explicit the motivation for invoking the notion of agency - be that if you're using the notion in your own thinking/research, or if you're engaging with someone else's thinking/research. 

Reflection 2: Meta on 'concepts of agency'

What does it mean that there are different, partially overlapping and partially conflicting, notions of agency? Should we expect to eventually converge to 'the one true notion of agency'?

With respect to the last question, I don't know whether we eventually will. I think it's a valid quest to try to deconfuse our understanding of agency in the same way that we deconfused, say, our understanding of temperature some hundred years ago. 

For now, however, it's often useful to adopt a pragmatic perspective. Different notions of agency (such as the ones listed above) pick out different phenomena/structures in the world. They may pick out broader/narrower versions of the same phenomena; they may pick out the same phenomena highlighting different aspects of it; or they may pick out (partially or entirely) different phenomena. For practical purposes, the main thing that matters is that a) we are precise about what phenomena/structures we mean to "pull out" and b) that any notion of agency we stipulate is coherent and well-defined (even sometimes we have to start with a work-in-progress definition).

The extent to which the four notions of agency discussed above vary highlights the need to be explicit about what notion you have in mind if you invoke the term. We should also remember that these different notions tend to come from different traditions of thought, and as such, what is considered the 'default interpretation' of the term 'agency' - and with it an entire set of assumptions and connotations that is invoked when using the term - will vary depending on what your intellectual upbringing was like. 

Finally, we can ask ourselves to what extent the notions of agency that Okasha has picks out in the paper overlap with the notion(s) of agency we often care about in Alignment. While there definitely is important overlap, there might be some aspects we care about that aren't (fully/appropriately) captured in any of the four concepts. As I don't want to explode the scope of this post, let me just leave it at this: it's not clear whether any of the four concepts captures (or is properly aimed at capturing) some notion of "long range consequentialism". (Nor is it entirely clear, for that matter, whether they should, i.e. whether "long range consequentialism" points at anything real and coherent).

Reflection 3: Agency and Autonomy

Okasha remarks that, for the 'organisms-as-agents thesis' (motivation 1), while the 'agency as intelligent behavior' (concept 2) notion seems most apt among the four, it fails (on its own) to "precisely capture organismic distinctiveness". Okasha goes on speculating: "The underlying reason, I suggest, is that none of the four concepts captures the idea of autonomy, which intuitively is what distinguishes organisms from other flexible goal-directed systems in nature. This conclusion fits with the position of Moreno and Mossio (2015), who argue that agency is one central component of organismic autonomy rather than the whole story."

Zooming out, this is a good example of how a given notion of agency may or may not capture all that one intended for it to capture. In this particular example, sometimes when we talk about agency, we intend to capture some notion of 'autonomy'. In the context of the paper, 'autonomy' was meant to point at what we intuitively mean when we want to say that an organism is 'autonomous' in a way that cells or organism making up that organism aren't. This notion of autonomy is not (at least not straightforwardly) captured by any of the four notions of agency captured here. (Worth noting that this is a fairly weak-sauce and largely inadequate 'definition' of autonomy that I've provided here, but for the sake of the point I'm trying to make, it shall suffice.)