This post is part of the work done at Conjecture.

This post has been written for the second Refine blog post day, at the end of the first week of iterating on ideas and concretely aiming at the alignment problem.

What constantly frustrates me about the old school philosophers of science, the Poppers, the Kuhns, even the Feyerabend, is their obsession with finding a few simple "rules of science" to apply anywhere and everywhere. This obsession for the overly general (even in the case of the anarchist Feyerabend) leads to leaving much value and insight on the table because all these compressions misses relevant insights in the history of science.

• Popper erects falsification and self-contradiction as the death of a theory, missing all the cases where throwing the theory away was not the correct move. (Examples: Kékulé on the structure of Benzene, Galileo and Copernic on the moving Earth)
• Kuhn pretty much defines a mature science as having only a single paradigm, where many of the advances in science required and thrived off pluralism (Examples: the methodological omnivory of historical scientists, the distance problem and chemical formula determinations in Chemistry)
• Feyerabend concludes from his insightful criticism of Popper's view that there is no rule in science and that "Anything Goes", thus refusing to see regularities that exists within the history of science (See levels of pluralism for an example of drawing on these regularities)

Once again, the issue is not that these thinkers contributed nothing: Popper pushed falsification to interesting places (like evolutionary epistemology); Kuhn articulated beautifully the role of normal science; Feyerabend made accurate and crucial criticisms of the existing methodologies.

The issue is instead that none of them just thought that the strategies that should be used in science, the criteria for what makes sense and is accepted, might depend on the context. Not in a relativist sense of having different notions of truth, but instead saying that epistemic strategy X, which works in field F, might not work in field G.

Nowhere is this lack of context sensitivity clearer than in the old-school belief that Physics is a representative science.

To see why I think this doesn't work, let's look at one concrete example: the dull function hypothesis. In the words of A. Zee, this is the strategy that assumes that when you have to find a function underlying some data, it will be a "boring" function that is simple and has all the good mathematical properties, like a constant or an exponential.

(Fly By Night Physics, A. Zee, 2020)

The unspoken assumption, again based on sense and experience, is that the
majority of the functions appearing in physics are fairly boring functions that
do not have especially wild properties.

And most of the time, in physics, such guesses actually work!

From this, should we expect the dull function hypothesis to generalize to all of science? Clearly not! It is completely possible (and confirmed by experience) that the functions underlying advanced ML tasks and social processes, when they can be inferred, are far more complex than the ones physicists can guess.

What this examples illustrates is how the successes of physics should not only be ascribed to the genius of physicist, but also to the easiness of the epistemic circumstances underlying most physical problems.

Historians and Philosophers of Science seem to have caught on, as they tend to pay far more attention nowadays to the specific context of historical discovery and the underlying parameters of the problems that scientists solved. Yet we are still missing good descriptions of toolboxes of epistemic strategies with the epistemic circumstances in which they work.

That's the work that I'm doing and want to accomplish at Conjecture, in order to help alignment research by proposing the right epistemic strategy for the right circumstance.