Regardless of the exact starting point, seekers of “True Names” quickly find themselves recursing into a search for “True Names” of lower-level components of agency, like:
Optimization
Goals
World models
Abstraction
This is the big missing piece for me. Could you elaborate on how you go from trying to find the True Names of human values to things like what is an agent, abstraction, and embeddedness?
Goals makes sense, but the rest are not obvious why they'd be important or relevant. I feel like this reasoning would lead you to thinking about meta-ethics or something, not embeddedness and optimization.
I suspect I'm missing a connecting piece here that would make it all click.
This is the big missing piece for me. Could you elaborate on how you go from trying to find the True Names of human values to things like what is an agent, abstraction, and embeddedness?
Goals makes sense, but the rest are not obvious why they'd be important or relevant. I feel like this reasoning would lead you to thinking about meta-ethics or something, not embeddedness and optimization.
I suspect I'm missing a connecting piece here that would make it all click.