We want to be able to point at Elsa’s case of value change and argue that it is problematic and should be prevented, and we want to be able to say that Daniel’s case of value change is fine and does not need to be prevented, without in either case basing our argumentation on whether or not loving jazz is a morally acceptable or not. As such, I argue that the relevant difference we are picking up on here pertains to the legitimacy (or lack thereof) of the value change process (in the sense I've described it above).

Is it really the relevant difference?

I think that there could be cases of acceptable illegitimate value change; that is, if both current I and I-as-in-CEV (in the future, knowing more, etc) would endorse the change, but it were done without a way to course-correct it. Metaphor: imagine you had to walk over a hanging bridge so that you couldn't stop in the middle at risk of injury.

So, in my opinion legitimacy can be based on nature of value change only, but acceptability is also based on the opinion of person in question.