I think this touches on the issue of the definition of "truth". A society designates something to be "true" when the majority of people in that society believe something to be true.
Using the techniques outlined in this paper, we could regulate AIs so that they only tell us things we define as "true". At the same time, a 16th century society using these same techniques would end up with an AI that tells them to use leeches to cure their fevers.
What is actually being regulated isn't "truthfulness", but "accepted by the majority-ness".
Would this multiple evaluation/regulatory bodies solution not just lead to the sort of balkanized internet described in this story? I guess multiple internet censorship-and-propaganda-regimes is better than one. But ideally we'd have none.
One alternative might be to ban or regulate persuasion tools, i.e. any AI system optimized for an objective/reward function that involves persuading people of things. Especially politicized or controversial things.
I think this touches on the issue of the definition of "truth". A society designates something to be "true" when the majority of people in that society believe something to be true.
Using the techniques outlined in this paper, we could regulate AIs so that they only tell us things we define as "true". At the same time, a 16th century society using these same techniques would end up with an AI that tells them to use leeches to cure their fevers.
What is actually being regulated isn't "truthfulness", but "accepted by the majority-ness".
This works well for thin... (read more)