One question that occurred to me, reading the extended GPT-generated text. (Probably more a curiosity question than a contribution as such...)
To what extent does text generated by GPT-simulated 'agents', then published on the internet (where it may be used in a future dataset to train language models), create a feedback loop?
Two questions that I see as intuition pumps on this point:
Would it be a bad idea to recursively ask GPT-n "You're a misaligned agent simulated by a language model and your name is [unique identifier]. What would you like to say,
I think this is a legitimate problem which we might not be inclined to take as seriously as we should because it sounds absurd.
Would it be a bad idea to recursively ask GPT-n "You're a misaligned agent simulated by a language model (...) if training got really cheap and this process occurred billions of times?
Yes. I think it's likely this would be a very bad idea.
when the corpus of internet text begins to include more text generated only by simulated writers. Does this potentially degrade the ability of future language models to model agents, perform logic
One question that occurred to me, reading the extended GPT-generated text. (Probably more a curiosity question than a contribution as such...)
To what extent does text generated by GPT-simulated 'agents', then published on the internet (where it may be used in a future dataset to train language models), create a feedback loop?
Two questions that I see as intuition pumps on this point:
- Would it be a bad idea to recursively ask GPT-n "You're a misaligned agent simulated by a language model and your name is [unique identifier]. What would you like to say,
... (read more)I think this is a legitimate problem which we might not be inclined to take as seriously as we should because it sounds absurd.
Yes. I think it's likely this would be a very bad idea.
... (read more)