What happens when, because it’s so quick and easy to churn out, 50% or more of the web is AI generated slush, which is then scraped and incorporated into the next generation of LLMs, which increases that percentage and in turn is then scraped, and so on, and so on?
How low can the quality of your training data drop before the results become intolerably bad? How do you raise the quality of that data without a massive investment of human labor? How much glue will be told to put on our pizza two years from now?
Generative AI could be a powerful tool, but even ignoring ethical considerations, this seems like a profoundly bad way to imement it.