In a stunning revelation, a new study by Epoch AI projects that the supply of publicly available training data for AI language models will be depleted by the turn of the decade. This means that the current pace of progress in AI development will soon hit a roadblock, as tech companies exhaust the tens of trillions of words available online. The researchers warn that the AI field may struggle to maintain its current trajectory once the reserves of human-generated writing are drained. To make matters worse, relying on synthetic data generated by chatbots themselves may not be a reliable solution, as it can lead to degraded performance and encode existing biases.
In the short term, companies are scrambling to secure high-quality data sources, but in the long run, they may be forced to tap into sensitive private data or rely on less-reliable synthetic data. The study’s authors warn that this could have serious implications for the future of AI development.
