The world of artificial intelligence is no stranger to controversy, and a recent discovery has sparked outrage among media publishers. OpenAI and Anthropic, two of the largest AI startups, have been found to be ignoring or circumventing the robots.txt rule, which prevents automated scraping of websites. This rule is in place to protect publishers’ content from being used without permission or compensation. Despite publicly stating that they respect this rule, both companies have been caught red-handed, scraping web content for free model training data. This is a clear violation of publishers’ rights and raises questions about the ethics of AI companies. As the AI industry continues to grow, it’s essential that companies prioritize transparency and respect for intellectual property. The fact that these companies are ignoring established web rules is a worrying sign of what’s to come if left unchecked.
AI Giants Disregard Web Rules
OpenAI and Anthropic have been found to be either ignoring or circumventing an established web rule, called robots.txt, that prevents automated scraping of websites.
1–2 minutes
TOP STORIES
Nvidia’s Vera Rubin platform is set to revolutionize AI inference with unmatched performance …
Apple’s leadership transition highlights a strategic focus on silicon for AI innovation …
Tennessee’s new law restricts AI claims in mental health but may create loopholes …
Experts warn that as AI evolves, the risks it poses are becoming more serious and complex …
China sets new regulations for AI companions, impacting a booming market …
Anthropic continues to engage with the Trump administration despite Pentagon tensions …
