Technology & AI

Google and Microsoft back OpenAI's robots.txt AI-training opt-out

The move is a proposed web standard letting sites tell AI companies not to use their content for training models.

By Standard Topic. AI-generated from the reporting listed under Coverage. Published . Updated .

This story is no longer being updated. It reflects reporting up to .

What happened

Google and Microsoft have signed up to OpenAI's plan for a new website crawling standard. Anthropic has also pledged to honor a shared robots.txt preference about using website content to train AI models. The signal does not block bots from crawling websites, and the proposal has not been endorsed by the IETF.

Why it matters

The standard could soon let website owners stop AI scraping with a single robots.txt line. It remains a pledge and is not yet an endorsed standard.

Coverage

More from Technology & AI

All Technology & AI news · Technology & AI story archive