The latest in AI, every dayAI News
← AI News

November 2, 2026 · Tech Times

OpenAI, Google, Microsoft and Anthropic Back a Shared robots.txt Signal for AI Training Opt-Outs

My take: Until now, if you wanted to keep the big AI companies from training their models on your content, you had to block each crawler separately: GPTBot, Google-Extended, ClaudeBot, and the list goes on. Since October 30, all four companies back a single robots.txt line (`User-Agent: __usage_ai-training` with `Disallow: /`) that covers all of them at once. For anyone publishing their own content, blog, photos, music or anything else, this is genuinely useful, practical information.

It's worth reading the fine print before celebrating. Anthropic already honors it in its crawler; OpenAI says it will "in the coming months" with no firm date; Microsoft promises to start February 1, 2027; and Google is targeting "early 2027." In the meantime, it's smart to keep whatever individual blocks you already have in place. And something important: the signal isn't legally mandatory, it doesn't cover what was already collected before October 2026, and if your site says nothing, the default assumption is that you're fine with training.

For anyone who creates content as a living or as part of their brand, this is the kind of technical detail worth fixing once and leaving in place. It doesn't change how often you show up in search (that's indexing, a different thing), only whether your work feeds a model's training without you choosing that.

If you publish your own content, have you checked your site's robots.txt, or is it still saying "yes" to everything by default?

Read at the source: Tech Times ↗

Want to use these tools? See the unbiased reviews or back to the news.