Do I have to change my Cloudflare settings for this?

Cloudflare says "Nothing, in almost every case. Your current settings carry over on their own." Existing bot preferences migrate automatically. Newly onboarded domains are offered presets based on whether the site runs ads, and any site owner can change the choice during setup or later from the Security Settings panel in the Cloudflare dashboard.

How is blocking AI training different from blocking search?

Cloudflare now sorts crawler activity into three behaviours instead of one allow-or-block decision. Search covers crawling to build a search index, Training covers crawling to train or fine-tune a model, and Agent covers user-directed agents visiting pages for a human. Disallow AI Training targets only the Training behaviour, so search indexing continues.

Will opting out of AI training hurt my search ranking?

Not for an operator Cloudflare calls Accountable. One of the four criteria is an explicit "assurance that opting out of AI training will not affect traditional search results." That guarantee is the point of the designation: before this split, refusing an AI crawler could also cost a site the search traffic from the same company.

Which rules do Apple, Google and Microsoft honour?

Google reads the Google-Extended disallow rule and offers a webmaster portal toggle plus metrics reporting. Apple reads Applebot-Extended and supports the nosnippet directive and paywalled-content labeling. Microsoft currently supports the NOARCHIVE meta tag and has committed to robots.txt support targeted for early 2027. All three qualified as Accountable.

What else does the Accountable designation require?

Beyond the search-ranking assurance, an Accountable operator must offer a way to opt out of AI training through robots.txt or a similar standard, a way to opt out of AI summaries set with the operator directly and through Cloudflare next year, and URL-level visibility into which pages were made available for training plus metrics on how content appeared in search.

Cloudflare · 2026-09-15 · major

Cloudflare adds a Disallow AI Training setting — search stays, training stops

Cloudflare's new Disallow AI Training setting blocks model training through robots.txt while leaving search crawling alone. Cloudflare also named Apple, Google and Microsoft "Accountable", and split bot rules into Search, Training and Agent.

Cloudflare blog graphic for the Disallow AI Training setting and Accountable crawler designation — Cloudflare

A site can now tell AI crawlers "index me, don't train on me" — and Cloudflare publishes the rule for it.

Quick facts

Maker	Cloudflare
New setting	Disallow AI Training
Bot classifications	Search, Training, Agent
Accountable operators	Apple, Google, Microsoft
Action needed	None — existing settings carry over
Delivered by	Bot Preference Sync (replaces Managed Robots.txt)

What is it?

A new Disallow AI Training setting pulls a crawler's two jobs apart: a site can stay in search results while refusing to be used as model training data. Cloudflare now sorts bot activity into three behaviours — Search, Training and Agent — rather than one allow-or-block decision per crawler, and adds a "Block on pages with ads" option.

How does it work?

The choice is published as a Disallow entry for the operator's training user-agent in robots.txt, written and kept in step with the edge rules by Cloudflare's Bot Preference Sync, which replaces the deprecated Managed Robots.txt. Each operator reads its own opt-out token: Google-Extended for Google, Applebot-Extended for Apple. Microsoft supports the NOARCHIVE meta tag today and targets robots.txt support for early 2027.

Why does it matter?

Blocking an AI crawler has often risked the search traffic from the same company, because a single user-agent did both jobs. Splitting Search from Training gives publishers a refusal that does not cost them their search listing. The Accountable label attaches named commitments — an opt-out path, URL-level reporting, and no search penalty — to the operators that meet them.

Who is it for?

publishers and site owners

Frequently asked questions

Do I have to change my Cloudflare settings for this?: Cloudflare says "Nothing, in almost every case. Your current settings carry over on their own." Existing bot preferences migrate automatically. Newly onboarded domains are offered presets based on whether the site runs ads, and any site owner can change the choice during setup or later from the Security Settings panel in the Cloudflare dashboard.
How is blocking AI training different from blocking search?: Cloudflare now sorts crawler activity into three behaviours instead of one allow-or-block decision. Search covers crawling to build a search index, Training covers crawling to train or fine-tune a model, and Agent covers user-directed agents visiting pages for a human. Disallow AI Training targets only the Training behaviour, so search indexing continues.
Will opting out of AI training hurt my search ranking?: Not for an operator Cloudflare calls Accountable. One of the four criteria is an explicit "assurance that opting out of AI training will not affect traditional search results." That guarantee is the point of the designation: before this split, refusing an AI crawler could also cost a site the search traffic from the same company.
Which rules do Apple, Google and Microsoft honour?: Google reads the Google-Extended disallow rule and offers a webmaster portal toggle plus metrics reporting. Apple reads Applebot-Extended and supports the nosnippet directive and paywalled-content labeling. Microsoft currently supports the NOARCHIVE meta tag and has committed to robots.txt support targeted for early 2027. All three qualified as Accountable.
What else does the Accountable designation require?: Beyond the search-ranking assurance, an Accountable operator must offer a way to opt out of AI training through robots.txt or a similar standard, a way to opt out of AI summaries set with the operator directly and through Cloudflare next year, and URL-level visibility into which pages were made available for training plus metrics on how content appeared in search.

Try it

Cloudflare dashboard → Security → Settings → Disallow AI Training