The only feature that CloudFlare uses that creates a crawl of your site is Always Online so we can display a limited cache of your site if your server goes down. The useragent we use is:
Mozilla/5.0 (compatible; CloudFlare-AlwaysOnline/1.0; +http://www.cloudflare.com/always-online
We crawl sites based on the CloudFlare pricing plan for the domain.
Free customers once every 9 days
Pro customers once every 3 days
Business and Enterprise customers daily.
Comments
Article is closed for comments.