3dcadmin@lemmy.relayeasy.com to Selfhosted@lemmy.worldEnglish · 3 days agoCloudflare blocking AI crawlerslemmy.relayeasy.comimagemessage-square33fedilinkarrow-up1346arrow-down12file-text
arrow-up1344arrow-down1imageCloudflare blocking AI crawlerslemmy.relayeasy.com3dcadmin@lemmy.relayeasy.com to Selfhosted@lemmy.worldEnglish · 3 days agomessage-square33fedilinkfile-text
Cloudflare trying to stop AI crawling somehow! https://arstechnica.com/tech-policy/2025/07/pay-up-or-stop-scraping-cloudflare-program-charges-bots-for-each-crawl/
minus-squareProntomomo@lemmy.worldcakelinkfedilinkEnglisharrow-up5·3 days agoI don’t think this blocks crawlers. About 1/5 websites uses cloudflare, the significant thing here’s is that AI scraping is now blocked by default on most of those sites, NOT crawling
I don’t think this blocks crawlers. About 1/5 websites uses cloudflare, the significant thing here’s is that AI scraping is now blocked by default on most of those sites, NOT crawling