AI company Perplexity is sneaking to get around blocks on crawlers, Cloudflare alleges | CyberScoop<\/title> <link rel=\"canonical\" href=\"https:\/\/cyberscoop.com\/perplexity-blocks-on-crawlers-cloudflare\/\"> <meta property=\"og:locale\" content=\"en_US\"> <meta property=\"og:type\" content=\"article\"> <meta property=\"og:title\" content=\"AI company Perplexity is sneaking to get around blocks on crawlers, Cloudflare alleges\"> <meta property=\"og:description\" content=\"Cloudflare said it received complaints from customers about Perplexity using stealthy tactics to evade network blocks against systematic browsing and scraping of web pages.\"> <meta property=\"og:url\" content=\"https:\/\/cyberscoop.com\/perplexity-blocks-on-crawlers-cloudflare\/\"> <meta property=\"og:site_name\" content=\"CyberScoop\"> <meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cyberscoop\/\"> <meta property=\"article:published_time\" content=\"2025-08-04T20:30:07+00:00\"> <meta property=\"article:modified_time\" content=\"2025-08-04T20:30:10+00:00\"> <meta property=\"og:image\" content=\"https:\/\/ddi.mohflo.net\/wp-content\/uploads\/2025\/08\/ai-company-perplexity-is-sneaking-to-get-around-blocks-on-crawlers-cloudflare-alleges-2.jpg\"> <meta property=\"og:image:width\" content=\"1920\"> <meta property=\"og:image:height\" content=\"1280\"> <meta property=\"og:image:type\" content=\"image\/jpeg\"> <meta name=\"author\" content=\"Tim Starks\"> <meta name=\"twitter:card\" content=\"summary_large_image\"> <meta name=\"twitter:creator\" content=\"@timstarks\"> <meta name=\"twitter:site\" content=\"@CyberScoopNews\">  <link rel=\"dns-prefetch\" href=\"\/\/securepubads.g.doubleclick.net\">\n<link rel=\"dns-prefetch\" href=\"\/\/use.typekit.net\">\n<link rel=\"alternate\" type=\"application\/rss+xml\" title=\"CyberScoop \u00bb Feed\" href=\"https:\/\/cyberscoop.com\/feed\/\">\n<link rel=\"alternate\" type=\"application\/rss+xml\" title=\"CyberScoop \u00bb Comments Feed\" href=\"https:\/\/cyberscoop.com\/comments\/feed\/\"> <link rel=\"stylesheet\" id=\"all-css-2\" href=\"https:\/\/cyberscoop.com\/wp-includes\/css\/dist\/block-library\/style.min.css?m=1752617955g\" type=\"text\/css\" media=\"all\"> <link rel=\"stylesheet\" id=\"all-css-6\" href=\"https:\/\/cyberscoop.com\/wp-content\/mu-plugins\/search\/elasticpress\/dist\/css\/related-posts-block-styles.min.css?m=1750115417g\" type=\"text\/css\" media=\"all\"> <link rel=\"stylesheet\" id=\"all-css-8\" href=\"https:\/\/cyberscoop.com\/wp-content\/themes\/scoopnewsgroup\/dist\/css\/frontend.css?m=1753281318g\" type=\"text\/css\" media=\"all\">\n<link rel=\"stylesheet\" id=\"typekit-css\" href=\"https:\/\/use.typekit.net\/itk2qbh.css?ver=13897d660a0ac2c9c7d1\" media=\"all\"> <link rel=\"https:\/\/api.w.org\/\" href=\"https:\/\/cyberscoop.com\/wp-json\/\"><link rel=\"alternate\" title=\"JSON\" type=\"application\/json\" href=\"https:\/\/cyberscoop.com\/wp-json\/wp\/v2\/posts\/85447\"><link rel=\"EditURI\" type=\"application\/rsd+xml\" title=\"RSD\" href=\"https:\/\/cyberscoop.com\/xmlrpc.php?rsd\">\n<meta name=\"generator\" content=\"WordPress 6.8.2\">\n<link rel=\"shortlink\" href=\"https:\/\/cyberscoop.com\/?p=85447\">\n<link rel=\"alternate\" title=\"oEmbed (JSON)\" type=\"application\/json+oembed\" href=\"https:\/\/cyberscoop.com\/wp-json\/oembed\/1.0\/embed?url=https%3A%2F%2Fcyberscoop.com%2Fperplexity-blocks-on-crawlers-cloudflare%2F\">\n<link rel=\"alternate\" title=\"oEmbed (XML)\" type=\"text\/xml+oembed\" href=\"https:\/\/cyberscoop.com\/wp-json\/oembed\/1.0\/embed?url=https%3A%2F%2Fcyberscoop.com%2Fperplexity-blocks-on-crawlers-cloudflare%2F&format=xml\">   <link rel=\"icon\" href=\"https:\/\/cyberscoop.com\/wp-content\/uploads\/sites\/3\/2023\/01\/cropped-cs_favicon-2.png?w=32\" sizes=\"32x32\">\n<link rel=\"icon\" href=\"https:\/\/cyberscoop.com\/wp-content\/uploads\/sites\/3\/2023\/01\/cropped-cs_favicon-2.png?w=192\" sizes=\"192x192\">\n<link rel=\"apple-touch-icon\" href=\"https:\/\/cyberscoop.com\/wp-content\/uploads\/sites\/3\/2023\/01\/cropped-cs_favicon-2.png?w=180\">\n<meta name=\"msapplication-TileImage\" content=\"https:\/\/cyberscoop.com\/wp-content\/uploads\/sites\/3\/2023\/01\/cropped-cs_favicon-2.png?w=270\"> <\/head><body class=\"wp-singular post-template-default single single-post postid-85447 single-format-standard wp-theme-scoopnewsgroup wp-child-theme-cyberscoop\" id=\"readabilityBody\"> <a href=\"https:\/\/cyberscoop.com\/perplexity-blocks-on-crawlers-cloudflare\/#main\" class=\"skip-to-content-link visually-hidden-focusable\">Skip to main content<\/a> <\/p>\n<div class=\"ad ad--top ad--top-desktop\">\n<div class=\"ad__inner\"> <span class=\"screen-reader-text\">Advertisement<\/span> <\/div>\n<\/div>\n<p> <main id=\"main\" role=\"main\" tabindex=\"-1\"> <\/p>\n<div class=\"ad ad--top ad--top-mobile\">\n<div class=\"ad__inner\"> <span class=\"screen-reader-text\">Advertisement<\/span> <\/div>\n<\/div>\n<section id=\"stickybar\" class=\"stickybar stickybar--newsletter js-stickybar\" readability=\"0.82\"> <button class=\"stickybar__close js-stickybar-close\" aria-controls=\"stickybar\"> <svg class=\"icon icon--close\" width=\"21\" height=\"22\" viewBox=\"0 0 21 22\" fill=\"none\"><path d=\"m.822.518-.805.805L9.695 11 .017 20.678l.805.805 9.678-9.678 9.677 9.678.806-.805L11.305 11l9.678-9.677-.806-.805-9.677 9.677L.822.518Z\" fill=\"currentColor\" \/><\/svg> <span class=\"visually-hidden\">Close<\/span> <\/button> <\/section>\n<article class=\"single-article content\">\n<div class=\"single-article__container js-single-article-content\">\n<header class=\"single-article__header \" readability=\"25.8375\">\n<div class=\"single-article__header-content\" readability=\"35.029213483146\">\n<p> Cloudflare said it received complaints from customers about Perplexity using stealthy tactics to evade network blocks against systematic browsing and scraping of web pages. <\/p>\n<p>  <br \/>\n<audio id=\"audio-player\" src=\"https:\/\/wp-tts-cdn.api.scpnewsgrp.com\/cyberscoop\/85447\/english.openai.mp3\"><\/audio> <\/p>\n<div readability=\"11\">\n<div>\n<p>Listen to this article<\/p>\n<p>  <\/p>\n<p>0:00<\/p>\n<\/p><\/div>\n<p>  <\/p>\n<p> <span id=\"tts-tooltip\">Learn more.<\/span> <span> This feature uses an automated voice, which may result in occasional errors in pronunciation, tone, or sentiment. <\/span> <\/p>\n<\/div>\n<p>  <\/div>\n<div class=\"single-article__cover-wrap\">\n<figure class=\"single-article__cover\"> <img data-recalc-dims=\"1\" fetchpriority=\"high\" width=\"640\" height=\"426\" src=\"https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2025\/08\/ai-company-perplexity-is-sneaking-to-get-around-blocks-on-crawlers-cloudflare-alleges.jpg?resize=640%2C426&ssl=1\" class=\"single-article__cover-image wp-post-image\" alt decoding=\"async\" fetchpriority=\"high\" srcset=\"https:\/\/ddi.mohflo.net\/wp-content\/uploads\/2025\/08\/ai-company-perplexity-is-sneaking-to-get-around-blocks-on-crawlers-cloudflare-alleges-2.jpg 1920w, https:\/\/ddi.mohflo.net\/wp-content\/uploads\/2025\/08\/ai-company-perplexity-is-sneaking-to-get-around-blocks-on-crawlers-cloudflare-alleges-2.jpg?resize=300,200 300w, https:\/\/ddi.mohflo.net\/wp-content\/uploads\/2025\/08\/ai-company-perplexity-is-sneaking-to-get-around-blocks-on-crawlers-cloudflare-alleges-2.jpg?resize=768,512 768w, https:\/\/ddi.mohflo.net\/wp-content\/uploads\/2025\/08\/ai-company-perplexity-is-sneaking-to-get-around-blocks-on-crawlers-cloudflare-alleges-2.jpg?resize=1024,683 1024w, https:\/\/ddi.mohflo.net\/wp-content\/uploads\/2025\/08\/ai-company-perplexity-is-sneaking-to-get-around-blocks-on-crawlers-cloudflare-alleges-2.jpg?resize=1536,1024 1536w, https:\/\/ddi.mohflo.net\/wp-content\/uploads\/2025\/08\/ai-company-perplexity-is-sneaking-to-get-around-blocks-on-crawlers-cloudflare-alleges-2.jpg?resize=600,400 600w, https:\/\/ddi.mohflo.net\/wp-content\/uploads\/2025\/08\/ai-company-perplexity-is-sneaking-to-get-around-blocks-on-crawlers-cloudflare-alleges-2.jpg?resize=252,168 252w, https:\/\/ddi.mohflo.net\/wp-content\/uploads\/2025\/08\/ai-company-perplexity-is-sneaking-to-get-around-blocks-on-crawlers-cloudflare-alleges-2.jpg?resize=506,337 506w, https:\/\/ddi.mohflo.net\/wp-content\/uploads\/2025\/08\/ai-company-perplexity-is-sneaking-to-get-around-blocks-on-crawlers-cloudflare-alleges-2.jpg?resize=1013,675 1013w, https:\/\/ddi.mohflo.net\/wp-content\/uploads\/2025\/08\/ai-company-perplexity-is-sneaking-to-get-around-blocks-on-crawlers-cloudflare-alleges-2.jpg?resize=1265,843 1265w\" sizes=\"(max-width: 1013px) 100vw, 1013px\"><figcaption> moolyboo, iStock\/Getty Images Plus <\/figcaption><\/figure>\n<\/p><\/div>\n<\/header>\n<div class=\"single-article__content\">\n<div class=\"single-article__content-inner has-drop-cap\"> <html readability=\"25.800511508951\"><body readability=\"53.214165261383\"><\/p>\n<p>Artificial intelligence startup Perplexity is using stealthy techniques to get around network blocks against systematic browsing and scraping of web pages, Cloudflare said Monday in a blog post.<\/p>\n<p>The alleged activity prompted Cloudflare, which received complaints from its customers, to take action against Perplexity.<\/p>\n<p>\u201cThere are clear preferences that crawlers should be transparent, serve a clear purpose, perform a specific activity, and, most importantly, follow website directives and preferences,\u201d <a href=\"https:\/\/blog.cloudflare.com\/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives\/\">Cloudflare engineers wrote<\/a>. \u201cBased on Perplexity\u2019s observed behavior, which is incompatible with those preferences, we have de-listed them as a verified bot and added heuristics to our managed rules that block this stealth crawling.\u201d<\/p>\n<p>It\u2019s the latest step from Cloudflare in its approach to crawling from AI systems, following <a href=\"https:\/\/cyberscoop.com\/cloudflare-ai-web-crawlers-pay-per-crawl-websites-data\/\">last month\u2019s<\/a> announcement allowing customers to block or charge fees from web crawlers deployed to scrape their websites and data.<\/p>\n<div class=\"ad ad--inline_1 \">\n<div class=\"ad__inner\"> <span class=\"screen-reader-text\">Advertisement<\/span> <\/div>\n<\/div>\n<p>Customers who disallowed Perplexity crawling activity in their robots.txt files \u2014 a file that instructs search engine crawlers which parts of a website they can and cannot access \u2014 told CloudFlare that Perplexity was still able to access their content. <\/p>\n<p>\u201cThese customers told us that Perplexity was still able to access their content even when they saw its bots successfully blocked,\u201d Cloudflare said. \u201cWe confirmed that Perplexity\u2019s crawlers were in fact being blocked on the specific pages in question, and then performed several targeted tests to confirm what exact behavior we could observe.\u201d<\/p>\n<p>Emails to a Perplexity spokesperson and media email address seeking a response were not immediately answered. But spokesperson Jesse Dwyer <a href=\"https:\/\/techcrunch.com\/2025\/08\/04\/perplexity-accused-of-scraping-websites-that-explicitly-blocked-ai-scraping\/\">told TechCrunch<\/a> that Cloudflare\u2019s blog post was no more than a \u201csales pitch,\u201d that the screenshots in the post \u201cshow that no content was accessed\u201d and the bot Cloudflare named \u201cisn\u2019t even ours.\u201d<\/p>\n<p>Perplexity has encountered allegations of <a href=\"https:\/\/www.wired.com\/story\/perplexity-is-a-bullshit-machine\/\">unethical web scraping<\/a> in the past. Most recently, the BBC has <a href=\"https:\/\/www.ft.com\/content\/b743d401-dc5d-44b8-9987-825a4ffcf4ca\">threatened to sue the company<\/a> over content scraping, one of many suits AI companies are facing, although some organizations have <a href=\"https:\/\/pressgazette.co.uk\/platforms\/news-publisher-ai-deals-lawsuits-openai-google\/\">signed deals<\/a> with AI firms, including with Perplexity.<\/p>\n<p>In its blog post, Cloudflare said OpenAI is an example of a company following recommended practices on crawlers and blocked behavior.<\/p>\n<p><\/body> <\/p>\n<footer class=\"single-article__footer\">\n<div class=\"author-card\" readability=\"7.7216117216117\">\n<div class=\"author-card__avatar\">\n<figure class=\"author-card__image-wrap\"> <img data-recalc-dims=\"1\" decoding=\"async\" class=\"author-card__image\" src=\"https:\/\/i0.wp.com\/ddi.mohflo.net\/wp-content\/uploads\/2025\/08\/ai-company-perplexity-is-sneaking-to-get-around-blocks-on-crawlers-cloudflare-alleges-1.jpg?w=640&ssl=1\" alt=\"Tim Starks\"> <\/figure>\n<\/p><\/div>\n<div class=\"author-card__details\" readability=\"10.901098901099\">\n<h4 class=\"author-card__name\">Written by Tim Starks<\/h4>\n<p> Tim Starks is senior reporter at CyberScoop. His previous stops include working at The Washington Post, POLITICO and Congressional Quarterly. An Evansville, Ind. native, he’s covered cybersecurity since 2003. Email Tim here: <a href=\"mailto:tim.starks@cyberscoop.com\">tim.starks@cyberscoop.com<\/a>. <\/div>\n<\/p><\/div>\n<div class=\"single-article__tags-container\">\n<h4 class=\"single-article__tags-title\">In This Story<\/h4>\n<\/p><\/div>\n<\/footer>\n<p> <\/html><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<div class=\"single-article__ads js-single-article-sidebar\">\n<div class=\"ad ad--sidebar js-single-article-sidebar-5 ad--rightrail_1 \">\n<div class=\"ad__inner\"> <span class=\"screen-reader-text\">Advertisement<\/span> <\/div>\n<\/div>\n<div class=\"ad ad--sidebar js-single-article-sidebar-4 ad--rightrail_2 \">\n<div class=\"ad__inner\"> <span class=\"screen-reader-text\">Advertisement<\/span> <\/div>\n<\/div>\n<div class=\"ad ad--sidebar js-single-article-sidebar-3 ad--rightrail_3 \">\n<div class=\"ad__inner\"> <span class=\"screen-reader-text\">Advertisement<\/span> <\/div>\n<\/div><\/div>\n<\/article>\n<div class=\"popular-stories popular-stories--single-post\">\n<div class=\"popular-stories__container\">\n<h2 class=\"popular-stories__title\"> More Scoops <\/h2>\n<p>  <\/div>\n<p>\n<\/div>\n<p> <\/p>\n<section class=\"latest-podcasts\">\n<h2 class=\"latest-podcasts__title\"> Latest Podcasts\t<\/h2>\n<\/section>\n<div class=\"top-categories\">\n<div class=\"top-categories__container\">\n<h3 class=\"top-categories__category-title\">Government<\/h3>\n<\/p><\/div>\n<div class=\"top-categories__container\">\n<h3 class=\"top-categories__category-title\">Technology<\/h3>\n<\/p><\/div>\n<\/p><\/div>\n<p> <\/main> <\/p>\n<div class=\"ad ad--bottom \">\n<div class=\"ad__inner\"> <span class=\"screen-reader-text\">Advertisement<\/span> <\/div>\n<\/div>\n<div id=\"interstitial\" class=\"welcome__container\"> <button id=\"close-modal-1\" class=\"welcome__clickable_area\"><\/button> <\/p>\n<div class=\"welcome__ad_wrapper\">\n<p> <button id=\"close-modal-3\" class=\"welcome__continue-button\">Continue to CyberScoop<\/button> <\/p>\n<\/p><\/div>\n<\/p><\/div>\n<p>   <\/body> <a href=\"https:\/\/cyberscoop.com\/perplexity-blocks-on-crawlers-cloudflare\/\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>AI company Perplexity is sneaking to get around blocks on<\/p>\n","protected":false},"author":11,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[235,384,632,4734,268,288,4735,4519],"tags":[236,388,633,4736,274,294,4737,4520],"class_list":["post-7863","post","type-post","status-publish","format-standard","hentry","category-ai","category-artificial-intelligence-ai","category-cloudflare","category-perplexity","category-privacy","category-threats","category-web-crawling","category-web-scraping","tag-ai","tag-artificial-intelligence-ai","tag-cloudflare","tag-perplexity","tag-privacy","tag-threats","tag-web-crawling","tag-web-scraping"],"featured_image_urls":{"full":"","thumbnail":"","medium":"","medium_large":"","large":"","1536x1536":"","2048x2048":"","chromenews-featured":"","chromenews-large":"","chromenews-medium":""},"author_info":{"display_name":"Cyber Scoop","author_link":"https:\/\/ddi.mohflo.net\/index.php\/author\/cyberscoop\/"},"category_info":"<a href=\"https:\/\/ddi.mohflo.net\/index.php\/category\/ai\/\" rel=\"category tag\">AI<\/a> <a href=\"https:\/\/ddi.mohflo.net\/index.php\/category\/artificial-intelligence-ai\/\" rel=\"category tag\">artificial intelligence (AI)<\/a> <a href=\"https:\/\/ddi.mohflo.net\/index.php\/category\/cloudflare\/\" rel=\"category tag\">Cloudflare<\/a> <a href=\"https:\/\/ddi.mohflo.net\/index.php\/category\/perplexity\/\" rel=\"category tag\">Perplexity<\/a> <a href=\"https:\/\/ddi.mohflo.net\/index.php\/category\/privacy\/\" rel=\"category tag\">Privacy<\/a> <a href=\"https:\/\/ddi.mohflo.net\/index.php\/category\/threats\/\" rel=\"category tag\">Threats<\/a> <a href=\"https:\/\/ddi.mohflo.net\/index.php\/category\/web-crawling\/\" rel=\"category tag\">web crawling<\/a> <a href=\"https:\/\/ddi.mohflo.net\/index.php\/category\/web-scraping\/\" rel=\"category tag\">web scraping<\/a>","tag_info":"web scraping","comment_count":"0","jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/7863","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/users\/11"}],"replies":[{"embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/comments?post=7863"}],"version-history":[{"count":0,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/posts\/7863\/revisions"}],"wp:attachment":[{"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/media?parent=7863"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/categories?post=7863"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ddi.mohflo.net\/index.php\/wp-json\/wp\/v2\/tags?post=7863"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}

{"id":7863,"date":"2025-08-04T15:30:07","date_gmt":"2025-08-04T20:30:07","guid":{"rendered":"https:\/\/cyberscoop.com\/?p=85447"},"modified":"2025-08-04T15:30:07","modified_gmt":"2025-08-04T20:30:07","slug":"ai-company-perplexity-is-sneaking-to-get-around-blocks-on-crawlers-cloudflare-alleges","status":"publish","type":"post","link":"https:\/\/ddi.mohflo.net\/index.php\/2025\/08\/04\/ai-company-perplexity-is-sneaking-to-get-around-blocks-on-crawlers-cloudflare-alleges\/","title":{"rendered":"AI company Perplexity is sneaking to get around blocks on crawlers, Cloudflare alleges"},"content":{"rendered":"