.svg)
.svg)
.svg)
Most scrapers fail on Cloudflare-protected sites because they're using the right tool on the wrong protection tier. They rotate proxies when they need sticky sessions, use datacenter IPs on enterprise-grade bot detection, or fix headers when the real issue is TLS (Transport Layer Security) fingerprinting.
The fix is matching your proxy type and browser stack to the specific Cloudflare tier your target is running. Standard residential proxies solve most CF WAF cases. ISP proxies handle session-sensitive targets. Premium residential + stealth browser is what CF Bot Management requires.
- ~80% of CF-blocked scrapers need a residential proxy upgrade, not a configuration fix
- Datacenter proxies on CF Bot Management is a category error. No settings will fix it.
- Sticky sessions outperform rotation on sites that build trust scores over time
- CF silently returns 200 OK with fake data on some targets. Validate response bodies, not just status codes.
You set up rotating proxies. You added realistic delays. You copied the Puppeteer stealth snippet from three different Stack Overflow answers. You even tested it manually. And Cloudflare still blocked every single request.
Here's the thing: the tutorials you followed were written for a Cloudflare that no longer exists. The platform's bot detection has fundamentally changed over the past 18 months. Most bypass guides haven't caught up, and using 2022 techniques against 2026 Cloudflare is exactly why you're still blocked.
This guide will show you exactly what CF detects now, which proxy tier defeats which protection level, and why your current setup is probably failing for a reason you haven't diagnosed yet.
Why You're Still Blocked (The Real Reasons)
Before jumping into fixes, it's worth understanding the actual failure mode. Because most people are solving the wrong problem.
You're Fighting the Wrong Detection Layer
The majority of scraping guides focus on IP rotation as the primary bypass strategy. That logic made sense in 2020. In 2026, Cloudflare's primary enforcement mechanism is behavior-level and fingerprint-level detection, not IP bans. Rotating IPs when your TLS (Transport Layer Security) fingerprint screams "automation" is like changing your license plate while your car is on fire. TLS fingerprinting works by analyzing how your connection handshake is structured. Every real browser has a distinctive pattern. Automated tools have a different one, and Cloudflare knows the difference.
A common experience among developers testing at scale: even with conservative request rates and proxy rotation, blocks hit almost immediately. The pattern is consistent across different setups. It feels less like a rate issue and more like session and behavior-level detection kicking in early. That's precisely right, and it's what most guides miss entirely.
How to Rotate Proxies for Web Scraping Without Triggering CF
Here's where it gets interesting. The rotation advice in most guides is actually backwards for certain targets. Some developers have found that leaving a residential IP sticky for several weeks produced better results than daily rotation. No CAPTCHAs, no verification loops, no session drops.
The reason? Cloudflare builds trust scores on sessions. Rapid rotation resets that score to zero every time. On sites that reward trust-building, a clean sticky IP beats a rotation strategy every single time. This doesn't mean stop rotating. It means rotate only when rotation helps, and recognize when it's hurting you.
The Silent Failure You Probably Haven't Caught
The nastiest Cloudflare defense isn't a 403 or a challenge page. It's a 200 OK response with empty or honeypot content. Your scraper reports success. Your dataset is garbage. You've potentially been running for hours collecting nothing.
This silent failure is one of the most expensive proxy mistakes in production scraping. Your monitoring needs to catch it before your pipeline does.
What Cloudflare Actually Detects in 2026
Understanding the detection stack isn't academic. It's the only way to know which layer is blocking you, and therefore which fix applies.
| Detection Signal | What CF Checks | Proxy-Solvable? | Common? |
|---|---|---|---|
| IP Reputation | Is this IP datacenter, flagged, or shared? | Yes | Very High |
| TLS Fingerprint (JA3/JA4) | Does your TLS handshake match a real browser? JA3 and JA4 are standardized methods for identifying the "signature" of a connection setup. | Partial | High |
| HTTP/2 Fingerprint | Do your request patterns look like a browser? | No | Medium |
| Browser Fingerprint | Canvas, WebGL, fonts, timezone, screen size | No | High (Enterprise) |
| Behavioral Signals | Mouse movement, click timing, scroll depth | No | Medium (Enterprise) |
The Three CF Protection Tiers (What Nobody Tells You)
This is the part every other guide skips. Cloudflare isn't one wall. It's three different walls at three different heights, and the tier your target is running determines everything.
- Cloudflare (CF) Free / Basic WAF (WAF stands for Web Application Firewall) - IP reputation + basic bot score. This covers around 80% of CF-protected sites. The right residential proxy tier solves it without a browser layer.
- CF Pro WAF - Adds TLS fingerprinting and browser challenges on top of the basic protection. You need a headless browser on top of a quality residential proxy to pass consistently.
- CF Bot Management (Enterprise) - Full machine learning scoring, behavioral analysis, and fingerprint correlation across sessions. Requires premium residential + stealth browser + behavioral simulation. Some of these targets have no reliable DIY solution.
1020 is typically a Web Application Firewall (WAF) firewall rule and is often IP-based. A challenge page that loops without resolving usually means fingerprint detection is active, pointing to a Pro-tier or higher setup. A Turnstile challenge appearing on every request despite a clean IP is a strong indicator of Bot Management.
What Changed in 2024 and 2025
Cloudflare Turnstile has become the dominant challenge method on CF-protected sites, widely replacing hCAPTCHA across most deployments. The key difference is that Turnstile has two modes: managed (invisible, passes automatically with good IP quality and a real browser) and interactive (visible challenge). Most guides still write about hCAPTCHA solving as if it's the only option. In practice, getting your IP reputation and browser fingerprint right eliminates the need for a solver on the majority of Turnstile-protected sites.
The other major shift: cross-domain fingerprint databases. CF now correlates behavior across multiple domains. A session that looks clean on one target can carry reputation from activity on another. This is why even fresh anti-detect browser profiles sometimes get flagged within minutes.
Bypass Cloudflare: The Proxy Tier Hierarchy with Honest Pass Rates
This is what you came for. No competitor publishes this honestly. Here is the actual performance breakdown by proxy type against each CF tier.
Sticky vs. Rotating Proxies: The Decision You're Probably Getting Wrong
The default assumption is "always rotate." That's wrong for a significant portion of scraping use cases. Here's the actual decision framework:
| Use Rotating IPs When... | Use Sticky Sessions When... |
|---|---|
| You need scale across thousands of requests | The site has login flows or session state |
| Target checks IP frequency, not session depth | CF builds a trust score over time on that IP |
| You need city or country diversity across requests | You're managing accounts or persistent identities |
| No persistent user context is needed | Site penalizes IP changes mid-session |
TorchProxies ISP Proxies are purpose-built for sticky use cases. They run on AT&T, Windstream, RCN, Wave, Spectrum, and Frontier infrastructure. Real ISP ASNs, static IPs, and the kind of session stability that residential rotation simply can't replicate.
The CGNAT Trap: Why Some Residential Proxies Still Fail
There's a less-discussed issue with certain residential IP pools. Some residential IPs route through Carrier-Grade NAT (CGNAT, a technique where an ISP assigns a single public IP address to many customers at once instead of giving each customer their own), where a single public IP is shared by hundreds of users simultaneously. From Cloudflare's perspective, this looks exactly like datacenter traffic: one IP generating high-volume activity from many different sources.
When evaluating proxy providers, ask explicitly whether they screen out CGNAT IPs. A pool that's technically residential but loaded with CGNAT addresses will have significantly worse CF pass rates than advertised.
The Complete Bypass Stack: Code and Implementation
Level 1: Residential Proxy Only (CF Basic / Free Tier)
For sites on CF Free or basic WAF with no bot management, a rotating residential proxy through the Python `requests` library is enough. The key is getting your headers right and not sending obvious automation signals.
import requests
import random
# TorchProxies rotating residential endpoint
proxy_host = "rp.torchproxies.com"
proxy_port = "10000"
proxy_user = "YOUR_USERNAME"
proxy_pass = "YOUR_PASSWORD"
# Properly formatted proxy dictionary
proxies = {
"http": f"http://{proxy_user}:{proxy_pass}@{proxy_host}:{proxy_port}",
"https": f"http://{proxy_user}:{proxy_pass}@{proxy_host}:{proxy_port}"
}
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"Accept-Language": "en-US,en;q=0.5",
"Accept-Encoding": "gzip, deflate, br",
"Connection": "keep-alive",
"Upgrade-Insecure-Requests": "1",
}
try:
response = requests.get(
"https://target-site.com",
proxies=proxies,
headers=headers,
timeout=10 # Fail fast if the connection hangs
)
# Always validate content, not just status code
if response.status_code == 200 and len(response.content) > 5000:
print("Success")
else:
print(f"Suspicious: {response.status_code}, {len(response.content)} bytes")
except requests.exceptions.RequestException as e:
print(f"Connection Error: {e}")
Level 2: Residential Proxy + Headless Browser (CF Pro / WAF Tier)
Once a site is running CF Pro or above, IP reputation alone is not enough. The browser fingerprint becomes the primary detection vector. You need the proxy to handle the IP layer AND a headless browser to handle the fingerprint layer.
In 2026, Playwright with stealth configuration is the most maintainable choice. Puppeteer with the stealth plugin is still solid for existing codebases. Selenium works but requires more overhead to suppress fingerprint signals.
from playwright.sync_api import sync_playwright
proxy_config = {
"server": "http://rp.torchproxies.com:10000",
"username": "YOUR_USERNAME",
"password": "YOUR_PASSWORD"
}
with sync_playwright() as p:
browser = p.chromium.launch(
headless=True,
args=[
"--disable-blink-features=AutomationControlled",
"--no-sandbox",
"--disable-dev-shm-usage",
]
)
context = browser.new_context(
proxy=proxy_config,
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
viewport={"width": 1366, "height": 768},
locale="en-US",
timezone_id="America/New_York",
)
# Suppress the webdriver property that CF checks
context.add_init_script("""
Object.defineProperty(navigator, 'webdriver', { get: () => undefined });
delete navigator.__proto__.webdriver;
""")
page = context.new_page()
page.goto("https://target-site.com")
# Validate content before trusting the response
content = page.content()
if "cf-browser-verification" in content:
print("Still hitting CF challenge - upgrade proxy tier")
browser.close()
Handling Cloudflare Turnstile in 2026
Turnstile is CF's replacement for hCAPTCHA and most of the solve-based CAPTCHA approaches you've read about. The good news: managed Turnstile (the invisible version) passes automatically when you're using a quality residential IP and a real browser context. You don't need a CAPTCHA solving service.
The interactive Turnstile is a different story. It appears when your IP reputation or browser fingerprint scores too low. The fastest path to resolving it isn't a CAPTCHA solver. It's upgrading to a cleaner IP. One important note: multiple failed Turnstile attempts from the same session escalate the challenge. If you hit an interactive Turnstile, rotate the session before trying again.
Why It Stops Working: The Diagnosis Guide
This is the section that gets skipped in every other guide. You got the scraper working. Then something changed and you don't know what. Here's how to figure it out.
The "Worked Yesterday, Broken Today" Problem
Cloudflare updates its ML models silently. No changelogs, no announcements. Bot score thresholds tighten, TLS fingerprint databases expand, IP reputation lists get refreshed. What passed last week may not pass today, and you'll have no external signal that anything changed.
But here's where it gets interesting: there's a difference between your proxy getting flagged and CF tightening rules for everyone. The diagnostic approach is different for each. If you're seeing sudden failures across many different IPs in your pool, it's likely a CF-side update. If failures are isolated to specific IPs, it's pool contamination.
Failure Diagnosis: Match Your Symptom to the Fix
CF Error Codes Decoded
- 1020 - Firewall rule. Check CF response headers for the specific rule. Typically IP-based and solvable with a proxy tier upgrade.
- 1015 - Rate limited. Different from a block. Slow your request rate and check if you're hitting per-IP or per-session limits.
- 1006 - IP address banned. The specific IP is on a blocklist. Rotate to a clean IP from a different pool segment.
- 1007 / 1008 - Geographic restriction (country block) and custom Web Application Firewall rule violation respectively. For 1007, use geo-targeted proxies matching the allowed region. For 1008, the block is from a custom rule set by the site owner.
- Challenge loop - The challenge JS keeps firing without resolving. This is almost always a browser fingerprint issue, not an IP issue.
The Scale Death Spiral: What Breaks at 50+ Concurrent Sessions
This is a pattern many developers hit: Puppeteer handles around 10 portals fine. Push to 50 and IP bans hit after a couple of hours. Playwright lasts longer but the human-like flow scripting feels endless. It's not random.
When you scale concurrency, you're also scaling IP reuse frequency within the same pool. If your pool has 10,000 IPs and you're running 100 concurrent sessions, the per-IP hit rate is much higher than at 10 sessions. CF's scoring notices this pattern.
What Never Works (Save Your Time)
The bottom line? A lot of popular advice hasn't been accurate for at least 18 months. Here's the honest list of things to stop trying.
- Free proxies on CF-protected sites. The IPs carry damage from every user who abused them before you. Not worth debugging. The reputation hole is too deep.
- Cloudscraper in 2026. Still works on CF-free sites. Unreliable on CF WAF and above. CF updates broke its challenge-solving logic repeatedly in 2024 and it's not consistently maintained.
- FlareSolverr. Community-maintained, breaks when CF updates, and it updates frequently now. Good for learning the mechanics. Not production-reliable.
- VPNs for scraping. VPNs share IPs with thousands of users, many of whom have abused the same IP. You inherit all of that reputation damage. Same ASN problem as datacenter proxies, but worse.
- Header manipulation as the primary bypass. Adding realistic headers helps at the margin but does nothing against TLS fingerprinting or browser fingerprint detection. It was meaningful in 2022. It's a rounding error now.
- Reusing sessions across different domains. CF cross-domain fingerprinting will correlate sessions. Always start fresh sessions for new target domains.
How to Choose the Right Proxy Provider for Cloudflare Targets
Most proxy provider comparisons are just affiliate link roundups. This is the actual checklist of what matters when your target is behind Cloudflare specifically.
- IP pool size and refresh rate. A 150M+ pool only helps if IPs are actively rotated out when flagged. Ask about the refresh cycle.
- ASN diversity. IPs concentrated in one or two ISP networks are more easily pattern-matched. Real diversity across Comcast, AT&T, Verizon, regional ISPs means cleaner individual IP reputation.
- Active reputation monitoring. Does the provider detect and remove flagged IPs, or just sell you access and let you figure it out?
- CGNAT filtering. Explicitly ask whether CGNAT IPs are screened out of the residential pool. Many providers don't disclose this.
- Sticky session quality. For ISP proxies especially, how stable is session persistence and for how long?
- Geo-accuracy. City-level targeting for residential. State-level awareness minimum for ISP proxies.
TorchProxies Standard Residential covers 190+ countries with city-level targeting, purpose-built for CF WAF-tier scraping. TorchProxies Premium Residential adds stricter IP sourcing for demanding targets. TorchProxies ISP Proxies run on major US carrier networks for maximum trust signal and session stability. For sites running Cloudflare Bot Management at the Enterprise level, TorchProxies X Residential is the hybrid plan that combines residential depth with ISP-grade trust scoring.
Conclusion: How to Bypass Cloudflare in 2026
Cloudflare isn't one wall. It's three overlapping detection layers, and the number of active layers depends on which plan the site owner is paying for. Most scrapers fail because they're applying the right solution to the wrong tier.
The mental model that works: identify the Cloudflare tier first, then match your proxy and browser stack to it. Standard residential handles the majority of CF-protected sites without any browser layer at all. Step up to premium residential and a stealth headless browser when you hit CF Pro. For the toughest Enterprise-tier targets, X Residential combines residential IP diversity with ISP-level trust signals to push through where other proxies stall.
And validate your responses. A 200 OK is not a success. The most expensive proxy mistake in production is a scraper that runs for hours and returns nothing because CF was silently serving honeypot content the entire time.
Frequently Asked Questions
robots.txt. Collecting personal data, bypassing paywalls, or violating computer fraud laws is a different matter entirely. When in doubt, consult legal counsel for your specific use case.