Why generic Bot Fight Mode fails in production

Cloudflare's default Bot Fight Mode (on Free plans) and basic Super Bot Fight Mode (on Pro plans) are all-or-nothing switches. They evaluate automated traffic using broad behavioral signals and issue mandatory JavaScript challenges (or blocks) before any custom rules can run.

In real-world environments, this approach creates major blind spots:

  • Server-to-server API callbacks: Payment gateways (Stripe, PayPal), headless CMS webhooks, and third-party integrations do not execute JavaScript and are automatically rejected.
  • Privacy browsers and ad blockers: Users browsing under Brave, Firefox with strict tracking protection, or corporate VPNs frequently trigger false bot heuristics.
  • No bypass or exclusion rules: On the Free plan, Bot Fight Mode cannot be scoped to specific paths or exempted with custom security rules. It applies globally or not at all.
  • Incomplete crawler mitigation: Sophisticated scrapers using headless browsers easily pass standard JavaScript challenges while genuine lightweight visitors get stuck.

The result is a system that fires on the wrong requests: a legitimate webhook is rejected because it is technically automated, while a real customer on a privacy-focused browser gets stuck in a challenge loop. The same granularity problem appears when the goal is filtering AI scrapers, as explained in our guide on blocking AI crawlers while keeping Googlebot.

The solution: Granular custom security rules

Instead of relying on a blunt global toggle, effective bot protection requires custom edge rules combining several precise signals:

  • Path-targeted rate limiting: Applying strict request quotas to critical endpoints (search, login, checkout, registration) without challenging standard page views.
  • Known bot verification: Explicitly checking cf.client.bot to verify legitimate search engines while blocking unverified crawlers pretending to be Googlebot.
  • Managed Challenge over JS Challenge: Using Cloudflare Managed Challenges (Turnstile-powered) which dynamically adapt difficulty and avoid broken user sessions.
  • API route exceptions: Isolate webhook URLs with dedicated token verification or ASN/IP checks to prevent transaction drops.

This approach only addresses automated traffic. When the blocked requests come from a normal browser hitting a form or a checkout flow, the culprit is usually a managed security signature rather than bot detection, which calls for a different fix described in our guide on security rule false positives on signups and checkouts.

Fixing Bot Fight Mode false positives step by step

Here is the exact sequence to resolve a false positive on a production zone without opening a protection gap:

  1. Identify what is actually blocked. In the Cloudflare dashboard, select your domain and open Security > Events. Filter on the "Block" and "Managed Challenge" actions and locate the entries whose service is Bot Fight Mode. Note the URI path, the user agent, and the ASN of each affected request. This step separates a genuine false positive (a payment webhook, a real customer) from automated traffic you actually want to stop.
  2. Disable the global toggle on Free plans. Go to Security > Bots and switch Bot Fight Mode off. On Free, it cannot be scoped to paths or exempted with custom rules, so as long as it stays on, your own rules cannot rescue the traffic it challenges. On Pro, Super Bot Fight Mode exposes per-category responses (verified bots, definitely automated, etc.), so you can usually keep it and only retune the configuration.
  3. Rebuild targeted protection with a custom security rule. Under Security > WAF > Custom rules, create a rule that challenges suspicious traffic while letting verified bots and webhook endpoints through:
(not cf.client.bot
  and not starts_with(http.request.uri.path, "/api/webhooks/")
  and cf.threat_score ge 30)

Set the action to Managed Challenge. The field cf.client.bot matches bots verified by Cloudflare through reverse DNS, so real Googlebot and Bingbot visits are never challenged. http.request.uri.path isolates your webhook endpoints, and cf.threat_score keeps the rule conservative so ordinary page views are untouched.

  1. Add rate limiting on sensitive endpoints. Under Security > WAF > Rate limiting rules, protect login and registration with a counting expression:
(http.request.uri.path eq "/login"
  and http.request.method eq "POST"
  and not cf.client.bot)

Count above 10 requests per minute per IP and apply a Block for 10 minutes. This stops brute-force scripts and spam bots without penalizing a human who simply submits the form twice.

  1. Exempt trusted server-to-server sources. Payment providers and monitoring services send callbacks from a small set of autonomous systems. Add a condition on ip.src.asnum matching your provider's ASN to a custom rule or a Skip rule, so their requests are never challenged regardless of their user agent.
  2. Verify over the next 24 to 48 hours. Revisit Security > Events the following day: the previously blocked legitimate paths should no longer appear, while blocked automated requests should still be counted. Adjust the cf.threat_score threshold only if the log shows the rule is now too permissive or still too strict.

Common mistakes that keep false positives alive

  • Switching Bot Fight Mode off with nothing to replace it. The complaints stop, but every scraper and credential-stuffing script now reaches your origin unchallenged. Always deploy the replacement rules before disabling the toggle.
  • Writing Skip rules that are too broad. Exempting every path containing /api/ feels safe, but most abusive automation also lives under API paths. Scope each exemption to exact paths and HTTP methods.
  • Filtering on the User-Agent string alone. Any script can send a Googlebot user agent. Only cf.client.bot is verified; a raw UA match blocks nothing serious and can block the wrong visitor.
  • Leaving challenges on webhook endpoints. External services never solve JavaScript challenges. A challenged webhook fails silently, and you discover the broken integration days later in a reconciliation report.
  • Tuning once and never looking back. Traffic patterns change: a rule tuned months ago can start challenging a new mobile browser or a new integration. A short monthly review of Security > Events keeps the rules aligned with real traffic.

Frequently asked questions

Why do only some visitors get stuck in challenge loops?

Bot Fight Mode scores each request against behavioral and network signals. Visitors on mainstream browsers with residential IPs usually pass without noticing, while privacy browsers, strict anti-tracking settings, corporate VPNs, and mobile networks with shared IP addresses trip the heuristics. The users who complain are not imagining it: they simply match the profiles the heuristic distrusts the most.

Does this work on a Free plan, or do I need to upgrade?

The core of the fix works on Free: custom security rules, Managed Challenges, and a free rate limiting allowance are all available. What Free lacks is any way to scope or configure Bot Fight Mode itself, which is why the recommended path is to disable the toggle and rebuild protection with your own rules. A Pro plan adds finer bot visibility and Super Bot Fight Mode configuration, but it is not required to stop the false positives.

Is it risky to disable Bot Fight Mode?

Not if you replace it first. The false positives disappear because the blunt heuristic is gone, but malicious automation returns just as fast if nothing is put in its place. The sequence above exists precisely to keep a safety net: verified bots still pass, suspicious traffic still faces a Managed Challenge, and login and signup endpoints stay rate limited.

How long does it take for the fix to take effect?

Rule changes propagate to the Cloudflare edge within seconds, so blocked users and webhooks recover immediately. The verification phase takes longer: watch Security > Events for 24 to 48 hours to confirm the new rules keep catching the automated traffic the old toggle was filtering. If a payment provider is still rejected after that window, identify its exact ASN and add it to the exemption.

Need expert help fixing bot filtering without breaking users?

CF Garage audits your security events, diagnoses false positives, and implements custom security & rate limiting rules tailored to your site. Fixed price, guaranteed results.

View our offers

Related topics