导航|地址|刷新
Posts tagged "Bots"

Skip to content

All Categories

AI

Developers

Radar

Product News

Security

Policy & Legal

Zero Trust

Speed & Reliability

Life at Cloudflare

Partners

Switch Site Language

English

Deutsch

Español

Español (Latinoamérica)

Français

Italiano

日本語

 한국어

繁體中文

 简体中文

 Português

Русский

Bahasa Indonesia

ภาษาไทย

Tiếng Việt

Polski

العربية

עברית

Svenska

Nederlands

Türkçe

Login Contact Sales

TAG

Bots

July 1, 2026
Unmasking the crawls with Attribution Business Insights

Cloudflare’s new Attribution Business Insights dashboard helps website owners understand crawler behavior, appetite, and potential value, fueling business-level conversations around crawl compensation.

Jin-Hee Lee and Oliver Payne

July 1, 2026
Announcing the Monetization Gateway: charge for any resource behind Cloudflare via x402

We're opening the waitlist for our Monetization Gateway, which will allow you to charge for any web page, dataset, API, or MCP tool behind Cloudflare. The charges will settle in stablecoins over the x402 open protocol, with no payments stack of your own to build.

Rohin Lohe,Justin Ridgely, and Will Papper

July 1, 2026
Content Independence Day, one year on: building the business model for the agentic Internet

One year after declaring Content Independence Day, a dynamic market for monetized content has officially emerged. In this report, we examine how the rise of autonomous AI agents is upending traditional search referrals and detail the new infrastructure required to support a sustainable web economy.

Arielle Weiss,Zach Albertson, and Emily Lanfear

April 21, 2026
Moving past bots vs. humans

As AI assistants and privacy proxies challenge the capabilities of traditional bot detection, the Web needs new models for accountability. We believe that control should remain with the client, and that an open ecosystem of anonymous credentials is key to preserving user privacy while protecting origins from abuse.

Thibault Meunier

February 27, 2026
The most-seen UI on the Internet? Redesigning Turnstile and Challenge Pages

We serve 7.6 billion challenges daily. Here’s how we used research, AAA accessibility standards, and a unified architecture to redesign the Internet’s most-seen user interface.

Leo Bacevicius,Ana Foppa, and Marina Elmore

October 30, 2025
Beyond IP lists: a registry format for bots and agents

We propose an open registry format for Web Bot Auth to move beyond IP-based identity. This allows any origin to discover and verify cryptographic keys for bots, fostering a decentralized and more trustworthy ecosystem.

Thibault Meunier and Maxime Guerreiro

October 29, 2025
One IP address, many users: detecting CGNAT to reduce collateral effects

IPv4 scarcity drives widespread use of Carrier-Grade Network Address Translation, a practice in ISPs and mobile networks that places many users behind each IP address, along with their collected activity and volumes of traffic. We introduce the method we’ve developed to detect large-scale IP sharing globally and mitigate the issues that result.

Vasilis Giotsas and Marwan Fayed

September 29, 2025
15 years of helping build a better Internet: a look back at Birthday Week 2025

Rust-powered core systems, post-quantum upgrades, developer access for students, PlanetScale integration, open-source partnerships, and our biggest internship program ever — 1,111 interns in 2026.

Nikita Cano and Korinne Alpers

September 23, 2025
Building unique, per-customer defenses against advanced bot threats in the AI era

Today, we are announcing a new approach to catching bots: using models to provide behavioral anomaly detection unique to each bot management customer and stop sophisticated bot attacks.

Jin-Hee Lee,Oliver Payne,Bob AminAzad,Viktor Chynarov,Aleksandar Pavlov Hrusanov, and Prajjwal Gupta

August 29, 2025
The crawl-to-click gap: Cloudflare data on AI bots, training, and referrals

By mid-2025, training drives nearly 80% of AI crawling, while referrals to publishers (especially from Google) are falling and crawl-to-refer ratios show AI consumes far more than it sends back.

João Tomé

August 28, 2025
A deeper look at AI crawlers: breaking down traffic by purpose and industry

We are extending AI-related insights on Cloudflare Radar with new industry-focused data and a breakdown of bot traffic by purpose, such as training or user action.

David Belson

August 28, 2025
The next step for content creators in working with AI bots: Introducing AI Crawl Control

Cloudflare launches AI Crawl Control (formerly AI Audit) and introduces easily customizable 402 HTTP responses.

Will Allen,Pulkita Kini, and Cam Whiteside

August 28, 2025
The age of agents: cryptographically recognizing agent traffic

Cloudflare now lets websites and bot creators use Web Bot Auth to segment agents from verified bots, making it easier for customers to allow or disallow the many types of user and partner directed.

Jin-Hee Lee

August 18, 2025
Announcing the Cloudflare Browser Developer Program

Announcing the Browser Developer Program: Cloudflare’s new collaborative program to help shape Cloudflare challenges that work seamlessly with your browser. Join us today!

Sally Lee and Oliver Payne

August 4, 2025
Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives

Perplexity is repeatedly modifying their user agent and changing IPs and ASNs to hide their crawling activity, in direct conflict with explicit no-crawl preferences expressed by websites.

Gabriel Corral,Vaibhav Singhal,Brian Mitchell, and Reid Tatoris

July 1, 2025
Introducing pay per crawl: Enabling content owners to charge AI crawlers for access

Pay per crawl is a new feature to allow content creators to charge AI crawlers for access to their content.

Will Allen and Simon Newton

July 1, 2025
Message Signatures are now part of our Verified Bots Program, simplifying bot authentication

Bots can start authenticating to Cloudflare using public key cryptography, preventing them from being spoofed and allowing origins to have confidence in their identity.

Mari Galicer,Akshat Mahajan,Gauri Baraskar, and Helen Du

July 1, 2025
Control content use for AI training with Cloudflare’s managed robots.txt and blocking for monetized content

Cloudflare is making it easier for publishers and content creators of all sizes to prevent their content from being scraped for AI training by managing robots.txt on their behalf.

Jin-Hee Lee,Dipunj Gupta,Brian Mitchell,Reid Tatoris, and Henry Clausen

Load more

Subscribe to receive notifications of new posts

Email address

We’ll never share your email address.

Subscribe

Thanks for subscribing! Check your inbox to confirm.

Getting Started

Plans

Contact sales

Partners

Find a partner

Startups

Under attack?

Domain name search

Public interest

Project Galileo

Athenian Project

Cloudflare for Campaigns

Project Fairshot

Impact/ESG

Resources

App innovation report

Cloudflare Radar

Case studies

Status

Support

Events

Blog

Solutions

SSE and SASE platform

Cloudflare AI Cloud

AI Security

Frontend Development Platform

Multi-Tenant Platform Development

Web Security Platform

Company

About

Careers

Investors

Press

Press kit

Global network

Compliance

Compliance resources

Trust

GDPR

Responsible AI

Transparency report

Report abuse

Developers

Documentation

Learning center

Community

Start building

Login

© 2026 Cloudflare, Inc.

|Your privacy choicesReport security issues|Privacy Policy|Terms of use|GDPR|Trademark

Search is temporarily unavailable.

Products

Solutions

Resources

Pricing

Login opens in a new tabContact Sales opens in a new tabStart Building opens in a new tab

opens in a new tab opens in a new tab opens in a new tab

All Categories

AI

Developers

Radar

Product News

Security

Policy & Legal

Zero Trust

Speed & Reliability

Life at Cloudflare

Partners

English

Switch Site Language

English

Deutsch

Español

Español (Latinoamérica)

Français

Italiano

日本語

한국어

繁體中文

简体中文

Português

Русский

Bahasa Indonesia

ภาษาไทย

Tiếng Việt

Polski

العربية

עברית

Svenska

Nederlands

Türkçe

LightDark

------
导航页打开网址