Test sites behind Cloudflare without switching bot protection off

Cloudflare's bot protection exists to stop automated browsers, and an AI testing agent is an automated browser. When a test tool opens a site behind Cloudflare, the request can be blocked or answered with a challenge page. The test tool then explores a challenge page instead of your app, and every test that follows fails for a reason that has nothing to do with your code.

The usual workarounds weaken the protection you put there on purpose: turning bot protection off for staging, allowlisting IP ranges that change, or trusting a User-Agent string that anyone can copy. TestOptim AI takes a different route and, with Protected by Cloudflare on, proves who it is on every request.

With Protected by Cloudflare on, TestOptim signs every request its test browser makes, so Cloudflare recognizes it as a verified bot rather than an unknown one. The signatures follow Web Bot Auth, which is HTTP Message Signatures (RFC 9421) with an Ed25519 key. Each signed request carries three headers:

Signature-Agent: "https://api.testoptim.com"
Signature-Input: …; tag="web-bot-auth"   (covers @authority and signature-agent)
Signature: …

The public keys are published at https://api.testoptim.com/.well-known/http-message-signatures-directory. A request comes from TestOptim only if its signature verifies against one of those keys, so nobody can pass as TestOptim by copying a header. The TestOptim Bot page lists the same details for your security team.

Signing is controlled by a Protected by Cloudflare switch on each project, and you usually don't have to touch it. Shortly after you create a project, TestOptim loads your site once in the background from its servers. If your site, or an API it calls on the same domain, answers through Cloudflare, TestOptim turns the switch on for that project.

Detection takes about a minute and can be skipped during busy periods. If you start an exploration straight after creating the project, that first run may begin before detection finishes and won't be signed. If you already know your site challenges bots, turn the switch on yourself when you create the project, or later under Settings. Detection only ever turns the setting on; if you switch it off, it stays off.

Projects that use the TestOptim CLI tunnel aren't checked, because TestOptim reaches them only through the tunnel. See testing localhost and private-network apps.

A signature lets Cloudflare tell TestOptim apart from other bots, but your own firewall rules still apply. If TestOptim is still blocked with the switch on, add a Cloudflare WAF custom rule that allows verified bots or signed agents. Outside Cloudflare, verify the signature against the key directory above.

Don't write rules on the User-Agent. TestOptim's browser sends a standard Chrome string, so the user agent alone can't tell its traffic apart from anyone else's. Match the signature instead.

Once Cloudflare lets it in, TestOptim treats your site like any other project. The exploration agent maps your pages and flows, test generation writes end-to-end test cases from what it found, and test runs carry them out in a real browser and file each bug with reproduction steps and the expected and actual result.

If exploration still can't get in, check that the switch is on and saved and that no other firewall or rate-limit rule blocks the traffic. If you can't allow it, point the project at a staging environment that isn't protected.