AI & Automation

A US Court Just Legalized AI Agents Browsing Your Site. On September 15, Cloudflare Will Block Them for You

2026.08.06 · 63 views
A US Court Just Legalized AI Agents Browsing Your Site. On September 15, Cloudflare Will Block Them for You

On August 4 the Ninth Circuit vacated Amazon's injunction against Perplexity's Comet, ruling that users, not the vendor, access a website. The legal gate is open. What actually decides whether agents get in is your CDN defaults and the 66% machine-readability score of your product pages.

Share:

A 21-Page Opinion Just Redefined Who Accesses Your Website

On August 4, 2026, the US Court of Appeals for the Ninth Circuit vacated a district court injunction against Perplexity. In a 21-page opinion, the panel wrote a sentence that matters to every site owner: it is the user who accesses Amazon's computers, and Comet's Assistant is merely a tool helping them carry out specific acts. The plaintiff was Amazon; the defendant was Perplexity, valued at roughly $21 billion. The battleground was the unauthorized-access prong of the Computer Fraud and Abuse Act.

This litigation has run since November 2025, and the timing is no accident. Adobe Digital Insights, which tracks over 1 trillion visits to US retail sites, reports that AI-sourced traffic to US retail grew 393% year over year in Q1 2026, after a 693% jump during the November-December 2025 holiday window. Conversion matters more: in March 2026 AI traffic converted 42% better than non-AI traffic, versus March 2025, when it converted 38% worse. Worst to best in twelve months. Once a channel starts checking out, platforms fight over the doorway. Amazon won its injunction on March 9, 2026, arguing agents must identify themselves, in substance an attempt to keep purchase intent inside its own walls.

Perplexity is not the only player. The Agentic Commerce Protocol from OpenAI and Stripe is already processing live transactions through ChatGPT's Instant Checkout, while Google has the AP2 payments protocol and agentic capabilities in Gemini for Chrome. All three compress "search, click, check out yourself" into "say one sentence, let the agent do it." The difference is route: OpenAI and Google negotiate with merchants through protocols, Perplexity routes around platform authorization through a browser. This ruling tells the market the second route survives CFAA scrutiny, for now.

For Taiwanese SMB owners and freelance developers, the significance is not legal, it is structural. Your site is about to receive a visitor that does not scroll, does not look at images, does not execute JavaScript, and reads only HTML. It is carrying a credit card. Below: how it reads your pages, what Cloudflare's September 15 defaults will do to you, and the four zero-cost things to fix first.

What the Ruling Decided, and What It Did Not

Circuit Judge Milan Smith Jr. wrote the opinion. The core holding is narrow: Perplexity may receive screenshots of a user's browser and may issue instructions to the Assistant, but those acts do not mean it gained entry to Amazon's servers. The California CDAFA claim fell on identical logic. The EFF amicus brief was singled out as articulating the nature of the system most clearly. But the range is deliberately short: the panel limited its holding to the record before it and said it was not establishing a new legal regime for agentic AI. Amazon's trademark and state-law claims survive, and the case returns to the district court. The law did not hand agents a permit; it declined to use a criminal statute to stop them.

The number that should worry you sits on the other side. Adobe's AI Content Visibility Checker benchmarked US retail and found machine-readability of 75% for homepages, 74% for category pages, 80% for FAQ pages, 82% for returns pages, and 66% for individual product pages. Best-performing brands hit 82.5% on homepages; the worst scored 54.2%. So while agentic traffic grew 393%, a third of the content on the pages that most need to be understood is invisible to machines. The court opened the door. The wall behind it is one you built.

Immediate Actions for Three Audiences

Brand owners and SMB operators

  • Check your CDN settings. In its July 1, 2026 changelog, Cloudflare split AI traffic into Search, Agent, and Training controls and announced that from September 15, 2026 new domains onboarding to Cloudflare get updated defaults: Training and Agent blocked on pages that display ads, Search still allowed. All customers can opt out before that date.
  • Make "are AI agents customers?" an explicit written decision, not something an engineer improvises in a WAF rule.
  • Price, stock, shipping cost, and return terms must be readable with JavaScript disabled. Those four fields are the entire basis on which an agent compares you.

Marketing and SEO practitioners

  • Build a GA4 custom channel group with regex capturing referrals from chatgpt.com, perplexity.ai, gemini.google.com, and copilot.microsoft.com. Get a baseline before optimizing anything.
  • Convert product spec sheets from images to HTML <table>. Agents do not read images; a spec graphic is a blank space.
  • Returns, warranty, and lead-time pages already score highest sector-wide (around 82%), so competitors do them well too. Spend the effort on product pages.

Developers and agencies

  • Run curl -s https://yoursite | sed 's/<[^>]*>//g' and read what survives. It diagnoses a client site in five seconds.
  • SPA projects need SSR or prerendering. A purely client-rendered product page is close to a blank sheet to an agent.
  • Make Product and Offer structured data a standard deliverable, including price, availability, shippingDetails, and hasMerchantReturnPolicy.

Tool Comparison: Who Sells Agent Readability

ToolPositionPricing modelBest forLimitation
Cloudflare AI traffic controlsAllow or block by Search / Agent / TrainingAvailable on the Free planAnyone already on CloudflareControls entry only, not page quality
Adobe LLM OptimizerScans machine-readability, suggests rewritesEnterprise, quote-basedRetail brands with thousands of SKUsOverkill for a ten-page brochure site
ProfoundTracks brand citations across AI enginesEnterprise subscriptionBrands with dedicated marketing teamsMeasurement only; changes nothing on site
DIY script + schemaPlain-text extraction plus structured dataLabor cost onlyAgencies and technical SMBsNo dashboard; you schedule the runs

What Nobody Tells You

  • The "42% better conversion" figure carries severe attribution bias. People clicking through from an AI conversation are already at the last mile of a purchase decision, a tiny denominator of extremely high intent. That is a selection effect, not proof the channel is stronger. Using it to move an entire budget will be hard to defend six months later.
  • Agent traffic is nearly invisible in GA4. Agents mostly run headless, do not execute GA4's JavaScript, and often send no referrer. The "AI referral" in your reports is only the tip where a human manually clicked. The truth lives in server logs and CDN bot analytics.
  • What won here was not openness, it was time for Amazon to switch weapons. Losing on the CFAA leaves terms of service, rate limits, device fingerprinting, and login walls fully intact, none of which need a court's permission. The next battleground moves from the courtroom to robots.txt and WAF rules.

The No-Subscription Alternative for SMBs

Four steps, one weekend, zero cost:

  • Step 1: measure the baseline. Fetch raw HTML for your homepage and three representative product pages, strip the tags, and check whether price, stock, and specs survive. Anything missing is your first ticket.
  • Step 2: add structured data. Product plus Offer on product pages, Service on service pages, Organization and FAQPage sitewide. Validate with the Rich Results Test rather than trusting your eyes.
  • Step 3: state a policy. Handle search, agent, and training crawlers separately in robots.txt instead of one blanket User-agent: *. Cloudflare's free plan already exposes three independent switches.
  • Step 4: build visibility. A GA4 custom channel group plus a monthly count of AI user-agents in server logs replaces most entry-level SaaS dashboards.

Frequently Asked Questions

Does a Ninth Circuit ruling have any force for a Taiwanese website?

No direct legal force, but the practical effect is real. Perplexity, OpenAI, and Google make product decisions inside the US regulatory environment, and when the legal brake releases, agent features ship globally. Taiwanese sites receive that traffic regardless. Prepare the technical layer, not the legal one.

Should I block AI agents or not?

Separate the three behaviors first. Training crawlers take your content to train models and return almost nothing, so blocking them is reasonable. Search crawlers bring citations and clicks, so blocking them cuts your own supply line. Agents act for real humans and currently convert best of all, so blocking them locks customers outside. Use three independent switches instead of one blanket block.

On a product page scoring 66%, what is the missing 34% usually made of?

Based on Adobe's analysis and field experience, four patterns dominate: specs rendered inside images, prices injected by JavaScript, reviews inside an iframe, and stock status conveyed only by color or icon. Convert all four to plain HTML text and scores typically climb past 80%.

Will Cloudflare's September 15 defaults affect my existing site?

The official changelog states that new domains onboarding to Cloudflare receive the updated defaults, and also that all customers can opt out before September 15. That sentence alone is reason enough to check your dashboard before the deadline, especially if your pages display ads.

My Take

The market reads GEO as "monitor my visibility inside AI," and a crowd of tools is selling dashboards. My read is the opposite: over the next 12 to 18 months, AI visibility monitoring tools will deflate first, while agent-readability remediation becomes the work that actually gets paid for.

The reasoning is direct. The measurement layer is being pulled back by the platforms; in early August Google began requiring sign-in to reveal more results, which already distorted a batch of visibility reports. A moat built on someone else's servers can be drained without notice. Machine-readability of your own pages cannot be revoked by anyone. It is an asset on your domain, it has a public benchmark to measure against, and it can be written into a contract. A deliverable that can be signed off always sells better than a report that changes every month.

For an agency like ScriptWalker there is a ready-made productization path: package an Agent Readiness Audit as a fixed-price, one-off engagement. Plain-text extraction comparison, structured data completion, SSR assessment, a three-tier crawler policy in robots.txt, and GA4 AI-source segmentation, delivered as a report with before-and-after scores. No long-term contract, no dashboard to maintain, and every time Google or Cloudflare changes a rule it generates a return visit.

Contact Us

If you are not sure what your product pages look like to an agent, or which settings to adjust before September 15, reach out:

Sources

Share:
AI & Automation Back to Blog