Apple Quietly Grew the Applebot IP Pool by 194%: Your Allowlist Now Blocks Two-Thirds of It

At 00:49 on 17 August 2026, a monitoring script run by the London consultancy Merj caught a change nobody announced: the CIDR file Apple publishes so sites can verify its crawler gained 21 network blocks in one pass. Eighteen /24s plus three /28s comes to 4,656 new IP addresses. A pool holding 2,400 addresses across twelve prefixes now holds 7,056 across thirty-three — an increase of 194 percent. Apple published no blog post, no press release, and did not amend a word of its support documentation.

This is not a sysadmin footnote, because Applebot changed jobs this year. On 12 January, Apple and Google put Gemini models underneath the next generation of Apple Foundation Models, reportedly for around a billion dollars a year. At WWDC26 on 8 June Apple confirmed Siri AI runs on Gemini, and the same day rewrote its About Applebot documentation to state in plain text that crawled content may train Apple foundation models and may serve as additional context when AI models generate output. Applebot has gone from indexing for Spotlight and Safari suggestions to supplying grounding data for an answer engine fielding well over a billion requests a day. Those jobs have opposite fetch profiles: an index can crawl slowly and lean on cache, while an answer engine needs freshness and concurrency. Allocating 4,656 addresses in one pass, all inside a single 17.166.0.0/16, is what an operator does before parallel fetching increases.

The comparison set is unambiguous. Cloudflare measurement from January showed Googlebot reaching 14.87 times more unique URLs than Applebot; Apple has always been the smallest of the major crawlers. OpenAI has run address ramps before model releases, but Merj says none matched this scale, and Apple had left this list untouched for years. Meanwhile Google moved its crawler IP files to a daily refresh in March 2025. Apple ships a file with no version number, no changelog, and an internal creationTime still reading 31 July — a fortnight before the ranges surfaced. The category is moving toward auditable. Apple is moving the other way.

Does this reach small businesses? Yes, starting in the least glamorous place: firewall allowlists, WAF rules and log parsers. If any were written against those twelve prefixes, they now recognise one third of what Apple advertises. What follows covers what changed, what three groups should do today, and why I think this expansion produces no measurable visits for you in the next twelve months.

What Changed: 33 Prefixes, 7,056 Addresses, One Timestamp That Does Not Fit

Apple offers two ways to identify Applebot. Reverse DNS, where the address resolves to a hostname inside applebot.apple.com and forward-resolves back. And a match against a CIDR prefix in the published JSON file. Only the second changed.

The old twelve prefixes sat in three unrelated neighbourhoods — 17.22.x, 17.241.x and 17.246.x — the signature of capacity accumulated across facilities over years: nine /24s giving 2,304 addresses plus three /27s giving 96. The twenty-one new prefixes look nothing like that. All fall inside 17.166.0.0/16 and arrive in tidy runs: six consecutive /24s from 17.166.20.0 to 25.0, six more from 150.0 to 155.0, six more from 232.0 to 237.0, plus three /28s. That is a single allocation carved out in one pass, not something that grew.

Two details deserve to stick. First, address count is not crawl capacity — one machine can serve from many addresses. The number establishes intent to spread requests across a wider surface, which is a signal about concurrency, not a promise about volume. Second, and more awkward: the address Apple uses to demonstrate reverse DNS in its own documentation, 17.58.101.179, falls inside none of the thirty-three published prefixes. Apple's own worked example of legitimate crawler traffic would fail a strict check against Apple's own list. The file has never been an exhaustive inventory, and any rule treating it as ground truth misclassifies traffic in both directions.

While you are in there: Applebot does not honour crawl-delay, and where robots.txt says nothing about Applebot but does address Googlebot, it follows the Googlebot instructions. A large share of sites are governing Apple's crawler with rules written for somebody else.

Immediate Actions for Three Kinds of Reader

Brand owners and SMB operators

• Ask whoever runs your infrastructure one question today: how many Apple prefixes are in our allowlist? If the answer is twelve, you are missing two thirds.
• Look at cost before exposure. Answer-engine crawling is bandwidth you pay for. Check your hosting ceiling and overage billing first.
• If you run a paywall, mark pages isAccessibleForFree: false. The page stays eligible for search results while its content is excluded from use as context in AI-generated output.

Marketing and SEO practitioners

• Write explicit Applebot rules into robots.txt. Do not let it inherit your Googlebot block — the commercial meaning of the two crawlers is completely different.
• Keep three independent controls straight: robots.txt governs whether it crawls, Applebot-Extended governs whether the data trains foundation models, nosnippet governs whether content becomes context for AI answers. Apple states plainly that opting out of training is not opting out of the index.
• There is no Search Console for Apple — no query-level reporting, no impressions, no CTR. Server logs are your only evidence. Start retaining them now.

Developers and agencies

• Replace CIDR matching with two-way DNS verification: reverse-resolve the IP, confirm the hostname ends in applebot.apple.com, then forward-resolve and compare against the original address. This is the only method covering unpublished addresses.
• Ship a cron job that pulls the CIDR JSON daily and commits a sorted diff to version control. Do not key change detection off creationTime — this time it misled everyone.
• If client sites sit on Cloudflare, note that from 15 September new domains get defaults that block Training-classified crawlers on pages carrying advertising, with Applebot named explicitly. Capacity opens in August, a default block arrives in September. Two forces pulling opposite ways.

Tool Comparison: Who Can Actually See Apple?

The column that matters is Apple coverage. The AI visibility subscription costing several hundred dollars a month is blind to an assistant surface about to run on Gemini and field over a billion requests a day.

What Nobody Is Telling You

• The person saying "let Applebot in" sells crawler monitoring. The "something big might be about to happen" line came from Merj's Ryan Siddle, and the alert that produced it is Merj's product. The observation is accurate; the conclusion is an inference. Address expansion establishes intent about capacity. It does not establish that fetch volume rises, and it certainly does not establish that anyone clicks through.
• Opening the gate has a bill attached. The attribution dashboard Cloudflare opened on 1 July showed crawl-to-referral ratios running from 118:1 to nearly 50,000:1. On the other side, Rutgers and Wharton research revised in April 2026 found publishers who blocked LLM crawlers via robots.txt lost roughly 7 percent of weekly traffic within six weeks. Neither direction is free. There is no safe default.
• The rules you set are not always honoured. TollBit measurement published 14 August 2026 found roughly 15 percent of identified page-fetching agents in Europe reaching URLs their operators had been told to avoid. robots.txt was never an enforcement mechanism for live agents.
• The free-citation window is closing. The ad dataset Similarweb opened on 17 August shows 26 percent of ChatGPT responses already carrying a sponsored placement, with close to 30 percent of ad-eligible Google AI Mode queries showing ads. The surfaces you feed today will sell the slot tomorrow.

The No-Subscription Alternative

• Daily diff. A cron job pulls applebot.json, sorts the prefix array, commits it to Git and emails on any difference. Ignore the creationTime field.
• Two-way DNS. In the log pipeline, reverse-resolve suspect addresses, confirm the hostname ends in applebot.apple.com, then forward-resolve back to the original IP. Only a match on both sides counts, and it covers addresses Apple never published.
• Compute the crawl bill. Use GoAccess or awk to pull daily Applebot request counts, bytes transferred and status-code distribution from the access log, then set them beside human sessions for the same period. That table drives your hosting and rate-limiting decisions.
• Hard-code the policy matrix. List Applebot and Applebot-Extended as separate blocks in robots.txt, mark paid content isAccessibleForFree: false, add nosnippet to pages that should not become AI context, and handle PDFs and images via the X-Robots-Tag response header. Section-level markup using hasPart is not supported.

Frequently Asked Questions

Should I let Applebot in, or block it?

Allow by default, but rate-limit and monitor. The asymmetry is the argument: the cost of blocking is measured and certain — Rutgers and Wharton found roughly 7 percent of weekly traffic lost within six weeks — while the cost of allowing is bandwidth you can measure and cap. The crawlers worth blocking are the ones with a persistently poor crawl-to-referral ratio and no conversions, and that call comes from your own logs.

Does blocking Applebot-Extended mean opting out of the Apple index?

No. Apple's documentation is explicit: even with Applebot-Extended disallowed and content tagged nosnippet, if robots.txt still permits Applebot to crawl, the content remains discoverable through Spotlight, Siri, Safari and other system-wide features. Opting out of training is not opting out of the index.

Why does an allowlist built from the official IP file still misfire?

Because the file is not an exhaustive inventory. The address Apple uses to demonstrate reverse DNS in its own support page, 17.58.101.179, is absent from all thirty-three published prefixes. Pure CIDR matching flags legitimate requests as impostors and misses genuine ones. Two-way DNS verification is the correct control.

Will the Cloudflare default on 15 September block Applebot automatically?

It depends on the account. New domains onboarding, new customers and Free-tier users get the new defaults, which block Training and Agent-classified crawlers on pages carrying advertising. Applebot is named because Apple states crawled data may train its foundation models, placing it in the multi-purpose bucket. Existing paying customers can override in the dashboard. Check before 15 September, not after traffic moves.

Is there any tool that shows whether Siri cited my site?

No. There is no Apple equivalent of Search Console — no query-level data, no impressions, no click-through rate. The only evidence is Applebot fetch records in your server logs plus occasional Apple referrers. If a vendor claims a Siri visibility score, ask where the data comes from before you sign.

My Take

The consensus reading is "Apple is entering AI search, go claim your visibility." My call is the opposite: this expansion produces no measurable traffic for ordinary sites in the next twelve months. Not because Apple cannot build it, but because the destination is structurally non-referring. The Siri answer surface has no ten blue links, no pagination, no Search Console, and Apple decides whether a cited source gets a link at all. You are feeding a black box driven by a competitor's model that discloses nothing back to you. Resourcing this as a land grab buys you a 2027 report with no denominator in it.

The correct frame is security and cost, not marketing. Three things are measurable today: who is crawling, how much, and what it costs you. All three can be instrumented this week without subscribing to anything.

For an agency like ScriptWalker, the opening is a small product nobody is selling: a crawler allowlist audit. Deliver four artefacts as a fixed-scope project — a gap report between the client's current allowlist and every operator's published IP file, a two-way DNS verification script, a robots / nosnippet / X-Robots-Tag policy matrix, and a crawl-cost table built from the client's own logs. Then attach a monthly retainer for diff alerts and quarterly reporting. The barrier is not technical skill; it is that nobody wants to spend an afternoon reading a JSON file Apple ships without a changelog. Cloudflare's 15 September default is about to manufacture a cohort of clients who need this explained. To outsource the work:

• Email: [email protected]
• Phone: 0916-224-047
• LINE: @ufv9089p

Sources

• First-party: Apple — About Applebot (updated 8 June 2026)
• First-party: Apple — Applebot CIDR verification file, applebot.json
• First-party: Ryan Siddle (Merj) — original LinkedIn post and alert screenshot
• First-party: Similarweb Investor Relations — AI Ads dataset launch (17 August 2026)
• First-party: Cloudflare Blog — AI traffic options for all customers
• Third-party: Search Engine Roundtable (17 August 2026)
• Third-party: PPC Land — Apple adds 4,656 IP addresses to Applebot crawler
• Third-party: PPC Land — Apple rewrites Applebot rules to feed Siri AI
• Third-party: CNBC — Apple picks Google Gemini to run AI-powered Siri
• Third-party: TechCrunch — Cloudflare policy and the 15 September default block