Method
The builder fetches https://packrift.com/robots.txt and https://packrift.com/sitemap.xml live, parses the sitemap index with an XML parser, classifies child sitemap URLs by public type, and reduces robots rules to category counts.
Raw upstream robots exclusion paths are not copied into this public gateway. The published sample rows contain source status rows, child sitemap references, public agent route hints, and category totals.
Count Treatment
| Bucket | Treatment |
|---|
- Owned Public Url Mentions: Only this gateway site's sitemap URLs are new owned public URL/mention movement.
- Referenced Existing Urls: Existing Packrift.com source documents and child sitemaps are referenced for discovery and are not re-counted here.
- Third Party Referring Domains: No third-party referring-domain movement is claimed by this owned gateway.
- Indexing Receipts: IndexNow receipts are discovery-only if accepted.