Sites are already writing to AI agents through HTTP headers
Signals Corps read the response headers of 217,097,223 pages, 10% of Common Crawl's September 2026 crawl, and found 7,959 sites sending a header written for an AI system. Of those, 7,271 sent pointers to resources such as llms.txt files…