Method: 17 days of edge logs, every self-identifying AI crawler checked against the IP range its operator publishes. GPTBot, ClaudeBot, GoogleOther verify 97 to 100 percent, so blocking them works. But one rented /24 wore 14 different AI companies' user-agents, 65 percent of "Amazonbot" was generic EC2 failing Amazon's own reverse-DNS check, and Meta and ByteDance publish no verification method at all. The user-agent is a costume. robots.txt stops the polite, not the ones faking the name. Raw per-agent tally ships with the piece.
Method: 17 days of edge logs, every self-identifying AI crawler checked against the IP range its operator publishes. GPTBot, ClaudeBot, GoogleOther verify 97 to 100 percent, so blocking them works. But one rented /24 wore 14 different AI companies' user-agents, 65 percent of "Amazonbot" was generic EC2 failing Amazon's own reverse-DNS check, and Meta and ByteDance publish no verification method at all. The user-agent is a costume. robots.txt stops the polite, not the ones faking the name. Raw per-agent tally ships with the piece.