Anubis: scraper defense - "weigh the soul of incoming http requests using PoW" \ stacker news ~security

Anubis weighs the soul of your connection using a sha256 proof-of-work challenge in order to protect upstream resources from scraper bots.

This program is designed to help protect the small internet from the endless storm of requests that flood in from AI companies. Anubis is as lightweight as possible to ensure that everyone can afford to protect the communities closest to them.

Anubis is a bit of a nuclear response. This will result in your website being blocked from smaller scrapers and may inhibit "good bots" like the Internet Archive. You can configure bot policy definitions to explicitly allowlist them and we are working on a curated set of "known good" bots to allow for a compromise between discoverability and uptime.

I suspect PoW defenses to increase a lot in popularity. CAPTCHAs are at end of life.

30 sats \ 1 reply \ @SimpleStacker 15h

I couldn't catch how this stops AI. Can't an AI bot do the POW?

Or is it more the idea that if this was widely deployed, it would become very costly for bots to scrape?

40 sats \ 0 replies \ @k00b OP 15h

I think the idea is less motivated bots will timeout loading the page. They probably defend themselves from pages/sites that take a lot of time and resources to load. Also, most crawlers afaik don't expect to encounter big CPU loads.

But you're right, one site will only slow a bot down which won't stop them.