pull down to refresh

At the moment and in the most recent version of commoncrawl: yes. After the 30. june 2023: no
If you want to have a taste of how it will be: try opening sites on linkedin (in your browser or Tor). Scraping is prevented easily and the API can be used for money.
reply
It's an arms race. There will be better tools for scraping, if there are not already. If these sites keep erecting walls they will end up affecting real users' UX which will backfire. So there's a limit to what they can do.
Information longs to be free and with enough determination it becomes free.
reply