Note by @fastfinge

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

1d

Tip: you can use fail2ban to ban crawlers. They often visit random likely urls, generating either 404 or 500 errors. A human will not generate 5-10 404 errors on your website per hour. Once it gets several an hour, ban the IP for 24 hours. Nukes stupid crawlers and scanners quickly, while requiring zero extra work from humans, and not screwing up accessibility in any way. Make sure, of course, all the links on your website don't generate 404 page not found or 500 server errors. But if you have so many broken links that real humans are getting banned by this, you need to fix your broken website anyway.

3

4

2

0

the esoteric programmer @esoteric_programmer@social.stealthy.club

1d

@fastfinge I'm using crowdsec for that nowadays, but the end game is much the same indeed, except crowdsec does a lot of that automatically

1

0

🇨🇦Samuel Proulx🇨🇦 @fastfinge@interfree.ca

1d

@esoteric_programmer There was a reason I didn't trust crowdsec, and I can't remember what it was. Half open source, maybe? Or something about how they compile the lists and sell them?

0