🇨🇦Samuel Proulx🇨🇦
@fastfinge@interfree.ca
Tip: you can use fail2ban to ban crawlers. They often visit random likely urls, generating either 404 or 500 errors. A human will not generate 5-10 404 errors on your website per hour. Once it gets several an hour, ban the IP for 24 hours. Nukes stupid crawlers and scanners quickly, while requiring zero extra work from humans, and not screwing up accessibility in any way. Make sure, of course, all the links on your website don't generate 404 page not found or 500 server errors. But if you have so many broken links that real humans are getting banned by this, you need to fix your broken website anyway.
