Blocking Crawlers/Bots With .htaccess

  • Trying to get a site running phpBB to not get crawled as much.

    We had to blow away a huge list of IPs recently, which has upped traffic. But I think the issue is that IPs that were once "bad" are now good, and doing this by IP is a moving target at best.

    Has anyone ever done this using .htaccess, and if so, how are you doing it?

    I saw a lot of article on Google about using a list of user-agents or know IPs, but figured I'd start here with the experts first.

Log in to reply