Results 1 to 4 of 4

Thread: Rewrite conditions being ignored

  1. #1
    Roll Tide! mobilebadboy's Avatar
    Join Date
    Apr 2004
    Location
    Mobile, AL
    Posts
    428

    Rewrite conditions being ignored

    I'm trying to block various spider/spam/harvester bots, and although pretty much every site gives the same code, it doesn't seem to work for me. After adding the rules I still see them coming back. And I've even spoofed my UA to do some testing, and I'm allowed to visit the site every time. Here's what I have:



    RewriteCond %{HTTP_USER_AGENT} ^almaden [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^ASPSeek [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^DTS\ Agent [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^EmailCollector [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^iaea.org [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^InternetSeer [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^Microsoft\ URL\ Control [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^NPbot [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^QuepasaCreep [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^TurnitinBot [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^Wget [NC]
    RewriteRule ^/.* - [F]
    Shawn Kerr .com

  2. #2
    Senior Member chromate's Avatar
    Join Date
    Aug 2003
    Location
    UK
    Posts
    2,348
    I don't know anything about Rewrite but you could easily use PHP to test for user agents like that.

  3. #3
    Roll Tide! mobilebadboy's Avatar
    Join Date
    Apr 2004
    Location
    Mobile, AL
    Posts
    428
    Yeah. If it was just one site and only a couple of bots I'd normally just do it that way. But since I plan to use this on all of my sites, and since all of my sites aren't set up the same (some sites use 1 common header, some don't) it'd be a whole lot easier to just edit 9 or 10 htaccess files. Plus I plan to expand the lists over time.

    And some of my sites (like visual intensity) already have a couple of hundred lines of detect/control code in the header. I'm trying to limit that more and more.
    Shawn Kerr .com

  4. #4
    Roll Tide! mobilebadboy's Avatar
    Join Date
    Apr 2004
    Location
    Mobile, AL
    Posts
    428
    Well, I ran across diveintomark.org and got it working. I noticed in his examples he was putting things into quotes. Every other site gave examples as posted above. I put everything into quotes and they all worked. Thanks Mark!
    Shawn Kerr .com

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •