View Full Version : IP Blocking

Todd W
03-25-2008, 12:00 AM
I use IP blocking to deny access to a page after a certain # of views.

Works good, but it's also blocking google and only indexing a small portion of my pages.

Does anyone have a list of google IPs that the spyder uses or a way to prevent this?

03-25-2008, 12:45 AM
The easiest (and safest) way would be to create a whitelist for search engine spider user-agents.

If you can only do it by IP you can gets a list from here: http://www.iplists.com/
Vaporizer review (http://twitter.com/vaporizer)

03-25-2008, 05:11 AM
Google is constantly adding new ips, so it is something you'll need to keep up on.

Todd W
03-25-2008, 08:18 AM
I could do it by user agent I guess, and check for search engines.