PDA

View Full Version : IP Blocking



Todd W
03-25-2008, 01:00 AM
I use IP blocking to deny access to a page after a certain # of views.

Works good, but it's also blocking google and only indexing a small portion of my pages.

Does anyone have a list of google IPs that the spyder uses or a way to prevent this?

rpanella
03-25-2008, 01:45 AM
The easiest (and safest) way would be to create a whitelist for search engine spider user-agents.

If you can only do it by IP you can gets a list from here: http://www.iplists.com/
________
Vaporizer review (http://twitter.com/vaporizer)

Chris
03-25-2008, 06:11 AM
Google is constantly adding new ips, so it is something you'll need to keep up on.

Todd W
03-25-2008, 09:18 AM
I could do it by user agent I guess, and check for search engines.