PDA

View Full Version : Banning Alexa Robot



Blue Cat Buxton
08-17-2004, 06:47 AM
How can I ban the Alexa robot from going through my sites? It gobbles up bandwidth for no percieveable benefit?

Can I simply use the robots.txt ?

Are there any downsides that I am missing if I do this?

Would it be better to limit it just to say the front page. If so, how can I best do this?

intelliot
08-17-2004, 10:42 AM
Yes, use robots.txt to ban ia_archiver from parts of (or all of) your site. The downside, of course, is that you won't get indexed by Alexa :)