PDA

View Full Version : Robots.txt help please



Blue Cat Buxton
07-20-2005, 05:57 AM
On my camera phones sites I use a script to redirect off site links called out.php

The link will be coded like this

cameraphonereview.co.uk/out.php?link=54338

I dont want this to be spidered (and particularly dont want to leak PR so I added this in my robots.txt file:

User-agent: *
Disallow: /disclaimer/
Disallow: /out.php/
Disallow: /out.php

Why then has google followed (and indexed) thousands of these? :confused:

What do I need to change ?

Chris
07-20-2005, 06:05 AM
put out.php in a directory and then block the directory with robots.txt

For instance here is the robots.txt file for this site:

http://www.websitepublisher.net/robots.txt

I put all the files I do not want Google to index in /scripts/

Also, use rel=nofollow and if possible change link= to id= just to make Google hate it even more.

Blue Cat Buxton
07-20-2005, 06:57 AM
Thanks Chris