Results 1 to 3 of 3

Thread: Robots.txt help please

  1. #1
    Not that blue at all Blue Cat Buxton's Avatar
    Join Date
    May 2004
    Location
    UK
    Posts
    932

    Robots.txt help please

    On my camera phones sites I use a script to redirect off site links called out.php

    The link will be coded like this

    cameraphonereview.co.uk/out.php?link=54338

    I dont want this to be spidered (and particularly dont want to leak PR so I added this in my robots.txt file:

    User-agent: *
    Disallow: /disclaimer/
    Disallow: /out.php/
    Disallow: /out.php

    Why then has google followed (and indexed) thousands of these?

    What do I need to change ?

  2. #2
    Administrator Chris's Avatar
    Join Date
    Feb 2003
    Location
    East Lansing, MI USA
    Posts
    7,055
    put out.php in a directory and then block the directory with robots.txt

    For instance here is the robots.txt file for this site:

    http://www.websitepublisher.net/robots.txt

    I put all the files I do not want Google to index in /scripts/

    Also, use rel=nofollow and if possible change link= to id= just to make Google hate it even more.
    Chris Beasley - My Guide to Building a Successful Website[size=1]
    Content Sites: ABCDFGHIJKLMNOP|Forums: ABCD EF|Ecommerce: Swords Knives

  3. #3

Similar Threads

  1. Robots.txt
    By incka in forum Search Engine Optimization
    Replies: 2
    Last Post: 04-10-2004, 06:59 PM
  2. robots.txt
    By delpino in forum Search Engine Optimization
    Replies: 2
    Last Post: 01-23-2004, 02:13 AM
  3. Is Google ignoring my robots.txt file?
    By flyingpylon in forum Search Engine Optimization
    Replies: 9
    Last Post: 01-13-2004, 09:47 AM
  4. robots.txt?
    By Mike in forum Search Engine Optimization
    Replies: 11
    Last Post: 12-24-2003, 09:27 AM
  5. robots.txt question
    By Kyle in forum Search Engine Optimization
    Replies: 6
    Last Post: 11-04-2003, 03:13 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •