Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit.
User-agent: * Disallow:
Printable View
Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit.
User-agent: * Disallow: