Results 1 to 11 of 11

Thread: What is the importance of robot.txt in Seo?

  1. #1

  2. #2

  3. #3

  4. #4
    Best Cabs in Jaipur
    Join Date
    Jan 2019
    Location
    Jaipur
    Posts
    14
    thanks for comment...

  5. #5
    Registered
    Join Date
    Jul 2019
    Posts
    162
    Robots.txt file is a very simple but a very powerful tool in our hand to protect our web pages, particular directory of our website from getting indexed in search engines.
    We can also block bots of a particular search engine through robots.txt file.

  6. #6
    Registered RH-Calvin's Avatar
    Join Date
    Mar 2017
    Location
    Forum
    Posts
    1,342
    Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.
    Cheap VPS Hosting | VPS Starting from $12 PER Year
    Cheap Dedicated Server | Unmetered Bandwidth | Free Setup and IPMI

  7. #7
    Best Cabs in Jaipur
    Join Date
    Jan 2019
    Location
    Jaipur
    Posts
    14
    Thank's Dear.

  8. #8
    Robot.txt also known as robots exclusion protocol (REP) is a text file created to guide the search engine robots (web crawler) which information on the website should or should not be processed or scanned. In other word it is file to communicate with the web crawlers.

  9. #9
    The robots.txt file, also known as the robots exclusion protocol or standard, is a text file that tells web robots (most often search engines) which pages on your site to crawl. It also tells web robots which pages not to crawl. Let's say a search engine is about to visit a site.

  10. #10
    Best Cabs in Jaipur
    Join Date
    Jan 2019
    Location
    Jaipur
    Posts
    14
    Thanks friends for your commenting.I get good knowledge. Again very very thanks.

  11. #11
    Registered
    Join Date
    Oct 2019
    Location
    India
    Posts
    20
    Have you started SEO without knowing the importance of robots.txt?

Similar Threads

  1. What is robot.txt in Seo?
    By maharanacab in forum Search Engine Optimization
    Replies: 12
    Last Post: 02-12-2019, 06:57 AM
  2. What is Custom Robot txts?
    By priyaroy in forum Search Engine Optimization
    Replies: 4
    Last Post: 01-02-2018, 11:02 PM
  3. USE of Robot.txt file in SEO
    By Bestseoservice in forum Search Engine Optimization
    Replies: 0
    Last Post: 08-19-2014, 04:00 AM
  4. Banning Alexa Robot
    By Blue Cat Buxton in forum General Management Issues
    Replies: 1
    Last Post: 08-17-2004, 10:42 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •