Results 1 to 6 of 6

Thread: Googlebot crawling behavior

  1. #1

    Googlebot crawling behavior

    Google doesn’t want to crawl 2 of my sites for some reason. One of the sites
    is almost a year old and has hundreds of back links, the site is indexed in Yahoo, MSN and AskJeeves, but the Googlebot never cared to visit it even once.

    I have sites fully crawled by Google, some sites just partially crawled and some not crawled at all. The funny thing is that most of my sites have similar HTML structure and similar # backlinks.

    Do you have similar problems getting crawled by Google? Have you find any solution for this issue?


    Thanks,

    Peter

  2. #2
    Future AstonMartin driver r2d2's Avatar
    Join Date
    Dec 2003
    Location
    UK
    Posts
    1,608
    Any idea if the domain has been owned before? Could it have been banned?

    May be worth emailing Google to ask.

  3. #3
    I'll email google, but I don't think they'll reply.

    I'm more interested to find out why some sites are crawled and other not, if they have exactly the same backlinks. I'm talking about content sites, not datafeeds or ecommerce sites.

  4. #4
    Future AstonMartin driver r2d2's Avatar
    Join Date
    Dec 2003
    Location
    UK
    Posts
    1,608
    You've done a 'site:www.the_domain.com' on Google? What does it say?

  5. #5

  6. #6
    Trench Warfare
    Join Date
    May 2003
    Location
    Australia
    Posts
    813
    Quote Originally Posted by ASP-Hosting.ca
    I'm more interested to find out why some sites are crawled and other not
    Because they have been banned/black-listed.

Similar Threads

  1. Web Crawling?
    By NickC in forum Website Programming & Databases
    Replies: 2
    Last Post: 06-05-2004, 12:03 PM
  2. Crawling Other Sites
    By Mike in forum Website Programming & Databases
    Replies: 25
    Last Post: 04-15-2004, 03:28 PM
  3. external css crawling
    By nohaber in forum Search Engine Optimization
    Replies: 2
    Last Post: 03-05-2004, 07:48 PM
  4. Google is crawling
    By Chris in forum Search Engine Optimization
    Replies: 23
    Last Post: 02-08-2004, 05:37 AM
  5. Google is crawling WP
    By MattM in forum Search Engine Optimization
    Replies: 5
    Last Post: 01-27-2004, 05:21 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •