PDA

View Full Version : Googlebot crawling behavior



ASP-Hosting.ca
03-29-2005, 09:41 AM
Google doesn’t want to crawl 2 of my sites for some reason. One of the sites
is almost a year old and has hundreds of back links, the site is indexed in Yahoo, MSN and AskJeeves, but the Googlebot never cared to visit it even once.

I have sites fully crawled by Google, some sites just partially crawled and some not crawled at all. The funny thing is that most of my sites have similar HTML structure and similar # backlinks.

Do you have similar problems getting crawled by Google? Have you find any solution for this issue?


Thanks,

Peter

r2d2
03-29-2005, 11:07 AM
Any idea if the domain has been owned before? Could it have been banned?

May be worth emailing Google to ask.

ASP-Hosting.ca
03-29-2005, 12:15 PM
I'll email google, but I don't think they'll reply.

I'm more interested to find out why some sites are crawled and other not, if they have exactly the same backlinks. I'm talking about content sites, not datafeeds or ecommerce sites.

r2d2
03-29-2005, 12:25 PM
You've done a 'site:www.the_domain.com' on Google? What does it say?

ASP-Hosting.ca
03-29-2005, 01:23 PM
site, allinurl and link don't return anything.

ozgression
03-29-2005, 07:11 PM
I'm more interested to find out why some sites are crawled and other not

Because they have been banned/black-listed.