PDA

View Full Version : getting deep crawled



eMEraLdwPn
08-06-2004, 01:28 AM
i just noticed only about 35,000 of my 200,000 or so subpages have been crawled / indexed by google, and i've been floating between a PR4 and 5 for several months now.. is there anything i can do to get it to crawl the rest of the pages? it would probably boost my PR back to 5 and up my traffic significantly if i got all the pages indexed

AndyH
08-06-2004, 02:04 AM
It appears to me that the higher PR you have, the more pages Google will crawl.

nohaber
08-06-2004, 03:27 AM
What site is it? How many levels deep are most of the pages? If you have 4,5 etc. level pages it should take a while for Google to index them.
If you change the linking structure to accomodate less clicks from the home page to get to all of the pages, that should help.

eMEraLdwPn
08-06-2004, 05:57 AM
it's my lyrics site. most of the links it never grabbed are like 4-5 pages in... the format goes something like: home --> browse --> artist --> album --> song

it has all the home and browse pages (all PR4), i think it has all the artist pages (like 13,000)... then it has some of the album pages and some of the song pages

i can't really think of any easier way to get to the album / song pages quicker, even if i made a site map...

nohaber
08-06-2004, 06:08 AM
i can't really think of any easier way to get to the album / song pages quicker, even if i made a site map... Yeah, I can see this.
Maybe it's a matter of waiting then. When Google indexes all of your pages, your PR should rise with such an amount of content.

eMEraLdwPn
08-06-2004, 06:13 AM
yea, i probably just have to go back to waiting then... i guess it's better that i'm getting the traffic i am now with only 1/6 the site indexed rather than the entire thing, it's just more frustrating knowing i could be getting so much more ;-)

joeychgo
10-01-2004, 04:13 AM
Ive been getting deep crawls from googl for weeks - jeteye also

about every other day

Emancipator
10-06-2004, 11:44 AM
interesting. Google this and spiders me daily for my content, i use it to get my friends sites spidered since it is daily hits. Google however doesnt do a deep crawl, it only goes a few pages deep.

I am interested in hearing approaches for deep crawls.

Blue Cat Buxton
10-06-2004, 11:48 AM
Would it help to throw some PR towards a subpage (sort of steer Google in the right direction?

paulfitz
10-17-2004, 06:57 PM
Jeteye has been annoying me recently - the hits mess up my 'unique visitor' stats as there are various Jeteye subdomains coming in on different IP addresses.

Well hopefully it will pay off with search engine results.