PDA

View Full Version : Creating a crawlable URL



jonnyhilfiger
03-17-2005, 04:41 PM
Hi, I need some help.

Doing some SEO on my car classified website www.purpleads.com and have hit a hurdle.

Am trying to get the car advert pages indexed (by google) but not having any success yet.

Because the site basically only has about 19 pages (all other pages being off these from DB queries) we initially created the 'Quicklinks' links to get some more pages crawled. This resulted in 200+ pages being indexed. The next step is to get some of the car advert pages indexed.

Bear with me.....Now to stop the car advert pages being scraped we put a cipher URL for each ad (don't ask me the technicallities ;) ) but this (or so we thought) prevented the ads being crawled as the URL ended up being huge. We implemented a ModReWrite a few weeks ago to shorten the URL to something like;

purpleads.com/ad.php?modTerm=BMW-used-cars-for-sale.html&id=480690&type=1&postcode1=ws5&postcode2=3ab

....but they still aren't being indexed. (http bit missed out so the full URL showed up in post)

So, my question is, is there anything obvious in that URL that would prevent it from being crawled and that we should ommit.

Sorry for being long winded but appreciate your help.

Best regards, John

Chris
03-17-2005, 05:50 PM
Sounds like you need to modrewrite it again. Ideally you'd want to remove the ? entirely.

Plus, with the .html in the query string Google may be seeing it as malformed.

MarkB
03-17-2005, 11:43 PM
Break it down to something like:

purpleads.com/ad/BMW/used/480690/1/ws5/3ab/

or summat :)

jonnyhilfiger
03-18-2005, 02:07 AM
Ok, thanks guys. We are going to try and implement your suggestions. Will get back to you with results.

Cheers, John

jonnyhilfiger
03-24-2005, 01:22 AM
Ok, we implemented a new more readable URL and its worked....but just our luck it looks like most of the engines crawled the site midway through getting it cracked as it's indexed some 'work in progress' if you know what I mean. What it meant though is that we have also implemented a piece of code which means if a page doesn't exist it will automatically redirect to a search results page, so not all bad.

Just need for things to settle back down again now.

Cheers all, John