PDA

View Full Version : Sudden tumble is Google for key phrase I've been in the tops for since 90s



KLB
07-27-2006, 09:19 AM
Okay this is really scarry. All of a sudden this morning I discovered I have tumbled in Google's SERPs for the search phrse "periodic table of elements" I have been on page one for this search phrase most of the time since the 1990's normally hovering around #6 - 8. All of a sudden this morning I discover I have tumbled to page five or #51 overall.

I have similarly fallen in the serps for all kinds of search phrases I am normally in the top ten.

The only change I think I made that should have had any impact on SERPs was I added the robots NOOPD instruction to my header's when it was announced.

Whatever happened with the my site in Google's index, happened this morning because my stats were just fine for yesterday, but today I'm taking a blood bath.

The only other major changes I have made to my site in the past month has been to disable my scripts that detect and block users who block my ads, to convernt over from validating to HTML 4.01 Transitional to validating to HTML 4.01 Strict, I started gzipping my files for those browsers that support this option.

Google is indexing my site (http://www.google.com/search?hl=en&q=site%3Aenvironmentalchemistry.com&btnG=Google+Search). The only catch is that they are showing 155,000 results when I only have 20,000 pages; however, Google has been doing this for about two years now. So I wouldn't think that this would be the issue.

Whatever happened, I need to figure out why and fast. Because if it isn't fixed quickly, I will be really hurting revenue wise.

Chris
07-27-2006, 10:07 AM
I'm in the same boat in regards to the number of results:

http://www.google.com/search?num=50&hl=en&lr=&safe=off&q=site%3Aonline-literature.com

No way do I have that many pages, a quarter of a million forum posts and maybe 100,000 content pages and they got me pegged at 1.1 million.

But I have not seen any drops.... except....

On monday when using DP's keyword ranking checker my survival site dropped from #1 on "wilderness survival" to #9. This shocked me because it has been #1 since the summer of 2001, shortly after it launched. It was also still #1 for "survival" which is the much harder term. I checked Google though and it was #1 still for both, so only the datacenter the tracker uses had it marked.

A few hours later I did the DP checker again and my position was back to normal.

So I don't know what happened or if it is related but that is the only similar change I've noticed.

There are reports of an update happening though:

http://feeds.seroundtable.com/~r/SearchEngineRoundtable1/~3/6082332/004227.html

KLB
07-27-2006, 10:27 AM
Traffic to some sections of my site is off by 95%. This is very scarry. I've never seen such a fall in the SERPs for my site and I noticed a drop in some Yahoo searches as well.

It isn't like I try any agressive SEO tactics and it isn't like my site is a junk MFA site. This is a very serious content site with original content. I invest thousands of dollars this year alone having highly qualified professionals write new articles for me.

Tiny temporary hichup or not, I've got to make sure I recover from this quickly and find ways to prevent it from happening again. I've been trying to diversify my traffic such that I don't depend on Google so heavily for traffic, but with a site as diverse as mine and each individual page or topic having only a narrow scope of interest makes it very hard to promote the site without search engines.

As far as indexing goes there is not a single bad internal link on the site (I tested recently).

Kyle
07-27-2006, 04:09 PM
KLB - Survive Outdoors is back in the SERPs on Google today after being basically off the SERPs for over a year. We have 14 DMOZ listings, Yahoo listings, and many 1 way links and 2 way link exchanges.

I have freaked out many times in the past with regards to Google's randomness when it came to my site. Some of it I'm sure was my mistake, when I put an affiliate shopping section on Survive (which was recently taken down - months ago).

After dealing with severe drops in rankings for 2 years now, I decided to work on important topics we cover, make the articles more complete and organized, and get more links. Keep in mind.. this work I was doing was NOT FOR GOOGLE. I was doing this to improve my rankings on MSN and Yahoo. This month, with 95% of my traffic coming from Yahoo and MSN, my unique visitors are usually between 6000 and 8000 visitors per day. My adsense earnings are wonderful.

I never thought our articles on Bee Stings and Snake Bites (both listed in DMOZ with "Bee Stings" and "Snake Bites") would fall out of the top 100 on Google. But thats what I have been dealing with for 2 years now on/off.

What have I been doing instead? Diversifying...

Two kinds of diversifying, developing garbage sites purely for money (hard to do, boring), then expanding your main focus (being env chemistry) on sub sites.

I have some sites that only profit off MSN now. They are for the most part, lower quality sites which make great profits. I can then take that money and invest in long term more important projects. Its annoying work. As we all know, working on websites where you have 0 interest in the subject matter and you know its purely for money are hard to stay motivated on.

When it comes to working on your niche, Enivornmental Chemistry.. again I can share some ideas that I have been doing with Survive Outdoors lately.

We are in the process of taking certain major subject matters and developing sub sites on them. These sub sites are going to cover the topic in depth, and be part of the "Survive Outdoors Network" or whatever you want to call it. This involves acquiring a great domain name for the topic at hand, then writing numerous articles on that topic. This is also a great way to diversify your traffic and rankings. Its also a great way to grow your domination on various topics.

Example of what we're doing.. www.ticks.net. This domain cost over $1000 from BuyDomains, but is a great long term investment. Ticks are a huge, growing subject matter with respectable CPC bids. I say respectable because we're talking about ticks here.. you'd think there would be little to no ads shown.

I hope these specific thoughts help keep your hopes up, and motivate you to grow your environmental chemistry site(s).

(haven't proof read this yet, I will after i submit it.. so look out for edits! :))

Kyle
07-27-2006, 04:46 PM
After looking more at your site, it's obvious you are very established with your numerous inbound links. The only thing I could say is maybe Google doesn't like the text links you have on terms like "debt consolidation", "loans", etc. I'm assuming this is from the digitalpoint co-op network.

I'm not sure why you are a member of this network?

KLB
07-27-2006, 04:47 PM
I finally got around to creating a site map. I don't think my site really needs it as I have been very careful with my site's link structure from the beginning and Google has done a very good job at indexing my site. The site map was around two megs (URLs only). I also confirmed via Google site maps that I had no serious indexing problems.

@Icebane

As far as having lots of good content, one thing my site has never been accused of is being short of content. Even back in 1996 one of my friends commented that my site was so large back then that he wasn't sure where my site stopped and the Internet began. Currently my site has around 20,000 pages (no query strings) including scores of pages of pure articles. Right now I'm in the process of updating my periodic table new information My table is one of the oldest periodic tables on the Internet and one of the most expansive available. I also try to add new articles every month, which are written by highly qualified professionals and when necessary, I send articles through a peer review.

My site isn't a MFA site and it has been around since long before AdSense existed. I take creating unique content and a high quality site very seriously. This isn't a get rich quick scam site, this is a very serious content site and always has been.

KLB
07-27-2006, 04:57 PM
After looking more at your site, it's obvious you are very established with your numerous inbound links. The only thing I could say is maybe Google doesn't like the text links you have on terms like "debt consolidation", "loans", etc. I'm assuming this is from the digitalpoint co-op network.
I've been hosting those text links for around a year now and they have never caused a problem.


I'm not sure why you are a member of this network?
Because I'm paid a very handsome sum to host those links and this allows me to pay writers to write new content. Would I prefer more relevant ads on my site? Yes I would, but I don't want to be totally dependant on AdSense for revenues and affiliate type ads have never done well on my site so I must sell links directly.

In the end unless we are doing this for a hobby we must sell advertising space one way or another and I much prefer to host text link ads than obnoxious flash ads or bandwidth hogging banner ads. I'm fairly confident that those links are not hurting my site's SEO and they aren't obnoxious to users so I don't see them causing any harm.

Kyle
07-27-2006, 05:07 PM
Oh i know you have lots of good content.. I was just trying to give some thoughts/advice based on my experiences. I am fully aware your site was not created for adsense.

Again, was just giving advice on diversifying across domain names.

With regards to the co-op not hurting your rankings, who knows... maybe someone you trust should reply to that one and give their opinions on putting the co-op on VERY legit, high quality sites (like yours).

Chris
07-27-2006, 05:34 PM
Yes, I can't seem to find the links Kyle is talking about, but consider them suspect #1 for causing this change. Links to unrelated sites are a big no-no.

I had the coop on this site, when it was only webmaster forums and there were only 10 people in it. As soon as Shawn opened it up I canned it, unrelated links like that look like spam.

One external link I had on my literature site for years was to my coupon site. I rel=nofollow'd it last year though because I did not want to take the risk, thats just one link, to my own site, and I even tried to make it on topic by saying "To save money on books."

Do paid links even make that much money. I was offered like $65 a month for a link on my literature site, for that little it is not even worth it.

KLB
07-27-2006, 05:55 PM
Do paid links even make that much money. I was offered like $65 a month for a link on my literature site, for that little it is not even worth it.

I won't make the amount public, but $65/month is chicken feed. Simply looking at my published advertising rates will give you an idea of how much I'm making on those links.

If I don't see a recovery in the next week I will ditch those links when this month's term is up.

Cutter
07-27-2006, 06:29 PM
I have noticed an across the board shakeup on Google for many terms. There are a few sites I type keyphrases in to the engine instead of bookmarking (don't ask me why), and some of the sites have dissappeared.

Shoemoney made a good point on his show tuesday. At one of the SES conferences he was able to actually talk with a google engineer who pointed out specific things as to why his site was ranking below the competition.

I suspect due to the quality of your site you could manage to get some leverage in understanding or getting your rankings back, especially if it is that dramatic. These are real people that work there and they want authoritative high quality sites ranking; sometimes the algorithm chages meant to get rid of spam have unintended consequences.

KLB
07-27-2006, 06:45 PM
Cutter,

I'd love to use the leverage you talk about but for starters I've got to get someone to listen.

Like you say there are issues all over the net with people reporting good sites getting dumped. We all like to turn a deaf ear to these complaints because 99.99% of the time the people whining really have junk sites or are trying to trick the serps using really sketchy SEO tactics. In my case I don't use any of these tactics. I don't have a big network containing scores of sites, I don't cloak, I don't keyword stuff, I try not to spam the indexes, I don't use doorway pages, I don't do any of that stuff.

I simply follow the motto that original compelling content is king. I try to add new content on a regular basis (actually today's events have distracted me from my effort to completely update my periodic table data with new information). I try to make sure my site validates to W3C specifications. I try to follow best practices for accessiblity.

I do everything everyone agrees are the best above board practices for website development and promotion. Yet today I'm finding my self buried five pages deep on search terms that I have been on page one since before Google was out of beta. It just doesn't make sense.

Kyle
07-27-2006, 10:49 PM
KLB - You really do not have to defend your site's level of quality... this is obvious. I really feel for you because of how much I used to ***** about being in the same situation.

I look at your site, look at your DMOZ listings, look at all your quality incoming links from .edu and .gov sources... I don't think you should worry. Things will pop back.

The co-op is extremely bad. It's the only thing I see wrong with your site.. and it's a big thing.

I thought my site was the worst example I had seen.. but yours takes the cake. I'm used to webmasterworld conversations on these Google problems where no one can list a specific example (per webmasterworld's rules). Thanks for sharing, I really appreciate it.

Cutter
07-27-2006, 11:49 PM
Hey KLB, have you sent them any e-mail messages explaining what happened to your site?

I would say:

I have X# of .gov incoming links, the site has been around for X# of years, I pay academic people to write original articles, but today I just lost X% of my Google traffic. I don't understand if there is something wrong I have done, or if possibly a mistake has been made somewhere.

There are cases where sites were specifically spamming google and have gotten back in on request after fixing the problem. I would suspect that they would be understandable with what has happened.

This is where I would go, as Matt Cutts has referenced on his blog:
http://www.google.com/support/bin/request.py

KLB
07-28-2006, 08:31 AM
Thanks Cutter, I followed your advice and sent the following email to them with a reference showing the bad search results:



Dear Google,

Following the advice on Matt Cutts' blog, I am contacting you via this form.

Yesterday after years of sitting at around #5 - #10 for the search phrase "periodic table of elements" our webpage http://EnvironmentalChemistry.com/yogi/periodic/ plunged from page 1 to page 5. My periodic table is one of the oldest periodic tables on the Internet and is one of the primary reference sources for large numbers of the periodic tables that are on the Internet.

This is not a new or spammy "MFA" site with little to no original content. This is an old very well established and respect website has been on line since 1995 (switched domains to http://EnvironmentalChemistry.com in 1999) with mountains of original content and original articles written by highly qualified professionals (site development timeline at: http://EnvironmentalChemistry.com/yogi/about.html#History).

Now I realize that you say this happens due to the automatic nature of your algos, however, I saw the same plummet across wide swaths of our site and yesterday our traffic fell by 2/3 from the day before. This isn't just another natural fluctuation; there was a major upheaval in the SERPs and it would appear that our site is being severely punished by the last major data push and/or back links to our site are not being recognized.

Although Google searches are showing almost no back links via "allinural" or "links" to our site I know for fact that there are thousands to tens of thousands of links to our site (excluding forum sigs and our own sites) including high quality links from .GOV, .EDU and .MIL websites (e.g. http://www.cdc.gov/nceh/publications/books/housing/cha02.htm, http://ss-cbiac.apgea.army.mil/resources/directory/chemical.html, http://ic.arc.nasa.gov/people/neveu/ChernobylArticles.html, http://www.bib.ub.es/www3/3dic_quim.htm, etc). In fact the Google search http://www.google.com/search?as_qdr=all&q=+environmentalchemistry.com+-site%3Aenvironmentalchemistry.com+-site%3Aklbproductions.com is showing me 114,000 results that reference my website (this excludes our own sites).

We do not buy or trade links via reciprocal link trading schemes. We do directly sell advertising on our site but the emphasis on this advertising is always about the ads targeting real users and selling this advertising is necessary for us to fulfill our primary mission, which is to produce high quality content that is educational and informative.

We work very hard to provide very high quality and original content. Even with our data sources like our periodic table of elements and chemical database, this information is based on offline research and data compilation (in fact we are currently working on our latest update for our periodic table). There are precious few website in our niche that have been around as long as our website or spend as much effort producing and making freely available the kind of content we have produced.

If there is something wrong on our site that is causing a penalty, I would appreciate knowing what is wrong. It just seems weird that a site as vast and well linked to as ours would suddenly fall so far in the serps in a 24 hr period.

Sincerely,
Kenneth Barbalace
http://EnvironmentalChemistry.com

Phone: 207-797-8202

Chris
07-28-2006, 10:34 AM
Just a thought Ken, have you checked to see if your site is in the supplemental index now? That happened with my art site, lots of nice links, long time online, then in May, boom, supplemental.

KLB
07-28-2006, 11:26 AM
That was the very first thing I thought of. I have found none of my pages in the supplemental index.

I did find that Google was having problems passing the correct chemical name strings to my server, which may have been causing empty pages to appear to them. Basically it was an issue with the hex character 'A0' not being replaced with the hex character '20' properly. I thought I had resolved this problem years ago with a series of search and replace statements, but when I started tracking bad chemical requests last night I found Googlebot was having problems. I was able to replicate the problem in Internet Explorer (but not Firefox) by copying and pasting the URLs Googlebot had requested in IE. After hours of trying I finally hit upon a series of instrutions that replaced the character A0 with the character 20 before sending the request on to the database. At least it now works in IE I'll have to wait for some results to see if it solves the problem for Googlebot. A doorway page penalty seems to be the most likely culprit. I'm making sure to send Googlebot 404 errors for requests to chemicals that don't exist in my database and I'll watch this to mannually add 301 redirects to get requests to the proper place when I can.

KLB
08-01-2006, 06:34 AM
Well I think I have done just about everything possible to make sure my site doesn't upset Google's spam filters and have submitted a reinclusion request. There isn't much more I can do but sit and wait to see if things start to improve.

The amount of money I'm losing because of this is devistating and I it is going to put a real crimp on my ability to pay for the articles I have commissioned for the next couple of months. It seems Google's little update is going to cost everyone but Google. My readers will be denied the benefit of new articles, my writers may not get paid for articles I commissioned and I will suffer from a serious drop in income. I blogged about this whole issue at: http://environmentalchemistry.blogspot.com/2006/07/googles-june-27thjuly-27th-update.html

Kyle
08-01-2006, 02:33 PM
Well I think I have done just about everything possible to make sure my site doesn't upset Google's spam filters and have submitted a reinclusion request. There isn't much more I can do but sit and wait to see if things start to improve.

When will you take the co-op links down?

KLB
08-01-2006, 03:07 PM
When will you take the co-op links down?
I'm contractually obligated to keep them up until August 18th. Remember I'm not trading links, I'm being paid to post advertisements and I do not advertise my PR when selling links. These ads will probably be cut at the end of the advertising term simply because of my loss of traffic reduces the value of the ad slot to the advertiser.

In honesty this should make no impact on my SERPs as my inbound links are not built on link exchanges or "linking schemes". If Google doesn't like the selling of links then they should ignore outbound links from sites that sell them and they should ignore recipical links. To punish a site for selling links/ads is a monopolistic behavior as they control 80% of the search market and sell ads/links on their own site and via their AdSense program. To punish sites for selling their ad space directly to the advertiser or telling people how the must manage those ads/links is an anti-competitive behavior and should line them up for anti-trust actions.

From a practical standpoint, beyond the web bug that tracks human visitors I don't see how Google could detect these links. Based on all of the discussions I have had with people in spite of the hype around the coop links, I do not believe that they have a positive or negative impact on people's search results.

Kyle
08-01-2006, 03:45 PM
I'm contractually obligated to keep them up until August 18th. Remember I'm not trading links, I'm being paid to post advertisements and I do not advertise my PR when selling links. These ads will probably be cut at the end of the advertising term simply because of my loss of traffic reduces the value of the ad slot to the advertiser.

Understood with regards to your contract.

You don't advertise your PR when selling links? So? You are still passing it.


To punish a site for selling links/ads is a monopolistic behavior as they control 80% of the search market and sell ads/links on their own site and via their AdSense program. To punish sites for selling their ad space directly to the advertiser or telling people how the must manage those ads/links is an anti-competitive behavior and should line them up for anti-trust actions.

Google is a search engine where determing quality is based on analyzing content, and weighing inbound links to that content. When a given page is linking to unrelated garbage (yes garbage, many of the co-op members use their weight and point it to spam garbage), Google has every right to penalize you. This is most likely an automated process. I highly doubt Google has manually penalized your site.

Regarding selling text links. Take a look at www.statcounter.com. PR10. Notice all the topic-related advertising links they sell in their left column. If you do back-link checks on these websites, statcounter always shows up. And most of them rank very VERY well for the competitive web development topics they are targetting.

In response to your flame on Google's monopolistic behavior, they have a solution you now. You have every right to sell text links to garbage sites. Just remember to put rel=nofollow.

I'm not generalizing about your text advertising options available on envchem, I'm specifically targetting your decision to defend the co-op network as an appropriate form of advertising revenue.


From a practical standpoint, beyond the web bug that tracks human visitors I don't see how Google could detect these links.

Very easily. If one of the many sites already flagged by Google as spam is in the co-op network, and Google sees you linking to this garbage when they crawl your site, the result is obvious.

Because it is often impossible to figure out exactly what is causing a drop in rankings, the best thing we webmasters can do is eliminate ALL negative factors. Even being paranoid at times just to make sure we maintain control of our Google rankings as best as we can (see Chris's thread regarding changing his Amazon coupons link at the botton of his literature pages to rel=nofollow)

Even when we remove all these negative factors, we still may get screwed by Google. Your problem could be temporary...it may resolve regardless of your co-op links. However, you could even remove your co-op links right now, and in 4 weeks your rankings pop back, but the improvement had nothing to do with the co-op removal. We will never know.

KLB
08-01-2006, 04:44 PM
Understood with regards to your contract.
Google is a search engine where determing quality is based on analyzing content, and weighing inbound links to that content. When a given page is linking to unrelated garbage (yes garbage, many of the co-op members use their weight and point it to spam garbage), Google has every right to penalize you. This is most likely an automated process. I highly doubt Google has manually penalized your site.

Again they should just ignore these links. In regards to the coop program itself, I didn't even know I was hosting coop links let alone know what coop was until a week ago. When the deal to sell links came to me almost a year ago I was under the impression the the buyer was selling links himself.

An automated process to penalize sites for specific outbound links would be really unfair to open participation sites like forums as they have little control of what sites they link to especially since not all forum/blog software provide the option to add the rel=nofollow flag. Besides it is forcing those sites to do Google's job and essentially do SEO, which according to Google is not what they want us to do.


In response to your flame on Google's monopolistic behavior, they have a solution you now. You have every right to sell text links to garbage sites. Just remember to put rel=nofollow.
It was not a flame and it was not an unfair comment. Google forcing us to put rel=nofollow in our sites to satisfy its editorial demands Is the hallmark of anti-competitive behavior. They are telling us how to sell our links. Their the ones who created this stupid mess in regards to links and skewed the selling of links from traffic to PR. I have avoided selling based on PR as much as I can and I place links in places where users are likely to see them.


I'm not generalizing about your text advertising options available on envchem, I'm specifically targetting your decision to defend the co-op network as an appropriate form of advertising revenue.
I am not defending coop. Had I known what the coop was before I took on those ads I don't know whether or not I would have accepted them. I have never really like the links but the money was very good and it provided important revenues that allowed me to buy more articles for my users to read.



Because it is often impossible to figure out exactly what is causing a drop in rankings, the best thing we webmasters can do is eliminate ALL negative factors. Even being paranoid at times just to make sure we maintain control of our Google rankings as best as we can (see Chris's thread regarding changing his Amazon coupons link at the botton of his literature pages to rel=nofollow).

This is exactly what Google wants, they want us to be paranoid and act out of panic. This is exactly why people are constantly chasing Google dances. If coop ads really were the problem, then why would it have taken so long for them to have an impact and why are there so many coop sites that are not being impacted by this issue? I agree that we should use best practices, but I do not believe in knee jerk reactions.


Even when we remove all these negative factors, we still may get screwed by Google. Your problem could be temporary...it may resolve regardless of your co-op links. However, you could even remove your co-op links right now, and in 4 weeks your rankings pop back, but the improvement had nothing to do with the co-op removal. We will never know.
In the end this all may be a mute argument as the buyer of those links wants human page views and without those human page view, advertising on my site is not as valuable to him thus buying links from me is not of value to him.

IF there was a problem on my site, I personally suspect it was with programming errors with my chemical database, which have been resolved. I tend to think, however, that what we are seeing is the affects of a really goofy update that was not counting backlinks correctly due to the fact that the "link" command has been mostly broken for the past month and only started working correctly today.

There have been days recently that Google was showing no links to my my site even when I could find over 100,000 webpages referencing my site by simply searching for my domain as a string in quotes. The other things I have seen too many examples of sites that are not using DPs Coop having similar problems. I have also seen a comment by Matt Cutts that there was a "bad data push" last month.

Look we all like Google because it seems to do a better job of indexing the Internet than its competitors; however, we must still look at it with a critical eye. If we do not hold Google accountable and we do not draw scrutiny to how they do their updates, even squeeky clean and saintly websites run the risk of getting caught up by a "bad data push" and Google runs the risk of losing its focus on resolving the problems with spammers registering domains and deploying billions of sub-domain spam pages designed to flood the serps.

I don't know about you, but on a daily basis I find countless examples of scraper sites scraping text from my site and deploying cloaking tricks to get page full of PPC ads in front of users. If Google really cares about spamming, DP's Coop is a non-issue in comparison to the sub-domain scraper pages I find on a daily basis.

Chris
08-01-2006, 05:15 PM
Kyle makes a good point about being paranoid, you have to be paranoid.

I'm not going to defend Google, I think they've lost their focus and instead of focusing on making the best possible serps, they're focusing on policing webmasters. These two things are related, but they are not the same thing.

Due to Google's erratic behavior you need to be paranoid, if a site's rankings are so important that you cannot afford to lose them, you need to be squeeky clean.

I got a PR 7 site that is so close to an 8 the little diddly subpages like my privacy policy is a 7 too, as is the forum. I've seen forums that put dozens of sold links in their footer. Judging by from what I hear through the grapevine I could do that, and maybe make 50k more a year or whatever. But I'd rather make 50k less per year, and make what I make for 10 or 20 years, than to make 50k more per year but only have it last for 2 more years.

Google's overzealous anti-seo crackdown isn't right, but you still have to deal with it and so be paranoid.

Another thing I don't do is link out. Almost all outgoing links on my sitesare redirects or nofollows or there simply are no links. I do not have any "links" page on any of my sites. Of course I like to conserve PR, but I also don't like the bank on the scruples of those I link to. In my years of doing this I can probably count the number of link exchanges I've done on one hand.

KLB
08-01-2006, 09:36 PM
Kyle makes a good point about being paranoid, you have to be paranoid.
I'm pretty paranoid about things, but I also realize that I have to maximize revenues. It isn't like I'm earning $100,000 a year. I'm in the $30,000 - $40,000 a year range and an extra $500 a month does things like pay for medical insurance and another article for my website. I've tried affiliate ads and banner advertising, but they don't work very well on my site. I really don't want to sell out to flash ads or intrusive ad formats. I really like text based ads and I don't want to be 100% dependant on Google. Being able to directly sell text ads and bring in an extra $10,000 a year from those ads is really important.


I'm not going to defend Google, I think they've lost their focus and instead of focusing on making the best possible serps, they're focusing on policing webmasters. These two things are related, but they are not the same thing.
I agree completely, but it is even worse. They have become so obsessed with the whole selling of PR issue that they are missing way bigger spamming issues. On any given day I will find new index spam where the domain had been registered within the previous two weeks and hundreds of thousands to millions of sub-domains generated from scraped content have been indexed by Google.

You can try it yourself. Use a service like GoogleAlert and have it track the top 100 results for key phrases off of your pages (uncommon names work great). Pretty quickly you will start finding results with a goofy six to eight digit .ORG domain name and a key word saturated sub-domain name. Often times theys pages are set up to look like a personal blog, but the content text is complete gibberish of smashed together scrapped phrases. When the links are inspected they are either CPC links or links to other sub-domains under the same domain. If you do an "allinurl" for the main domain you will normally find between 100,00 and several million results and a whois of the domain normally reveals that it is less than two weeks old. Our friend Nintendo discovered the worst offender of this sub-domain spamming scheme when he found a 14 day old domain that had around one billion pages indexed by google with references to it mostly via sub-domains of the domain.

Yahoo and MSN seem to be doing a good job weeding these spammers out, but Google seems unable to exclude them. Like I said, I'm finding new spamming domains using this trick almost every day via a search tracking service that uses Google's API.

Here is an example of this from yesterday:

domain: dunalus.be
sample sub-domain: blue-tooth-ac-power-supply-model-5012.dunalus.be
results: 48,000
domain registered: Jan 10 2006

Here is another example:

domain: tvntv.kom.pl
sample sub-domain: oxygen.tvntv.kom.pl
results: 28,900
domain registered: unknown

These were pretty modest examples as I have found some with as many as 1,000,000 results returned for "link:" searches. This is what makes me so mad about Google's obsession with how we sell links, while they are chasing us around they are allowing hundreds of millions of spam pages flood their search results with pure MFA pages (with AdSense ads none the less).


Due to Google's erratic behavior you need to be paranoid, if a site's rankings are so important that you cannot afford to lose them, you need to be squeeky clean. So punish the honest and keep them scared, while the dishonest just keep churning out millions of MFA pages based on scrapped content. Ya that makes sense... NOT!


Google's overzealous anti-seo crackdown isn't right, but you still have to deal with it and so be paranoid.At the same time we should be drawing attention to this issue, Nintendo may have driven us all nuts over at SitePoint, but he is right about this issue. They are so busy chasing middle of the road SEO stuff that they have lost focus of the really bad spammers.


In my years of doing this I can probably count the number of link exchanges I've done on one hand.
Boy your generous. One of the first things I figured out was not to give out links for free and don't do link exchanges when others are willing to link to my site without asking for anything in return.

What I really want to find are some corporate sponsors who are interested in targeting my user base.

Chris
08-02-2006, 05:46 AM
I would do popunders before untargetted link sales. You get what? 5k uniques a day? One popunder paying at $4 CPM would net you $20 a day, $600 a month, which would make up for the link sales loss.

I've always had them and users don't seem to mind that much, I get a complaint maybe once every 3 months, and I've got thousands of backlinks despite them.

KLB
08-02-2006, 08:11 AM
I so hate pop over/unders, that I'd feel I would have sold my soul to the devil if I used them. I think it is hypocritical to use advertising methods I'm not willing to accept in my own surfing habits.

Oh my advertiser did give me clearance to add rel=nofollow to his links. Maybe this will help.

Shawn
08-02-2006, 03:42 PM
Ken, please call me immediately. I called you but received no answer.

I know why you're not ranking in Google (not speculation, 100% guaranteed). It's not your text links, it's not the advertising on the page. Way off.

Call me tonight, anytime.

Shawn
08-02-2006, 04:10 PM
Check your PMs.

KLB
08-02-2006, 04:30 PM
Sorry, we went out to supper as we had a power failure here. Just love this heatwave

Chris
08-03-2006, 09:09 AM
You know Ken nocache can often make you look like a cloaker, it still might have mattered, maybe the coop links were one trigger and the nocache a second trigger. Either by itself is nothing, but together they trigger some sort of spam filter.

KLB
08-04-2006, 10:38 AM
Okay, I have extracted my self from the coop links by refunding the advertiser for the balance of days left on their advertising. There shouldn't be a single questionable thing left on my site.

In regards to the no cache issue, Google has no business penalizing sites for not wanting other sites to display cached versions of their pages. If they are concerned about cloaking, they can always download copies of pages via their Googlebot UA and via a "real" browser UA and see if there is a significant difference. Heck any Google employee could spoof Googlebot's UA in their browser and see that Googlebot and real users get the exact same thing.

Chris
08-04-2006, 10:40 AM
Nevertheless, I understand its a recent change and its worth taking off to see if it could be the cause.

KLB
08-04-2006, 11:10 AM
Here's my reason for keeping the no cache instructions on my site. Without that in place, faux web directories (e.g. spammers) can use this as an excuse to serve up cached versions of my pages with their ads in their search results and then use the no cache flag as a shield to hide behind when it comes to copyright infringement complaints.

I have enough troubles with my pages getting copied, scrapped and and usurped by every get rich quick artist trying to build MFA sites. I don't need to give them a pass to cache my pages.

Until my "little" tumble in Google's SERPs, I had been issuing on average one DMCA take down notice and cease and desist letter per week. Sometimes for articles that were only days old. The last thing I want to do is compound this problem.

Besides I seriously doubt my not letting search engines display cached versions of my pages has any bearing on my seach results (especially when Google publicly states it doesn't have an affect).

Chris
08-04-2006, 11:38 AM
Well, like I said, its worth a shot. Its a recent change you made, just to rule it out.

I've never heard of copyright infringers using that as an excuse, and even if they did they still would have to take it down once told.

KLB
08-04-2006, 12:07 PM
Well, like I said, its worth a shot. Its a recent change you made, just to rule it out.
It isn't a recent change, I've been using the no cache thingy since Google first announced they supported it (many years now).


I've never heard of copyright infringers using that as an excuse, and even if they did they still would have to take it down once told.
They say an ounce of prevention is worth a pound of cure. I've seen several MFA "directories" with "cache" links in the past year and had to issue a C&D at least once. By using the no cache flag I remove an excuse.

It would be morally and ethically wrong (e.g. doing evil) for Google to punish or flag sites for simply trying to protect their copyrights by adding ROBOTS NOARCHIVE to their pages especially since Google has stated that it has no impact on search.

Shawn
08-04-2006, 12:18 PM
Let me play the devil and ask:

What if things stay this way? What's in Ken's future?

KLB
08-04-2006, 12:48 PM
Let me play the devil and ask:

What if things stay this way? What's in Ken's future?

I'm not sure what you are asking, but there were two primary canidates as to why I got thumped by Google: 1) goofed up URLs and a lack of 404's for non-existant chemicals in my chemical database caused Google to index many more pages than actually existed potentially creating massive duplicate content penalties; 2) DigitalPoint Coop ads which I was being paid to host.

Before running off half cocked and making changes to things that should have no impact on how I do in the SERPs, I think it is prudent to let things work themselves out now that the DP coop ads are gone and I have fixed the problems with my chemical database.

The more I think about it, the more I believe that I got hit hard for linking to a bad neighborhood site via the DP coop ads and these links are now gone.

All remaining links/ads are to clean sites and there are very few sites that I actually link to from my pages.

I have a reinclusion request pending and I'm continuing to monitor my 404 errors for any more goofy stuff in my chemical database.

Really when one looks at my site as a whole, I'm pretty conservative as to how I run it. I don't buy or trade links, I don't have any "links" pages, I don't cloak, I don't do any browser sniffing, SEs and real browsers get the exact same files and my code is exceptionally clean.

Heck my site even works well in text only browsers like Lynx. If anyone wants to see how my site looks to SEs or text only browsers, simply turn off JavaScript and CSS and then surf my site (heck, you could even turn off images). I don't think you will find many sites that work as good as my site in text only mode. I even think someone using a dynamic Braille display would be quite satisfied with my site's accessiblity. Honestly, visit my site with all your browser bells and whistles turned off. I think you will be amazed -- especially when you see the site with all the bells and whistles turned on.

Shawn
08-04-2006, 01:04 PM
I'm asking what happens when Google doesn't come back and rank you. Do you have more websites that you make an income off of? Will you keep working on EnvironmentalChemistry.com with no Google rankings?

That sorta stuff. I know how it's ideal to think your site is good and has been ranked well forever thus should be highly ranked, but be realistic. Google de-listed your site. Not dropped in rankings, but de-listed. I can't find you when I search for "EnvironmentalChemistry."

What's the gameplan if and when you're not back in Google in 1, 6, 12 months?

KLB
08-04-2006, 01:52 PM
I'll cross that bridge when I come to it. Google is indexing my pages on a daily basis and I have a reinclusion request in. I try not to deal with what ifs.

Kyle
08-04-2006, 03:26 PM
If you rely on Google, you're not just "dealing in what ifs", you are relying on what ifs...

KLB
08-04-2006, 03:44 PM
I've been working hard, especially this summer, to reduce my reliance on Google. This is why I started providing an RSS feed for my site, started a related blog and have always had a comment on my site asking people to link to my site and to tell their friends about my site. It is also why I've shifted from being a pure reference site to also publishing original news articles. I'm working very hard at building up my source of non-Google traffic.

I'm not quiting and I'm not waiting for Google I am trying to move forward with my plans. Just last night I published a new article and I'm sending out a copyright release for an article I plan on publishing in two weeks. I just finished an major update of my periodic table of elements and will probably move forward on another database project I have been putting off for a couple of years.

One bright spot is that Yahoo has really liked my changes and has started indexing more of my pages than ever before. I am hoping that MSN follows suit.

My niche has always been a challenge because I am not a big company, government agency or major university and in may cases I compete head to head with major university projects.

Chris
08-04-2006, 04:16 PM
Why don't you have a forum?

KLB
08-04-2006, 05:18 PM
Why don't you have a forum?
I have debated this issue for years now and I'm not sure its the wisest idea. Here's why, every school year I start getting loads of emails from high school kids who basically give me their chemistry homework assignment and expect me to do their homework.:brickwall

I actually ended up burying my contact us link at the bottom of the page for this exact reason. It is also why my contact us page has a lecture about me not doing their homework for them. At one point in time during the school year we would get at least six emails a day from students wanting answers to question that were on the page they had visited immediatly before clicking on my contact link. As my mom says too many students, can't read and won't think.

I'm certain that a forum would basically turn into a whole bunch of kids posting their homework hoping someone else will do it for them and then start *****ing (uh griping) when someone didn't do their work for them. This is the last thing I want and I don't want to have to police a teenage trolls who come along just to cause trouble -- the email I get from those teenage trolls is bad enough.

Now I am interested in adding some kind of commenting system for my articles (kind of like blog comments). I think this could work well as I would issolate it off to just the articles where kids tend not to go. The thing is, I don't know what commenting system to use that would actually tie into my my existing system seemlessly. I might have to code something of my own (yuck!).

Chris
08-04-2006, 06:03 PM
The thing with a forum though is that it insulated you from traffic downturns in SEs because your forum has an existing core of stable visitors.

I think you're really handicapping yourself by not having one.

You'd also be suprised, my literature site I'm sure has just as much homework people on it, and it really isn't that big of a problem in the forums, and finding volunteer moderators is pretty easy as well.

If you wanted you could do like what I do here with the marketplace forum. Rather than trying to moderate self promotion (or in your case homework help solicitations) I just allow anything to be posted in the marketplace forum. Its like a pressure release valve. If you made a subforum specifically for homework help it'd hopefully keep such posts out of the rest of the forum.

KLB
08-04-2006, 07:08 PM
Well I've discused this with my mother who is the real chemist type in the family and she is game for trying the forum so I think we will be moving forward with it.

Shawn
08-04-2006, 08:21 PM
www.physicsforums.com pulls it off well.

Owned by Greg B, they have a section for homework requests -- which isn't bad to have a place for (so they don't spam other places in your forum). But, the rest of the forum is composed by graduates/doctors discussing related issues in physics.

You can definitely have a quality forum.

paul
08-04-2006, 09:16 PM
I would think your site would be a natural for a science fair connection. At some times of the year there are thousands of searches per day from people looking for science fair projects.

KLB
08-11-2006, 09:47 PM
I thought I'd give everyone an update on how things are going.

While I am contemplating the forum thingy, I'm putting it on the back burner and focusing on what will make the biggest impact on my site immediately and am doing what I'm best at. I've been working on my site pretty much 12-15 hours a day (lately I'm down to 6-7 hours of sleep a night). My first focus was fixing some 301/302 issues, making sure certain database driven pages returned 404 errors properly when there were no records (e.g. page) found. At the same time I worked on dealing with goofy URL issues that could cause duplicate content. Once I had that pretty much under control (it took several days) I shifted my focus towards updating my periodic table (which I had started before the July 27th disaster).

I'm now working on updating my chemical database and adding in new data sources. I had planned this update for this winter, but decided it was prudent to do it now. Merging chemical data sources requires some serious data mining and manipulation, which can make reading a phone directory seem exciting. While a fair amount of the process can be automated, it still requires a great deal of manual massaging of the data, which is probably about as hard as massaging an elephant.

While I'm doing this update much earlier than originally planned and much more intensely than I would normally do it, this update does complete a vision I've had for my site for about seven years.

When I started my site, one of the first things I put on my site was a periodic table of elements, which quickly grew to be one of the more comprehensive periodic tables on the web. Shortly there after I put together a section on DOT transportation placards which are those colored diamond signs on the sides of trucks and rail cars carrying hazardous materials. I built this section as a public education project for one of my hazardous materials/firefighting classes I was taking at the time. Shortly there after, while on a hazardous materials team and working for a company that transported hazardous materials, I started building a database of chemicals. Eventually people started suggesting the database would be a good addition to my site.

From their suggestions I got the idea that eventually I could collect enough data about the chemicals that I could build drillable paths through my website that would allow a person to start at say my periodic table and travel from an element, to chemicals that contain the element to emergency response guidance for the chemical and to shipping regulations and placarding requirements for that chemical. Likewise I wanted a user to be able to start at a placard and follow the path from the placard to chemicals that require the placard in question all the way back to the periodic table of elements. The whole idea was that people would not only be able to drill into a specific subject as far as they needed to, but that kids would start to build a connection between their having to study the periodic table to how those elements relate to the real world.

This update will complete the path from my periodic table to the placarding pages. For instance one can now go to my periodic table, select nitrogen (N), and go to page three, which contains some chemical compounds made with nitrogen. From there they could select "hydrogen cyanide" (HCN), which leads to a page containing all the data I have collected for this compound including shipping regulations and emergency response guides. On the page for "hydrogen cyanide" the reader could then select one of the links on the shipping regulations table for hazard class, which would lead to an explanation of the hazard class in my placarding section. Or if they wanted the reader could review the emergency response information and select the proper guide page for the type of hydrogen cyanide the reader is interested in and see all of the initial response guidelines for this chemical.

What I learned when working on hazardous materials teams is that sometimes one needs is the ability to drill down and find identify information that allows one to cross reference data in various technical manual. Sometimes, simply being able to relate the name of a chemical to a CAS number or other identifier is critical. This is why I started my data collection project so many years ago; I needed a way to cross reference all of the different HazMat resources.

What I'm afraid of is that Google is throwing penalties on my site because of what it sees are doorway pages, when in fact, I'm simply providing the users of my site with all the data I have collected to help them in their research efforts.

If Google doesn't like some of my pages because they "lack content" then Google should ignore those pages like the other SEs do, rather than hitting an entire site with penalties. I shouldn't need to turn off and deny access to data simply to make Google happy.

While I don't know for sure that this is why I got dropped from the serps it is my operating theory. As such, I have been working myself to death trying to do a few weeks what I had planned to do over several months.

Chris
08-18-2006, 07:39 AM
Ken apparently some people have seen movement in Google results, how bout you?

KLB
08-18-2006, 08:12 AM
I'm back for two phrases that I am aware of. They are "Environmental Chemistry" (#1) and "Placards" (#1). I'm still being killed for "periodic table of elements" and my traffic is still way down.

I'm still working like a dog updating my chemical database, although I was out of town most of this week and not able to do much more than simply check my email.

The phrase that placement recovery will have the single biggest impact on my traffic will be "periodic table of elements"/"periodic table" (without quotes).

FPU
08-19-2006, 06:14 AM
What you have to watch are your total of indexed pages, since they might drop 90% this week and come back next week you never know if you will be in the results, if the page is not indexed you will not come up for those terms in the SERP's!

Google has been accused of selective indexing, but I see that MSN is also having problems holding indexed pages recently too!

I guess there is too much WWW for any of the SE's and none of them can index everything so they purge the indexes and start over and this is the cycle they seem to be following!

organ
08-21-2006, 09:36 PM
Recently, google tumble my anime site and index large numbers of trash pages about my site.
http://www.animegirlmovie.com/forums/calendar.php?c=1&week=1276387200
http://www.animegirlmovie.com/forums//showthread.php?t=6
I hates google
:flare:

KLB
08-22-2006, 05:20 AM
Why would Google rank your forum very well? It has only one thread, one post and one member.

organ
08-22-2006, 06:15 AM
the new forum just launch several days ago, but the old forum exist for half year.
Google index the old forum's DB.

KelliShaver
08-22-2006, 02:43 PM
And this is Google's fault now?

KLB
09-12-2006, 01:28 PM
I thought I'd give people an update. I'm slowly recovering for more search phrases. As of the last couple of hours I have started to see a recovery on some datacenters for one of my most important terms ("periodic table of elements"). This isn't a big money term, but it is an important traffic term for me that tends to create the most word of mouth publicity for my site as students share my site with classmates and teachers.

Hopefully this means a recovery of my other search terms is around the corner.

KLB
09-18-2006, 06:12 AM
On Saturday Sept. 16th, I expierenced what appears to have been a complete recovery in Google's SERPs. Sunday was also the highest traffic day I have seen since late last winter and today is starting off very strong.

I don't know if this is a temporary thing or if this is a sign of things to come.

I do appreciate all of the advice people gave me at the height of my crisis. I am still working on some of the updates a started two months ago and hope to complete them within the next few weeks.

This past two months has been a very scary and stressful time for me.

Chris
09-18-2006, 10:02 AM
congratulations.

Kyle
09-18-2006, 08:11 PM
Ya I'm happy for you too Ken.

KLB
09-18-2006, 08:37 PM
People will start seeing me in the forums more again now that these problems have been resolved.

I still have lots of work ahead of me, and I'm trying to complete a massive project I started shortly after this mess got started, but I don't feel like I have to try and focus all waking hours on disaster recovery. Now I can focus on disaster prevention.

One goal is to improve my non-Google traffic if at all possible.

Chris
02-09-2007, 09:53 AM
Something similar may have just happened to me a few days ago.

My survival site has been #1 on Google for "survival" for 3 years, and poof, is gone now. But it still is #1 on wilderness survival and a bunch of other terms. Just "survival" it is gone from. Strange.

KLB
02-09-2007, 10:28 AM
Interesting. You'll have to keep us posted as to how this evolves.

FPU
02-24-2007, 02:38 PM
Google is still having problems indexing pages properly, I notice this all the time, unless they are doing selective indexing hand jobs excluding sites for some political reasons.

I have pages that show up in all the other engines that used to rank high in Google and all of a sudden they are no where to be found any more in Google but ranking fine for the same terms on all the other engines.

So my above theory is correct, either Google is still not indexing the web fully or the way they move pages to supplemental based on age is a factor, or they are excluding certain sites for political reasons.

Chris
03-04-2007, 09:09 AM
And we're back. #3, not #1, but better than not being listed at all.

I did send in a reinclusion request, don't know if it helped or not, I didn't get a response.

KLB
03-04-2007, 11:01 AM
No wonder we are so paranoid; Google keeps throwing us bones without us knowing if they actually do anything or not.:brickwall

FPU
03-04-2007, 01:31 PM
I know one thing, don't use sub-domains on an ASP hosting plan, they have destroyed most blog networks that use sub domains.

One reason is, affiliate marketers buy these programs, use sub domains for drug sites, gambling and other types of affiliate landing pages, so if you put up your content site on one of these networks and use a sub domain, you may have zero hope if Google is penalizing the entire domain for having these sorts of pages.

Chris
03-12-2007, 09:32 AM
And down again.... Google is being quite erratic.

cameron
03-12-2007, 12:56 PM
I had been steadily climbing for a particular keyword and was dropped completely a couple of days ago too. It really hurts at my income level. I'm not losing the icing on the cake, I'm losing the flour and the eggs.

FPU
03-16-2007, 10:06 AM
And down again.... Google is being quite erratic.


One thing I have noticed is how Google is still having capacity problems, the SERP's you have are determined by how many pages you have that are not supplemental pages.

When your indexed pages change so do your SERP's, if they wipe out your indexed pages or 90% are supplemental you are not going to show up in the search results at all and are lucky if your pages show in results as supplemental once in a while.

All this is related to Google's capacity problem, it has been since the indexed page bug which the CEO admitted is truly a capacity problem. Google has lowered the amount of indexed pages for most sites versus what they had in the past, I know of forums that had millions of indexed pages, but today they have 140,000.

This is why the index is deleted and rebuilt all the time on new parameters.

Chris
03-16-2007, 10:13 AM
That isn't the problem though, the page is still indexed, and still ranking at the top for other terms, it is this one specific term that it vanished from.

jonnyhilfiger
03-16-2007, 02:46 PM
You're not alone :bawling:

One of my sites since the new year (site is 3 years old) has been going from normal traffic levels to being cut in half, to then come back to normal for a week to 10 days then back down again, then up again, then down again etc. Currently traffic is halved.

I'm not even going to try and wonder why, I'm sure it will sort itself out in the end.

Google, can't live with 'em can't live without 'em

John

ZigE
03-28-2007, 03:52 PM
Hmm, one of my sites just tumbled to the bottom of page 5 aswell. After making steady gains to almost top10. No effect whatsoever on my regional google results, which is where my traffic comes from, thankfully (touch wood).

Strange.

MaxS
04-10-2007, 10:43 AM
On a somewhat related note, my website which has been #1 for about 6 weeks, has fallen to #16.

I'm hoping, and am pretty confident, that it's only a matter of time until I'm back at #1.

Chris
04-10-2007, 12:32 PM
I'm still going up and down, 2 days up, 2 days down. Its annoying.

MaxS
04-11-2007, 04:21 AM
I went from #1 (for about 6 weeks straight) to #16 for a day, and now I'm back at #1.

Phew...