PDA

View Full Version : Sudden bandwidth useage increase



paul
01-06-2006, 11:37 AM
I woke up to a bandwidth overrun message from my hosting provider this morning. One my old sites (1999) suddenly sucked up almost three normal months of bandwidth useage in one evening. Looking at the stats I see that IP 66.249.65.44 was apparently the issue and as near as I can tell that IP belongs to Google.

This is a static html site with about 125 pages that has been fully indexed for years.

Can anyone tell me what was going on?

Todd W
01-06-2006, 12:21 PM
Google spydered your site.

James
01-06-2006, 12:38 PM
That's not Googlebot's IP address. Looks like someone was just being an arse.

Todd W
01-06-2006, 12:49 PM
If 125 static html pages with no downloads (no movies, big flash) etc, made you go over your BW limit.. maybe your BW limit is to low ??

I didn't look up the IP just assumed you were correct.

Westech
01-06-2006, 12:56 PM
That IP does belong to Google (according to http://www.arin.net/whois/ ), so it's most likely one of their various bots. Do you have any logs of the page requests that would show the User Agent for those requests? That could help you figure out exactly what's going on.

James
01-06-2006, 01:06 PM
Ah, apologies, I just looked up a list, which I guess was outdated.

paul
01-06-2006, 01:24 PM
Here is what the Webalizer stats for user agent show:

1 6756 27.23% Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET
2 2947 11.88% Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
3 1016 4.09% Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

My site normally uses less than 1 gig of bandwidth per month. They used almost 5 in one night.

r2d2
01-06-2006, 01:42 PM
Maybe your stats havent updated yet, because that doesn't look like googlebot - I have this in webalizer:

6 2787 3.74% Googlebot/2.1 (+http://www.google.com/bot.html)

also that doesnt look like anywhere near enough traffic to use 5Gb with plain html pages.

The 'Top 10 of X Total Sites By KBytes' stats near the middle could reveal more.

paul
01-06-2006, 01:53 PM
Maybe your stats havent updated yet, because that doesn't look like googlebot - I have this in webalizer:

6 2787 3.74% Googlebot/2.1 (+http://www.google.com/bot.html)

also that doesnt look like anywhere near enough traffic to use 5Gb with plain html pages.

The 'Top 10 of X Total Sites By KBytes' stats near the middle could reveal more.

1 504 2.03% 404 2.00% 4836614 95.94% 61 2.00% 66.249.65.44

Thats where I got the Google IP.

Masetek
01-06-2006, 04:03 PM
Man, something is really wrong there! Google raped your site bad. Maybe you should email google, give them that data and ask them whats up.

Westech
01-06-2006, 11:47 PM
Here's some info you might find useful: http://www.google.com/webmasters/bot.html

paul
01-08-2006, 08:14 AM
Thanks for the responses. Things seem to have settled down without me doing anything. Below are the stats for the last few days. The starting and ending bandwidth numbers are what I consider "normal" for my site. As much as I can tell, all that activity has not changed anything. I will await the next update with more than the usual anxiety :)

3 5338 15.97% 4310 15.74% 1854 15.15% 667 16.05% 658 21.37% 50182 0.81%
4 5150 15.40% 4297 15.70% 1917 15.67% 612 14.73% 603 19.58% 1797426 28.93%
5 4111 12.30% 3441 12.57% 1618 13.22% 569 13.69% 547 17.77% 2879848 46.35%
6 4297 12.85% 3417 12.48% 1685 13.77% 552 13.29% 511 16.60% 1349960 21.73%
7 4365 13.06% 3773 13.78% 1612 13.17% 561 13.50% 537 17.44% 54485 0.88%

James
01-08-2006, 06:16 PM
I'd suggest that you still ask Google what's up, to see if you can prevent this from happening again; you don't want them doing this every few weeks.

*PS* \/\/007!!!11!1! 1337'th p057!11!!!*asterisk!1!