Aftermath: Results of a Hacker Attack

August 27th, 2009 by Chris

My literature site was one of the tens of thousands infected by a worm recently. This new type of attack works by not attacking insecurities on your site, initially I was worried it was from a backdoor or other weak script on my server, but rather it attacks webmasters at home by infecting work PCs and then sniffing for FTP passwords on that PC or on another PC on the network.

I am not sure if it was my network/PC that provided the entry, or that of someone who was doing work on the site at the time, but entry was gained via FTP. Luckily, since each user FTP account is restricted to their own directory, no system files were affected.

However, let me start at the beginning.

In late July I was told by my host that my server was dying. The hard drive was on it’s last leg and needed to be swapped out. They could give me a new hard drive, but then I’d be responsible for moving sites over, and there would likely be some downtime, I don’t like downtime, it costs me money. I argued with them asking why they could not merely mirror the drive, they said they didn’t do that sort of thing. Finally I settled on just getting a new server, but since I had paid for some serious upgrades to the current one I was worried about getting railroaded on the price on the new one.

Kudos to The Planet they didn’t railroad me at all. Instead they offered me a better server for about $100 less per month. Needless to say I took it, but that added a large deal of work for me to do in securing the new server, doing setups, moving files over, etc.

I had server hardening done by a company I had used in the past and proceeded to move most of the server files over. I got all of the static site files move, shut down the forums on the old servers, copied over the SQL databases, turned the forums on at the new servers, and tossed .htaccess files on the old server to redirect all requests to the new IP to cover the period of DNS uncertainty following a move.

Then I bought a rental property, and doing the negotiations with the realtor coupled with my wife going back to work and me becoming the primary daytime caretaker of my then 10 or 11 week old son (since I work from home), I was quite busy. After arranging our purchase agreement for the property my brother and I spent two days straight, working dawn to bedtime, gutting and renovating the upstairs unit. It was on Monday, the first day of our two day blitz, that the hackers attacked.

That weekend prior I had noticed a script not working under MySQL 5 (old server had MySQL 4, it was a left join issue). The script creator, a friend, whom I have an arrangement with (he handles the software, in exchange I provide him with content), was told and started working on it, and an hour or so after he started the hackers hit.

Hundreds of IP addressed logged into the server and started replacing index files, this was in the wee hours of Monday morning, I didn’t notice. I went to work at the rental building the next day also without noticing. I didn’t notice in fact until 3 AM on Tuesday morning when I got up to feed the baby. I was just too damn tired on Monday to notice and because there was no Apache downtime, I didn’t get any alerts as I would have in such a situation.

So I stayed up until 6 or 7 AM on Tuesday fixing the problem, that was a rough night. This hacker attack inserted iframes into index pages that would initiate a drive-by-download when users visited. Unfortunately for them, they failed. Their goal is to stealthily insert the code and then have it go unnoticed for weeks or months, in my case it was easy to see, and I would have noticed it easily had Monday been a normal day.

See, they inserted their iframe into a block of PHP code on my index page, oops. All they did was break the PHP causing the index page to throw a parsing error. No infected page was served to users, and the homepage of the site was effectively down. Now, you never like having your homepage go down, but this site gets almost all of it’s entry traffic through subdirectories, and having the homepage break for a day, thus informing me of the attack, would be better than having nothing happen and having me not notice it.

So, 3 AM on Tuesday morning I notice the site broken. I think it’s odd for me to have left a parse error like that on a life page without double checking, so I pull up the file, check it, see the malicious iframe, and immediately go into defense mode. First step, change all passwords. Second step, fix the index page. I SSH’d into the server and scanned for other changed files, using linux’s timestamps they were easy to find. They had changed about 50 or so index.php files (50 may seem like a lot, but the site has 4000+) in subdirectories. However, those files were all deprecated, the site switched to using cached plain index.html files awhile ago, so again, no infected files were served to users.

At this point I still didn’t know how access had been gotten, I was worried about script vulnerabilities most of all, and I looked, and looked, and looked, and couldn’t find anything. I was especially doubtful it was a script vulnerability because nothing was inserted into a MySQL database, and the php files that were edited could only be edited by root. I knew they didn’t have root, both because I was confident in the security of my root access, and also because only one site on the server was compromised (likewise, none of my other sites on any server were compromised).

I also suspected, though I did not yet know, that a keylogger on my own PC could have been the culprit, so I installed a few new AV programs and did a lot of scans to make sure my system was clean.

Eventually I figured out it was an FTP attack, but I felt fine about it. Passwords had been changed, logs checked and they didn’t FTP to get any important files, they didn’t touch any of my backend files where the database work is done, just those handful of index.php files with the iframes that never got served to visitors.

So, like two weeks later, I get a notice from one of my forum members. When visiting some of the pages on the site they were warned by Google it had malicious code. So I go check Google Webmaster Central where I have an account with this site verified, and yes, they report malicious code on many pages as recently as the current day. They’re also supposed to email you when that happens, I never got the email. They had apparently had parts of the site flagged for 10 days or more. Additionally, they had pages flagged that the hackers never touched. Had I not been busy with baby and business I may have noticed traffic plummeting, but I normally get traffic dips this time of year between school semesters, so I’m not sure I would have (though, this is now the lowest traffic has been in nearly 8 years probably).

So for the last few days I’ve been wracking my brain trying to figure out how Google is seeing malicious code on pages where I have gone over them, again and again, with a fine tooth comb. I even rebuilt apache entirely on the server. I could find nothing, and Google kept reporting seeing it.

So today I was going to finally cancel the old server, I had left it up because stupid search engine crawlers don’t update their DNS quickly enough. Regular ISPs will do it in hours, but for some reason search engines can go weeks without updating DNS. So I SSH’d to the server to check things out, I checked netstat and sure enough, a bunch of Yahoo Slurps and a few Googlebots were still poking around on it. I thought, this is stupid, I know I setup an .htaccess redirect, why isn’t it working? So I go and read the .htaccess file.


I had never checked the old server after the attack, I never changed the password on it. Turns out both the new and old servers had been attacked simultaneously, and while the new server had the attack stopped quickly, the old server let the attackers keep on coming for weeks. Eventually they replaced my redirecting .htaccess with one of their own which redirected visitors to their spam site, this in effect infected all URLs on the site.

So, for the handful of Googlebots that were using the old DNS (and anyone else), when they’d visit the site they’d see the infection and flag the URL, but since most Googlebots had the new DNS, they saw a clean site and that is why I was seeing inconsistencies.

I’ve lost a lot of money over this incident, probably over a thousand dollars in lost ad revenue because of the traffic loss, assuming I can get the Google warnings removed quickly now. Assuming there is no long term losses in traffic or search rankings I think I got off light. It is a lot of money, but it was an important lesson to learn. I never thought to check the old server. Money lost or not, I’m just happy to finally know the cause of the malware flags, and to know that I have corrected the issue. I’ll be able to stop stressing out over something I couldn’t figure out.

What follows is a list of IP addresses that were involved in the attack on my servers. Obviously these IPs are just infected members of the hacker’s botnet, but I thought it was worthwhile to block them in my firewall, and I include them here if any of you want to do the same. The first IP in the list has special significance, it was the IP that did the .htaccess modifications on the old server, all the other IPs just edited (over and over) index files.

3 Responses to “Aftermath: Results of a Hacker Attack”

  1. Anthony Cea  Says:

    So you are saying that your local PC was hijacked with malware and they got your server passwords from that situation ?

    I guess you need to run SpyBot Search & Destroy more, I was forced to go back to it recently since my AV software was not stopping all the threats that you pick up simply from surfing the web now days.

  2. Roland  Says:

    Hello Chris,

    Several of my sites got infected by this attack, too. We were puzzled at first because sites on two different hosting accounts were hacked in exactly the same way. I didn’t immediately think that one of my office PCs would be the problem.

    In the end we did isolate the cause of the infection – a laptop used to administer all the sites involved – and rebuilt it.

    Interesting to read someone else’s experience on this – having two servers running concurrently at the time of the attack was an unfortunate coincidence for you!



  3. Geoff  Says:

    I found your article whilst searching for the IP that made a similar attack on one of my sites in early August. My attacks started after installing FileZilla. (a games server in Romania) appears in my FTP logs 5 times. I’ve seen the same IP in the log of another site that has been destroyed by a similar attack. Complaints to abuse at the service provider have not produced a response.

Leave a Response

(Email field must be filled in)

Top of page...