Time for some investigative work. This post may get unapologetically technical.
So I began by looking at my access logs from the day that the compromise occurred, I started with just a full browse of the log to look for anything anomalous. Discovered a lot of hotlinking to images on my site! Squashed that problem (go ahead and try it! You might have to clear your cache first to see the effects….)
I suspected that looking for POST requests would be a good place to start since there were likely to be fewer of them.
grep POST access.log.0
Which revealed, among other things:
18.104.22.168 – - [29/Feb/2012:06:27:48 -0800] “POST /wp-content/themes/minicard/auroreashley.php HTTP/1.1″ 200 143 “-” “-”
In the previous post, I mentioned the backdoors — well here it is! So it looks like the attack vector was via a backdoor uploaded to the current theme. I went and scanned that directory:
-rwxrwxrwx 2 USERNAME pg7788 11005 2012-02-29 06:27 archive.php
-rw-r–r– 1 USERNAME pg7788 28278 2012-02-27 16:37 auroreashley.php
-rwxrwxrwx 2 USERNAME pg7788 10606 2012-02-29 06:27 comments.php
So the backdoor was installed 2 days prior to the attack. The attack itself occurred yesterday, so it’s good to know that it was a quick turnaround from detection to removal, and also good to know they only had 2 days to do damage. At this point, I now knew which date to look for, so I looked in the access logs from the 27th.
Tried the following:
- [delphinus]$ cat access.log.2012-02-27 | grep ashley (nothing)
- [delphinus]$ cat access.log.2012-02-27 | grep POST (nothing out of the ordinary)
Then I remembered that since WordPress requests are done using pretty URLs, the extension “php” shouldn’t be mentioned anywhere in the requests, so I tried: cat access.log.2012-02-27 | grep php
Scanning through the results, I see these three lines:
22.214.171.124 – - [27/Feb/2012:16:37:09 -0800] “GET //wp-content/themes/default/installer12.php HTTP/1.1″ 200 217 “-” “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; Maxthon; .NET CLR 1.1.4322; .NET CLR 2.0.50727)”
126.96.36.199 – - [27/Feb/2012:16:37:09 -0800] “GET //wp-content/themes/default/images/installer12.php HTTP/1.1″ 200 222 “-” “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; Maxthon; .NET CLR 1.1.4322; .NET CLR 2.0.50727)”
188.8.131.52 – - [27/Feb/2012:16:37:09 -0800] “GET //wp-content/themes/minicard/installer12.php HTTP/1.1″ 200 213 “-” “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; Maxthon; .NET CLR 1.1.4322; .NET CLR 2.0.50727)”
The interesting thing here is that the attacker tried three different places, though it appears it was all in one second of one another, so it’s also possible there were three separate files. The “installer12.php” file was removed by the time I looked for it. Just to be I searched: find ~/ -name installer*.php but found nothing.
Immediately after this line, I saw two requests to wp-cron.php, which I had been previously dismissing as maintenance, since the IP address was dreamhost. Up until now, I had been assuming this was regular maintenance. But the fact that it specifically pointed to the theme that was the point of entry for this hack, I decided it was worth investigating.
184.108.40.206 – - [27/Feb/2012:17:47:58 -0800] “POST /wp-cron.php?doing_wp_cron=1330393678 HTTP/1.0″ 200 353 “-” “WordPress/3.3; http://mjhill.net”
220.127.116.11 – - [27/Feb/2012:20:07:32 -0800] “POST /wp-cron.php?doing_wp_cron=1330402052 HTTP/1.0″ 200 353 “-” “WordPress/3.3; http://mjhill.net”
Quick search on google for “installer12.php” — shows up in some random foreign websites in very weird places, like “/images/”. Something is fishy here. This means two things:
- The attacker was able to spoof my host or had shell access
- The attacker was using wp-cron.php to do the dirty work
I also found an article on wordpress.org about others who had similar problems, and came to the same conclusion.
At this point, I need to look at wp-cron.php and find out what is referencing cron jobs: where doing_wp_cron equals 1330393678 and 1330402052.
Browsing into wp-cron.php source, after verifying that mine matched the canonical source, it appears that the numbers are merely timestamps and may not be uniquely useful in themselves; HOWEVER, it does mean that I need to look at the cron table for WP, which is loaded by _get_cron_array(). After some digging, this appears to be the query that is ultimately pulling data from the DB:
So, time to investigate the options table for this blog. There were a few things I expected to see (“scheduled”, “scheduled_delete”, etc.) and then two rogue entries:
Now I don’t know for a fact that these entries are junk, but they seem odd. The rest of the options table looked legit. Not enough to go by here.
Back to the log.
I thought this time I would filter based on finding “minicard”, the theme that was first compromised. (Backdoor was created at 4:37 on 2/27, so anything nefarious before that should be useful).
After the installer was run, this shows up in the logs:
18.104.22.168 – - [27/Feb/2012:22:03:51 -0800] “GET /wp-content/themes/minicard/js/jquery.fancybox/$%28this%29.attr%28 HTTP/1.0″ 200 13199 “http://mjhill.net/wp-content/themes/minicard/js/jquery.fancybox/$%28this%29.attr%28″ “HuaweiSymantecSpider/1.0+DSEemail@example.com+(compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR ; http://www.huaweisymantec.com/en/IRL/spider)”
The string: …jquery.fancybox/$%28this%29.attr%28 evaluates to: jquery.fancybox/$(this).attr( — which is jQuery. This looks syntactically similar to an open-ended SQL injection attack.
Under the assumption that the attacker might have uploaded the payload before injecting my host to launch it (and so before the exploit popped the IP address would be different), I searched instead for: cat access.log.2012-02-27 | grep 2012:16 – the year and hour (date is implied, since this is the log for one day only).
I’m seeing some curious traffic sourcing from what I was passing off as a robot before:
[delphinus]$ cat access.log.2012-02-27 | grep 2012:14
22.214.171.124 – - [27/Feb/2012:14:38:12 -0800] “GET /robots.txt HTTP/1.0″ 200 260 “-” “Mozilla/5.0 (compatible; MJ12bot/v1.4.2; http://www.majestic12.co.uk/bot.php?+)”
126.96.36.199 – - [27/Feb/2012:14:38:14 -0800] “GET /about/img_5243-1 HTTP/1.1″ 200 473 “-” “Mozilla/5.0 (compatible; MJ12bot/v1.4.2; http://www.majestic12.co.uk/bot.php?+)”
188.8.131.52 – - [27/Feb/2012:14:38:18 -0800] “GET /about/img_5243-1/ HTTP/1.1″ 200 13219 “-” “Mozilla/5.0 (compatible; MJ12bot/v1.4.2; http://www.majestic12.co.uk/bot.php?+)”
184.108.40.206 – - [27/Feb/2012:14:54:06 -0800] “GET / HTTP/1.1″ 200 4102 “-” “Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:220.127.116.11; aggregator:Spinn3r (Spinn3r 3.1); http://spinn3r.com/robot) Gecko/2010040121 Firefox/3.0.19″
Now what’s interesting here is that the last entry’s IP matches with who it claims to be (spinn3r, a blagosphere indexing robot with a tragically meaning-sounding name):
18.104.22.168.in-addr.arpa name = 174-36-228-156.robot.spinn3r.com.
…but the first three don’t match.
22.214.171.124.in-addr.arpa name = 126.96.36.199.triolan.net.
Thinking that perhaps Majestic12 was some hacking group, a google search reveals a blog post from a couple years ago where many people were being crawled and attacked by someone pretending to be the M12 bot. Ok, now we’re onto something. I’ve at least been able to identify the attacker — all incidences of that M12 bot showing up in this time period should be considered hostile. That article does say that “1.4.x” should be a legitimate version of the bot, it was also from a couple years ago. Inconclusive.
UPDATE: I’ve determined that the attacks came from kasserver.com, though I don’t know if kasserver.com was complicit in the attacks.
Though the day prior to this one had additional hits come in from the M12 bot from a different IP, this one resolving to: 188.8.131.52.in-addr.arpa name = evo-hl21-1.gameservers.net.
More to come later. I need to research some more, and I’ve contacted Dreamhost support to see if they can pull anything from their logs.