Tales from a base64 wordpress hack, part 1: detection

So as I mentioned in my previous post, this blog was compromised earlier today, with the nefarious base64() hack. I’m still picking up the pieces, and viewing this as being pushed into finally switching to a new theme (I’ve been thinking about it for a while now).

The good news is that it’s not the end of the world, and if you are feeling adventurous, you can recover your blog from this and probably have most of your data intact still.

At this point, I can post about detection and stopping the bleeding. Hoping to have a followup about better prevention, if I can figure out how they compromised a fully-updated WordPress installation in the first place.

Detection

First off, are you in the clear? If the hack was done cleanly and not by some amateur like the idiot that hacked mine, your blog will not show obvious signs of infection. I have read reports that this particular hack only manifests for users using Internet Explorer and on Windows, since those users are most susceptible to whatever malware it’s pointing you to.

This is going to require some scary terminal stuff. Bear with me. I’ll try to make it as painless as possible. If you don’t know what “SSH” means, or have never logged into your server via the Terminal, then find someone who has and ask them to teach you. It’s good to know this stuff.

  1. Log on to your server via SSH (FTP will not work for this)
  2. Navigate to your blog directory
  3. Type this command:
    grep -R “base64″ *.php | awk ‘{ print $1 }’

What this does is search for instances of the string “base64″ in all your PHP files of your blog, then shrink it down so that it doesn’t fill up your whole screen. If you see nothing come up, you’re probably ok. If you see anything, it might mean you’re infected (if you see a bunch of things show up, you’re most certainly infected).

  Let’s assume you found a couple results. We need to see if they’re infected, so pick a couple at random and inspect them with the cat command.

  1. cat thefilename.php

You’re looking for “eval(base64_decode(….” in the beginning:

If you see that. You’re most certainly infected. This means several things:

  1. Every PHP file in your blog is infected. EVERY ONE. Trust me. I’ve looked. Just so we’re clear (this applies only to .php files, as far as I can tell):
    1. Every core WordPress file is infected
    2. Every plugin you have installed is infected
    3. Every theme file you have is infected
    4. Some of your uploads directory (and pretty much anything under wp-content/) is likely infected as well.
  2. There is more than likely a “backdoor” on your server right now so that the attacker can sneak back in later.
  3. The attacker may have access to your database and/or web server via credentials it found while crawling your WordPress files. (there are ways to check this)

The last one is particularly scary, but from what I’ve seen so far, they tend to just do URL hacking (hacking around your site by using the backdoor) rather than full on SSH shell logins.

Just to be safe, let’s test. In the shell again (you still have that open, right?), type this:

  1. last -i | grep $(whoami)

Let it run for a couple minutes, or until the dates showing up are a week or so old.

If you don’t know your IP address, go find it first, then compare that number to the numbers showing up. They should be similar but may not be identical. If your current IP looks like 111.222.333.444 and the IP in the output looks like 111.222.333.434, it’s probably fine. If it looks like 111.233.333.444, that’s potentially bad. You will want to submit a support ticket to your hosting provider immediately and either give them a screenshot or just paste the text into the email.

Beginning to Recover

The process of recovery will take a while. But it’s doable. And you can do it!

For starters, we need to start fresh. This means re-installing WordPress clean. Don’t worry, your data is safe in the database, and we’ll have you back up and running in minutes. I promise.

  1. Navigate to the directory just above where your blog is located (eg. if your blog is in ~/blog/, then navigate to ~/)
  2. Move the directory containing the blog to a backup location that is NOT web accessible. (eg. move ~/blog/ to ~/blog.hacked/)
  3. Create a new directory with whatever your blog was originally called. (eg. create ~/blog/)
  4. You should now have the old installation in a safe place (~/blog.hacked/) and your blog URL should now point to an empty folder. (You can test this right now)

Now we need to download the most current version of WordPress, unzip it, and copy over our configuration, and then it should be ready to go!

  1. Navigate to the directory where your blog should be contained (not the “hacked” old blog data, the so-fresh-and-so-clean-clean location)
  2. Download the latest WP version:
    wget  http://wordpress.org/latest.zip
  3. Unzip the archive (it will automatically unzip into a “wordpress” subdirectory, this is ok)
    unzip latest.zip 
  4. Move the files in that new folder into the correct place
    mv wordpress/* .
  5. Delete that empty directory now
    rmdir wordpress 

Do not access your blog via the web just yet. It’s still not ready. We need to tell it where your database is first, which is held in the wp-config.php file — but before we can copy that over, we need to sanitize it and update the security settings.

  1. Use your favorite text editor (you can use pico if you’re new, or vi or emacs if you’re strong like bull) to edit wp-config.php in the old (hacked) installation (I’ll use pico here, though I am normally a burly vi user):
    pico ~/blog.amhill.net.hacked/wp-config.php
  2. The window will probably fill up with a bunch of familiar-looking garbage. This is all on one line, so all we need to do is delete the first line (press ctrl+K in pico, or dd in vi. I don’t know what to use in emacs). The whole line should disappear, and you should see “WordPress’s config file” as the next line. Be sure that the very first line of the file is this (and add it, on its own line, if it’s not there)
    <?php  

Browse around this file, while you’re in here. We’re going to do a couple things to batten down the hatches a little bit. For starters, we need to make sure your config file is using some of WordPress’s newer security salts.

  1. Scroll down through the file and look for a line that begins with:
    define(‘AUTH_KEY’, …
    define(‘SECURE_AUTH_KEY’, …
    define(‘LOGGED_IN_KEY’, …
    define(‘NONCE_KEY’, …
    If you don’t have it, we’re going to add it. If you DO have it, we’re going to replace it, to ensure that any logged-in sessions your attacker may have are immediately ended.
  2. Go here and copy the text into an empty line somewhere in your wp-config.php file — somewhere in the middle is fine. Pick a spot that looks comfy.
    https://api.wordpress.org/secret-key/1.1/salt/
  3. Look for a line that starts with:
    define(‘DB_NAME’ …
    define(‘DB_USER’, …
    define(‘DB_PASSWORD’, …
    define(‘DB_HOST’, …
    If you don’t see it anywhere, great! You can skip this part.
    If you do see it, then continue.
  4. Copy those 4 lines from your terminal into a scratch pad somewhere (Notepad, gEdit, Textmate, whatever)
  5. Delete those 4 lines from wp-config.php, save, then exit the editor.
  6. From the terminal, we’re going to create a new file using your favorite editor from earlier (again, using pico in this example). We’re just going to use the full URL for your blog with a “.php” at the end.
    pico ~/blog.amhill.net.php
  7. In this file, enter
    <?php
    then, on a new line, enter the four lines you put into the scratch pad (define…  x 4), then, on a new line
    ?>
    Save and exit.
  8. We need the absolute location of this file, so at the terminal, type:
    echo ${HOME}/blog.amhill.net.php
    (except use whatever file you just created a moment ago instead of “blog.amhill.net.php”)
    The output should be something like:
    /home/elecmahm/blog.amhill.net.php
    Copy this path over to your scratch pad.
  9. Open up your wp-config.php file again (remember, it’s still over in the old-and-busted location)
    pico ~/blog.amhill.net.hacked/wp-config.php 
  10. Somewhere near the beginning (line 2 or 3 — around there), add in this line:
    require_once(‘/home/elecmahm/blog.amhill.net.php’);
    except use whatever you jotted down from earlier instead of “/home/elecmahm…” -(be sure that you wrap that string in quotes, as above!)
    Then save and exit.

One last thing we need to do to get you up and running, and this is up to you if you want to do it or not: Your uploaded data is contained in wp-content — this has all of your plugins, themes, uploaded images / documents, etc. It also has a crapton of infected scripts and is probably where the backdoor is located.

There’s a great article worth reading over on ottopress.com called How to find a backdoor in a hacked wordpress. I recommend reading it as it will help prime you with what to look for. I was able to find my (hopefully only?) backdoor file within a few minutes; they aren’t hard to spot if you know what you’re looking for.

Jump back into your terminal and navigate into the murky depths of your hacked wordpress installation (the one that isn’t live). You want to get into the wp-content directory.

  1. cd ~/blog.amhill.net.hacked/wp-content/
  2. ls -la  <– that’s a lowercase “l”, not a numeral one, both times

First of all, remember that every single gorrammed PHP file is infected here. The file “index.php”, for example, should only contain the following:

<?php
// Silence is golden.
?>

But it probably contains a bunch of other crap. You can clean it out like earlier, if you like, but it will be really time consuming. Really, all we want to do here is salvage your uploads. You’re just going to have to bite the bullet and re-install all of your themes (hope they weren’t heavily modified, or you kept backups!) and all of your plugins. So based on what we’ve seen so far, we know that the attacker corrupted all the PHP files. Chances are, you’ve never uploaded a PHP file in your uploads directory (if you have, that’s risky business!). So do this:

  1. Navigate into the /wp-content/uploads/ directory
  2. Locate all PHP files within this directory by typing:
    find . -name *.php
  3. You will probably see one or more files show up; take note of these filenames.

See that file that showed up on mine? anettaauroora.php? It will probably have a different name on yours, but that file is bad news. Take a peek inside of it with your favorite editor to see what I mean. This file is different than the other ones we saw — it’s obfuscated in a similar way, but it actually contains a backdoor that the attacker can use to further manipulate files on your server; re-infecting you at a later date, if s/he wants to.

Either delete it, or move it to a safe place (and rename it if you do!) — just get it out of there. Once it (and any of its friends) is out, we can copy the uploads back over. Again, substitute the directories you use on your server here for the ones I’m using.

  1. First, make sure the uploads directory exists in your squeaky-clean wordpress:
    mkdir ~/blog/wp-content/uploads 
  2. Now copy the contents of the hacked uploads folder into the clean one:
    cp -R ~/blog.amhill.net.hacked/wp-content/uploads/* ~/blog/wp-content/uploads/

That’s it!

 Re-activating the Website

At this point, you should be able to navigate to your blog at its URL and it should work! You may see only a blank page if you were using a non-default theme, but it at least should not give you an error.

Navigate your browser to your admin area, and log in as your admin user, just like normal. First, we’re going to refresh the permalinks:

  1. Click on “Settings” then “Permalinks”.
  2. Click “Save Changes”.

Next we’re going to force it to use a default theme, for now.

  1. Click on “Appearance”
  2. Click on “Themes”
  3. Pick any of the default themes (eg. twentyeleven) and click “Activate” for that theme

Lastly, we’re going to verify that all plugins are disabled:

  1. Click on “Plugins” and choose “Installed Plugins”
  2. Click the checkbox at the top of the plugins list to select all
  3. Under “Bulk Actions” choose “Deactivate”
  4. Click “Apply” — all plugins should be deactivated now.

Navigate back to the public-facing side of your blog and verify that it is indeed showing your data, albeit in a very boring and lame way. That’s good, at least we’re stable now.

One final thing. We need to verify that there were no additional users created, and also change the passwords of any existing users.

  1. Click on Users, then All Users
  2. Look for any strange users — there should be a single Administrator account, and any other accounts should be Editors. Delete any accounts that look strange, and on the remaining accounts change the passwords.

Further Reading for the Ambitious

If you are feeling salty, I highly recommend running through the Hardening WordPress guide over on the official WordPress codex. These will help tighten down the security even more and hopefully prevent future attacks. In particular, definitely do the file and directory permissions in that article.

I’m going to do some logfile analysis and see if I can identify how the attacker got in, since my WordPress was the most current version at the time of the attack.

Cheers!