No one likes to discover that their website has been hacked. It’s frustrating, and can be time-consuming and expensive to fix. You might be asking yourself, “How could this happen?” “Why did they target us?” and “What could we have done differently?”
An Attack is a Learning Experience
Many organizations are embarrassed that their site was breached and remain reluctant to admit it to anyone. Smaller organizations in particular find the discovery of their site being hacked as something they need to quickly recover from, then bury in the back corner of corporate history. While this may seem to make sense from the perspective of company image and branding, it is actually to one’s detriment.
Burying previous security breaches makes it that much more difficult to foster an internal culture, where you can better protect yourself in the future. You need to be open about what failings led to your site being hacked in the first place and take steps to prevent a recurrence.
- Was the site code out of date?
- Were the permissions shared too widely?
- Did you fall victim to a widely known exploit?
No technology is untouched by human hands, so (again) owning the failures that allowed the hack to occur will better prepare you to prevent future attacks.
No Site Goes Unhacked Forever
There is a quote about computer security that goes like this:
“The only secure computer is one unplugged, in the middle of the desert, and buried under 20 feet of concrete. And I’m not sure about that one.”
Hacking is inevitable. Your site will be hacked. If it hasn’t been yet, it will be in the future. Attacks are rarely correctly portrayed in films, where someone in a dark, damp basement finds some clever way to destroy your site. Most often, a site will be breached by a collection of autonomous systems -- a botnet -- repeatedly trying known exploits against any site that appears to be susceptible.
Drupal, a well known content management system, has a dedicated security team. They discover and receive reports of possible exploits on a regular basis. The largest in recent history, Drupalgeddon, affected a large number of sites. Even my personal blog was hacked days after the exploit was announced and patched.
So, your site has been hacked. It’s real. It’s happened. Now what?
Recovering From a Backup
Once your site has been hacked, you must treat the entire system as compromised. The server, the files, the database are now effectively poisoned. Even if you know the exactly when the breach occurred, how it was performed and what was immediately affected, additional exploits might have been installed as backdoors.
If you are fortunate, you will have a backup from hours or days before your site was hacked. If so, your first job is to revert to the last backup before the hack, called the Last Known Good State:
- Lock down the compromised site or take it offline completely. You do not want to risk your users being exposed to the hack and have their Personally Identifiable Information (PII) shared with a nefarious third party.
- Carefully migrate any content created since the hack. Text can be verified by eye, but in the case of file uploads it is best to request new ones from the post’s original author.
- Apply any security patches or software updates to your site. Otherwise, your site will remain susceptible to a botnet. Worse, your site is now on a list of known exploitable sites by the attacker, and will be re-hacked again without vigilance.
- Invalidate all passwords. This includes database credentials, SSH logins, and all user passwords. Any one of these could have been captured by an attacker.
- Bring the reverted, updated site back online. Again, be honest and open as to what happened to your site. Communicating what happened, what you did to recover, and the mitigation steps you are taking to reduce the likelihood of future attacks can actually increase customer loyalty as it shows transparency and honesty.
What If I Don’t Have a Backup?
If you don’t have a backup, or a very old backup, it might not be worth trying to recover from the Last Known Good State. Instead, you’re going to have to accept that you cannot guarantee you have removed all possible exploit points. Often, this process can be like pulling weeds: You pull one clump, and days later you find another had sprouted a short distance away.
Simply updating your site’s software to the newest version isn’t enough, as exploit code may remain in un-updated portions. For Drupal sites, this can be contrib modules, theme code, or custom modules written to support core site features. Sometimes, an attacker will duplicate an installed module in your sites/all/modules directory and create an exploited copy in sites/default/modules, making it harder to discover. Particularly insidious intrusions will create users accounts that do not appear in admin pages, install serialized exploit code in cache tables in your database, and other ingenious methods.
If you do not have a backup, the best you can do is attempt to rebuild the site:
- Completely remove all Drupal core code and contrib modules. All code from original site is considered poisoned and must be removed.
- Download new copies from Drupal.org. Be sure to download the same major version for each, otherwise your be rebuilding and updating at the same time.
- Carefully examine file uploads. Exploits can be hidden in your file upload directory if your site was compromised. Typically, web server settings prevent their execution, but it’s still good to check.
- Run security audit tools. Use Hacked!, Site Audit, and the Drupalgeddon modules to find additional exploit points in your database.
- Clean out database-stored exploits. This requires some expertise!
- Update Drupal core. This should be to the most recent version within your major version (i.e. update from 7.42 to 7.54, but not 7.42 to 8.3).
- Update contrib modules. If warranted, update to the most recent major version.
- Audit custom modules and themes. Exploit code might be seeded here, and must be manually verified.
Know When To Ask For Help
Cleaning out known exploits from your site requires experience and time. It is important to know when you’re in over your head and need to call in some outside help.
If you are facing the worst-case scenario, you could be looking at a complete site rebuild and a painstaking examination of your database and file uploads. Contacting a reliable web firm can help this process go faster, have fewer side effects and be more complete. While the remediation costs may be expensive, the investment can save you time and money and avoid damage to your brand in the long run.
Create a Recovery Plan
You’ve been hacked. You’ve recovered. You will be hacked again -- but you can recover more easily the next time.
If your hosting provider or agency provides regular backups of your site, enable them. When my personal blog was hacked, I was fortunate to have a backup from the previous week. This considerably reduced my recovery time. Maintaining regular offsite backups is the foundation of all exploit recovery.
When hack is discovered, know how to react. Develop a contingency plan to place an exploited site in read-only mode, or a kill-switch to take it offline completely. Try to minimize the amount of time a compromised site remains online, as it can damage your users and your brand. Practice performing a full recovery on a regular basis to verify your procedures are working.
Apply Updates Regularly and Promptly
Your website is not a printed book, sitting on a shelf in a pristine library. It’s more like your dishwasher. You don’t think much about it when it’s working well, but you certainly do when you’re stuck washing plates by hand! It requires regular maintenance, care, and sometimes repair to continue bubbling along.
Outdated software is one of the primary reasons why hacks occur. Exploits are often discovered by security team members or agencies. Once a patch has been developed and announced, exploiters also update their toolkits to compromise unpatched sites. This can be done on a massive scale due to Cloud Computing. You need not be a high-value target to be attacked; even personal blogs can be can used as adware, driving search results for profit.
If your organization is facing a knowledge gap in maintaining your site, it may be time to contact an experienced web firm such as TEN7. Our on-going support services provide timely security updates and rapid deployment to your existing hosting provider. Our support contracts include Tractorbeam; a multi-tier, comprehensive off-site backup service to help you recover from hacks, hardware failures and geographic disasters.
Summary
No one likes to discover their site has been hacked. But life and your site do go on. By knowing how to recover from a serious breach, developing a contingency plan for the next time, and performing regular updates, you can reduce the likelihood of being hacked in the future as well as the pain and panic that goes with it.
