A couple of months ago, a friend mentioned that his company’s website was under attack. The CEO had received an email threat by a person claiming to be from Russia. He went on to threaten further DDoS attacks if the CEO didn’t send 20 Bitcoins. To prove his point he took down the company website for an hour that afternoon and followed up with a similar attack the next day to show he meant business. The owner was at a loss, and hoped the hacker would move on to bigger fish.
The DDoS attacks continued against this small company that relies on its website to reach customers. The outage, while not long, caused panic around the company. I spent the next week or so gathering updates from my friend on how they were dealing with the attacks. When the CEO refused again to pay the extortion fee, the attacks became more intense and increased in duration. The CEO considered paying the fee, hoping that would bring an end to the attacks, but he was concerned it would only be the beginning of further extortion attempts.
This week I want to take a look at how MSPs, VARs and other IT providers can assist small companies as they come under a DDoS attack. Better yet, what services can you offer to help prepare for such an attack? How many of you have sat with the owner of a company and explained just how devastating a DDoS attack can be? These are not easy conversations to have, but they are necessary and helpful. It’s almost too easy to say, “Oh, that won’t happen to us. We are too small!” But imagine if someone attacks your site during the busiest time of the year? The results could be financially devastating.
What Didn’t Work
I want to state up front that I’m not a network administrator. I’ve setup my fair share of temporary networks during my time as a technical product manager at Microsoft for events such as product launches, trade shows, and conventions. A group of us traveled around the globe installing and managing event networks that were in place for one or two weeks. So I’m familiar with basic network designs, but mostly for temporary networks.
When the attack hit the company, customers began calling and complaining they could not access the website. Those who called were those most interested in placing orders so sales didn’t exactly grind to a halt. But the company’s reputation took a hit. Employees didn’t realize there was a problem because all internal servers were humming along fine.
Google provides a detailed Digital Attack Map that shows real-time DDoS attacks.
The CEO decided to ignore the hacker. He also refused to pay the Bitcoins the hacker was demanding in exchange for halting the attacks.
At this point, the CEO notified their ISP to see what could be done. The ISP didn’t seem to grasp what had happened and was of little help to stop further attacks. The CEO decided to send all their traffic through a service that would mask their IP. The service would filter the traffic and only pass along the legitimate traffic to the company. In basic terms, it was a proxy/filter service that comes with several other benefits such as caching static parts of the website. While nobody was convinced this service would work, it was the least expensive option.
A few days later the attacks began again, and this time they lasted for several hours and were more intense, making the website unreachable rather than merely sluggish. It was clear the proxy service was no match for the hacker who was kind enough to send another email explaining how easy it was to bypass the protections the company had put in place. He lowered the amount of Bitcoins to 16 and promised to tell the CEO how to setup his network to thwart a similar attack in the future. A hacker with a morals!
Like clockwork, the hacker would notify the CEO when he planned to take down the website, and when that time came, the site was attacked and taken offline. This went on for a couple of weeks. Company morale was low and things were not looking good.
What Did Work
Over the past decade the company’s website had never been down for more than 30 minutes. But over the past couple of weeks, the site had been taken down for several hours in the middle of the day. Support technicians were exhausted from dealing with confused and upset customers. Sales personnel were flooded with calls from people who wanted to place orders but were blocked from doing so on the website.
Like many small companies, this one did not have a dedicated IT team. They have programmers and designers and engineers who manage the website’s content and keep the servers running. But they did not have anyone versed in how to deal with a DDoS attack, so they looked for help from a local security consultant. The consultant spent a couple of days at the company observing protocols, running tests on the network and diving into the network logs to better understand the nature of the attack.
The only hardware that was installed onsite was a WatchGuard Firewall.
Once he felt like he had an understanding of the threats facing the company’s networking he implemented a number of changes. I can’t go into great detail about them here, but I understand the few remaining public servers were brought in behind the company’s firewall. The company’s chat server and VOIP servers were replaced with cloud-based equivalents. Going to a CDN to deliver content from multiple data centers to our customers was the next move the company website. The biggest expense was installing a WatchGuard Firewall. Once these actions were taken, the attacks stopped and have not returned.
How MSPs Can Help
Had this company been working with a MSP, there’s a good chance the attacks wouldn’t have happened. The CEO was caught completely off-guard and unprepared for the attack. An MSP could have helped explain the risk this company was taking by running public servers, not using an CDN and failing to have an adequate firewall device in place. These are all steps that an MSP can help a CEO or company owner understand the importance of.
MSPs can help small businesses utilize networking monitoring services are available so the company can be the first to know when there’s an problem. One service the MSP might offer is to install and manage these services while the owner runs his business. When an attack reaches a certain threshold, the MSP can act to mitigate downtime on behalf of the company. The MSP isn’t just managing a networking, but also the company’s reputation.
Along with helping small businesses manage their network, MSPs can provide valuable training inside the company. Sharing best practices when it comes to network management and threat protection are services companies of all sizes need. Knowing that someone is monitoring their networking, analyzing threats and keeping them in the loop can provide peace of mind to an already overworked small business owner.
MSPs may want to consider targeting these type of services to small companies with a strong e-commerce presence and high revenues but that maintain a small IT staff. These are the sites which are most vulnerable to DDoS attacks today. It’s difficult to know how may company CEOs decide to wave the white flag and pay the hacker what he wants, but that’s a terrible long-term solution.