When A&R Logistics, the largest transporter of plastic resins in the U.S., began experiencing unexplained, catastrophic power problems in its Chicago-area data center, company executives had to act fast.
"We have 750 drivers on the road, from San Diego to Boston and everywhere in between," said Rick Blanchard, vice president of technology at A&R. "They need to know what products to load on their trucks, and where and when to deliver them. If a driver doesn't show up on time with the right products, that can literally shut down manufacturing operations for hours, or longer."
"When that happens," Blanchard said, "typically we don't get a second chance."
A&R, which was founded in 1969, implemented a Disaster Recovery as a Service (DRaaS) solution from Peak 10, allowing the company failover its production IT environment to a high-performance disaster recovery infrastructure that would enable the company to provide uninterrupted service to its customers.
A good disaster recovery (DR) program is critical for any type of organization today. Given the current cyberthreat environment, as well as the potential for a broad range of natural disasters, businesses must be prepared to carry on whether they are hit with random power outages, ransomware or a Category 4 hurricane.
While there are a broad range of DR solutions available, deciding on the best approach to DR really comes down to the organization and its priorities.
"Before you begin developing your DR plan, it is crucial for IT and business leaders to understand the overarching business plan and objectives," said Steve Renda, vice president of product at Peak 10. "Plus, every department within an organization contributes its own portion to the business continuity management plan. Ultimately, the plan must operate in tandem with the organization's business strategy, otherwise, business leaders will be attempting to recover things that were, not things that are."
Avoiding data disasters
Data backups are perhaps the most critical aspect of a DR plan. No matter what happens, if a company has a complete backup of its data in an offsite — and hopefully unaffected — area they are much more likely to be able to recover and resume business as usual.
A&R originally wanted to plan its failback to its production environment in Chicago. But because the power problems in the data center persisted, Peak 10 proposed an alternative: recreating A&R's production environment at Peak 10's colocation facility in Louisville, near A&R's corporate headquarters.
Once the production environment was in place, A&R and Peak 10 set up disaster recovery infrastructure in Atlanta. In the end, A&R had recovered to the cloud, reestablished its production IT in a colocation facility, and implemented a permanent remote DRaaS infrastructure.
When considering disaster recovery options, location becomes a key determining factor. For ThreatMetrix, a cloud provider that secures and authenticates digital identities for more than 2 billion online financial transactions each month for 4,500 customers around the world, the location of its backup data matters.
As an international company, ThreatMetrix has to deal with new and emerging data privacy requirements such as the European Union's General Data Protection Regulation (GDPR). The company needed a location that was both EU-friendly and highly secure given that many of its customers are financial institutions.
ThreatMetrix chose to work with Verne Global, which houses its data center in Iceland. Iceland was recently voted the world’s safest location for a data center out of 37 nations in the 2016 Data Center Risk Index, published by Cushman & Wakefield.
"For a lot of our customers, we are a key part of their DR plan as well," said Phil Steffora, chief security officer at ThreatMetrix. "We are in the critical path for a lot of large businesses across all types of industries, from e-commerce, to banking, to tax prep, etc. So we take the responsibility of having that availability very seriously."
Working with Verne Global also provides ThreatMetrix another advantage: a greener solution. Verne Global uses renewable energy sources to run its Iceland-based data center, and for ThreatMetrix, that was an important consideration as well.
Testing, testing, testing
Once a company has chosen a DR solution or chooses to construct a plan in-house, testing is key. But testing can be difficult because there aren't many viable options for businesses looking to train professionals in active cyber defense.
"Most cybersecurity assessment programs, while well-intentioned, are highly theoretical and based on known cyberattack practices. The reality, however, is very different," said Chris Thompson, senior managing director and head of financial services cybersecurity and resilience at Accenture Security. "Fast-moving, dynamic threats are creating new challenges every day.”
Nearly 80% of bank executives said they were confident about their cybersecurity strategies, but lacked the ability to test their strategies in a real-world testing environment, according to a report released by Accenture in April.
Some companies, such as IBM, are looking to help solve this problem. Last fall, IBM Security announced plans to invest $200 million in its incident response capabilities, including a new headquarters and a Cyber Range where participants can get hands-on, real-world experience responding to cyberattack using live malware, ransomware and other hacking tools.
"Scenario-based real-world testing has provided us some of the best insights into our DR plans," said Steffora. "We do scenarios where we move our entire management team offsite and reconstitute our business from a remote location. We test that we can do those things, and we learn things along the way each time we do it. Planning becomes better with each iteration."
It is essential for a company to test its DR plan in various realistic scenarios, agreed Renda.
"Many companies assume that simply designing and implementing a DR plan and solution is adequate, but our experience with our customers tells us differently," Renda said.
"From a plan standpoint, a company really benefits from ensuring that their plan anticipates every scenario that might occur. They also need to ensure that their applications behave in a recovery scenario as they anticipate, which can be very different than they behave in a steady state environment."
