How to Prepare Business Infrastructure for Crisis

TL;DR

Prepare for severe disruption by identifying cyber and operational risks, forming a cross-functional emergency team, and documenting business continuity and disaster recovery procedures before a crisis begins. MacPaw also reduced dependencies by moving office infrastructure to the cloud, establishing backup communications and payroll providers, buying spare laptops, arranging satellite internet, and restricting production changes through a controlled code freeze.

Transcript

What if we could see cyber threats in real life? Yesterday's technology can no longer stop them, but they can be stopped with consistent cybersecurity that protects work everywhere. Zero trust with zero exceptions, by Palo Alto Networks. Hello and welcome to this installment of our RSAC 365 webcast series. I'm your host, Casey Zirkus, senior conten... Read More

Key Insights

• Early preparation is driven by interpreting warning signs as business threats, not merely consuming news. MacPaw began planning around three to four months before the war after reviewing invasion warnings, predicted dates, and satellite images of military forces gathering near Ukraine's borders.
• Risk assessment is most useful when threats are translated into specific consequences. MacPaw divided its exposure into cyber and operational categories, covering attacks, unauthorized access, malicious changes, staff availability, payroll disruption, connectivity loss, sanctions, mobilization, and hardware supply problems.
• A cross-functional emergency team is responsible for maintaining company stability and operations during a crisis. MacPaw selected people from different services and products, gave them emergency responsibilities, and asked them to relocate to western Ukraine or outside Ukraine before the war when possible.
• Business continuity and disaster recovery plans can use simple, accessible formats. MacPaw documented plans in Google spreadsheets that identified covered assets, scope, step-by-step procedures, a primary procedure, and a backup procedure to follow if the main approach failed.
• A controlled code freeze reduces the possibility that routine changes will destabilize working systems during an emergency. MacPaw limited infrastructure and product changes to emergency-team members, while other employees needed the team's approval before making changes considered sufficiently safe.
• Communication redundancy is essential when the primary workplace channel may become unavailable. MacPaw retained Slack as its normal company platform and selected Signal as an alternative, asking roughly half a thousand employees to install it on personal devices before the emergency began.
• Cloud migration reduces dependence on offices that may lose power or become inaccessible. MacPaw moved its remaining office infrastructure, including Mac Minis used by quality assurance engineers, continuous integration and delivery agents, and an on-premise VPN, into the cloud before the war started.
• Operational resilience requires backups beyond technical systems. MacPaw bought laptops in advance, arranged satellite internet, prepared multiple payroll providers, drafted customer delay notifications, and rented coworking space in western Ukraine so part of the team could relocate to a safer region.

Questions & Answers

Q: How should a company begin preparing for a major crisis?

A company should begin by treating credible warning signs as potential business threats and asking what consequences could follow. MacPaw reviewed reports of a possible Russian invasion, assessed related cyber and operational risks, and then created mitigation plans. Its preparation started around three to four months before the war and included an emergency team, continuity procedures, disaster recovery plans, and infrastructure changes.

Q: What cyber risks should an emergency preparedness plan cover?

MacPaw's cyber risk list included attacks against web services and corporate social accounts, unauthorized access through lost or confiscated devices, unauthorized production infrastructure changes, malicious code injection, source code theft, and phishing. These risks were especially relevant because the company had a strong pro-Ukrainian position and anticipated that evacuation or occupation could expose devices and systems to additional threats.

Q: What operational risks did MacPaw identify before the war?

MacPaw identified the possibility that key employees could become unreachable, payroll could be interrupted, or mobilized team members might stop working. The company also considered sanctions, its status as a company operating in a war zone, loss of internet connectivity, and disruption to the hardware supply chain. Each identified risk became a basis for creating a corresponding mitigation plan.

Q: How does an emergency team support business continuity?

An emergency team gives selected employees one primary responsibility during a crisis: maintaining company stability and operations. MacPaw formed its team with people drawn from different services and products. When possible, those employees were asked to move to western Ukraine or leave Ukraine before the war, helping ensure that essential decisions and operational work could continue from safer locations.

Q: Why should a company use a code freeze during an emergency?

A code freeze limits changes that could cause failures when personnel, communications, and recovery capacity are already under pressure. MacPaw created a special operating mode in which ordinary employees could not change products or infrastructure. Emergency-team members could make changes directly or approve another person's change after deciding that it was safe enough for the circumstances.

Q: How can a company make crisis communications more resilient?

A company can make communications more resilient by preparing an alternative channel before its normal platform becomes unavailable. MacPaw used Slack for regular company communications and chose Signal as its backup. Employees were asked to install Signal on their personal devices in advance because coordinating installation across a company of roughly half a thousand people would not be a quick action during an emergency.

Q: Why did MacPaw move its office infrastructure to the cloud?

MacPaw moved office infrastructure to the cloud because its office could become inaccessible, occupied, or affected by power outages. The remaining local systems included Mac Minis used by quality assurance engineers, Mac Minis acting as continuous integration and delivery agents, and an on-premise VPN. Moving these resources outside the office reduced dependence on the Kyiv location, and the migration finished before the war began.

Q: What practical backups can protect operations during a disaster?

MacPaw prepared several operational backups for different failure scenarios. It bought laptops in advance to address hardware supply disruption, arranged satellite internet for possible connectivity loss, and made additional payroll providers available in case normal payments were interrupted. It also rented coworking space in western Ukraine, prepared customer notifications about potential response delays, and documented both primary and backup recovery procedures.

Summary & Key Takeaways

• MacPaw began preparing three to four months before Russia's full-scale invasion after foreign media and intelligence agencies warned of a possible attack. Although company leaders hoped war would not begin, they treated the warnings as a credible threat, assessed the resulting business risks, and created mitigation plans before the emergency occurred.

• The risk assessment separated cyber threats from operational threats. Cyber concerns included attacks on web services and social accounts, unauthorized device access, production changes, malicious code injection, source code theft, and phishing. Operational concerns included unavailable staff, interrupted payroll, mobilization, sanctions, war-zone status, internet loss, and hardware supply disruption.

• MacPaw established a cross-functional emergency team, business continuity and disaster recovery procedures, customer notifications, a controlled code freeze, backup communications through Signal, cloud-hosted infrastructure, spare laptops, satellite internet, additional payroll providers, and coworking space in western Ukraine. Existing remote-work readiness from the COVID period reduced the amount of additional preparation required.