Internet Down!

How to prepare for the next internet outage.

On Tuesday, July 9th, 2024 at approximately 12:05pm, Time Warner Spectrum experienced a statewide outage of internet and voice services.  This outage impacted thousands of businesses, banks, health care clinics, and numerous others across Texas that relied solely on internet carrier services from Time Warner Spectrum. 

Many of those businesses that experienced this outage came to realize just how dependent their business is on the internet.  Were you one of them?

However, some businesses were prepared for this event and did not experience this internet outage.  These organizations implemented a backup internet failover solution that automatically switched over to another internet carrier.  These clients continued to work as usual and did not experience this Spectrum internet outage. 

To ensure your office remains productive during such disruptions, here are best practices focusing on failover solutions, resiliency, redundancy, and the essential processes to maintain operations.

1. Implementing Failover Solutions

Failover solutions are essential for maintaining internet connectivity when your primary internet service experiences an outage. Here’s how you can set up effective failover strategies:

Secondary Internet Connection: The most straightforward approach is to establish a secondary internet connection with a different internet carrier (e.g. AT&T, Astound Broadband, Google, etc.) This ensures that if your primary internet fails, the secondary connection can immediately take over. For optimal results, choose a provider that operates on a different network infrastructure to avoid overlapping vulnerabilities.

Automatic Failover Devices: Utilize automatic failover devices such as Dual-WAN routers or load balancers. These devices can seamlessly switch traffic from your primary internet to the secondary one without manual intervention. Configure these devices to prioritize your primary connection and automatically switch to the secondary one only when the primary fails.

Mobile Hotspots: As a backup to your main and secondary internet, consider using mobile hotspots or 4G/5G LTE connections such as AT&T or Verizon. While they may not match the speed and bandwidth of your main connections, they can provide essential connectivity during emergencies.

2. Building Network Resiliency

Network resiliency ensures that your IT infrastructure can withstand and quickly recover from internet disruptions. Key strategies include:

Redundant Hardware: Invest in redundant hardware components such as switches, routers, and firewalls. This setup helps to ensure that if one piece of equipment fails, another can take over without impacting your network’s operation.

Regular Testing: Conduct regular failover tests to ensure that your backup solutions work as expected. Simulate outages to verify that the failover processes are smooth and that all systems automatically switch over without data loss.

Cloud-Based Applications: Where possible, rely on cloud-based applications and services that can be accessed from anywhere. This approach reduces the impact of local internet outages by allowing employees to continue working from different locations or devices.

3. Ensuring Redundancy

Redundancy is a core component of any failover strategy. Implementing redundant systems involves:

Dual Internet Connections: As mentioned earlier, having two internet carriers is crucial. Ensure that your secondary carrier is not dependent on the same infrastructure or backbone as your primary carrier. This strategy minimizes the risk of simultaneous outages affecting both connections.

Diverse Routing: Use diverse routing paths to connect to the internet. For instance, one connection might use fiber optic cables while another uses copper or satellite connections. Diverse routing helps avoid a single point of failure affecting your entire connectivity.

4. Firewall Requirements for Redundancy

To implement a redundant internet carrier setup, you’ll need a firewall capable of handling multiple WAN connections:

Multi-WAN Firewalls: Opt for a multi-WAN firewall that supports load balancing and automatic failover. These firewalls are designed to manage traffic across multiple internet connections, ensuring a seamless transition between your primary and secondary internet providers.

Configuration: Properly configure your multi-WAN firewall to recognize and prioritize your primary internet connection. Set up rules for automatic failover to switch to the secondary connection when necessary and monitor performance to ensure all traffic flows smoothly.

5. Cost vs. Benefits of Redundant Internet Carriers

Investing in redundant internet carriers can be costly, but the benefits often outweigh the expenses. Consider the following:

Cost Considerations: The costs of maintaining multiple internet connections, along with the necessary hardware and configurations, can add up.  For small to mid-sized businesses, this might involve substantial ongoing expenses.

Business Continuity: The primary benefit of having redundant internet carriers is business continuity. An outage can lead to lost productivity, missed deadlines, and customer dissatisfaction. By having a failover plan, you protect against these risks and ensure that your operations can continue smoothly even during disruptions.

Risk Management: For many businesses, the risk of downtime outweighs the cost of redundancy. Evaluate your company’s tolerance for downtime and potential financial losses to determine if investing in redundant carriers is a prudent decision. 

A good way to measure the cost vs benefit is to determine the cost of one hour of lost productivity, sales or invoicing.  When compared to the cost of an additional internet carrier - often less than $100 per month – it can be easy to justify. 

* * * * * * *         

 Mitigating internet outages requires a strategic approach to failover solutions, network resiliency, and redundancy. By investing in secondary internet connections, leveraging automatic failover devices, and using robust firewalls, you can safeguard your business against the disruptive impact of internet carrier failures.

RIATA can help your company with implementing these failover solutions and ensure more resiliency and redundancy of your network and internet access.  Contact RIATA today and we’ll be glad to provide a no obligation consultation to help you better understand your options and total costs.

About the Author:  Tommy Wald is CEO of RIATA Technologies, a Managed IT Services Provider headquartered in Austin, TX.  He can be reached at TWald@RiataTechnologies.com or (737) 249-9697.