Network Disaster Recovery

Equipment

Each NDR technology trailer contains a telecom infrastructure element that is present in a normal AT&T central office. Installing, configuring and testing the equipment in advance eliminates the delay of having "first-off-the-line" equipment sent from vendors after an office has been lost, and it also eliminates the need to find a suitable replacement building in an area devastated by a disaster. The recovery trailers allow the equipment to arrive at a stricken city ready for service, in powered containers that, when interconnected, serve as a temporary network office.

The equipment is installed in bays/racks down each side of the trailer, leaving a working aisle down the center—like the layout of equipment bays in a permanent office. The equipment is powered by battery, through a rectifier, and the batteries are constantly charged from commercial power, from a large portable generator (e.g., 600kW) or from a dedicated generator (usually built in to the front of each trailer). Fiber-optic and/or copper-T3 cabling connections to the equipment are made in internal and/or external cabling bulkheads on each trailer.

The technologies present in the trailers have evolved since the early 1990s following the changes in the AT&T network. The NDR recovery fleet has more than 320 pieces of recovery equipment, including thirteen trailers dedicated to the restoration of our IP network, as well as high-speed fiber-optic "transport" trailers. These IP recovery trailers support our Common Backbone Network (CBB) with traffic traversing at data rates as high as 100 gigabits. When fully-equipped, the IP trailers can scale up to a capacity of over 15 terabits per second.

AT&T's AGN network in the global markets is supported with disaster recovery trailers staged in Europe. Some of the AGN equipment is installed in fly-away containers that can be shipped by commercial air carrier. Following the earthquakes in Chile in February 2010, AGN recovery nodes were flown to Santiago and set up near a permanent telco office in case aftershocks made that building, and its equipment, unusable.

NDR's recovery equipment is maintained in warehouses across the United States and in Europe—positioned strategically so the equipment can reach AT&T's network offices quickly. The equipment doesn’t sit idle. It is kept powered-up and on-network so updates can be installed and so it can be tested both on-site and remotely. NDR's warehouse teams are responsible for the health and welfare of the equipment inside the trailers and for the running gear that lets the equipment travel. It's a unique blend of skills that crosses the spectrum of tasks from machining parts to configuring core network routers.

Emergency Communications Vehicles

An NDR ECV and charging station deployed in Tallahassee, FL in 2016

NDR establishes broadband and wireless voice and data connectivity from disaster sites using one or more Emergency Communications Vehicles (ECV). An ECV uses a satellite link to provide NDR with command communications during the initial phase of a recovery effort. The ECVs have also been used to provide command and humanitarian relief communications capability to other emergency responders.

Satellite COLTs

NDR satellite COLT deployed for the Pine wildfire in California in 2016

AT&T uses Cells on Wheels (COWs) and Cells on Light Trucks (COLTs), self-contained mobile cell sites, to provide extra cellular capacity to restore communications after a disaster. The mobile sites can be used to replace the service of a failed permanent cell site and they can be used to supplement the cellular capacity of an area that has increased demand. The NDR team uses Satellite COWs and COLTs to establish first-in communications when terrestrial connections to the AT&T Network are not immediately available.