Wednesday, 14 November 2012

New York Outages

New York and New Jersey suffered a lot of damage from Hurricane Sandy. Even now a couple of weeks after the impact of the damage is still effecting  businesses. Apart from the obvious damage to timber frame buildings, the main damage effects appears to be:

  • Loss of power
  • Lack of fuel
  • Results of flooding.
While I have enormous sympathy for those involved I can't but help think many of the after effects of this storm could have been mitigated or avoided if there had been some proper planning and infrastructure provision beforehand. Much of the problems seem to arise from trying to provide infrastructure on the cheap. Saving pennies has cost many more dollars. At some point in history an accountant may have shaved a few dollars from the budget for a company's infrastructure, but that same accountant will not stand up and proclaim "Mea Culpa" when the business is out of action for a couple of weeks as a consequence of those "savings". There are those who say this could be covered by business continuity insurance because it is only a 100 Year Storm. You'd trust the insurers to cover all costs and rebuild your company? Not!

Let's first look at fuel. In the case of Gasoline there were service stations which could not pump fuel because the pumps depended on mains electricity from the Utility Company. When the power went down the service stations became useless. There should be federal laws which require the provision of  weather protected standby generators at service stations to provide protected power to pump the fuel.

Then there was the problem of fuel shortages. The supply of gasoline in normal circumstances is a "just-in-time" facility where the service stations depend on regular delivery of fuel every few days by tanker trucks. If that delivery service is interrupted the service stations hold only a couple of days supply and quickly dry up. The situation is worsened by panic buying by consumers. When everyone decides to fill their gas tank to the top it places strains on the limited supplies causing them to run out more quickly. It leads to rationing such as this case. This whole situation could be mitigated if service stations were forced by law to hold a minimum of 15 days stock in their underground tanks. The 15 day buffer would make fuel supplies considerably more resilient to supply shock. The finance cost of holding 12 extra day's supply is not massive.

Your organisation may have standby power generators with a 24 hour fuel supply with a resupply contract with a local fuel depot. Have you actually checked whether the fuel supplier can honour those commitments in the event of a regional disaster such as Hurricane Sandy? Flooding might prevent despatch of the trucks. I recently saw a comment on a LinkedIn forum where an organisation fuel resupply company could not pump fuel into the delivery tankers because they also had lost mains power from the utility company. You should diversify suppliers and actually check their delivery arrangements in the event of a regional disaster, don't rely on contractual obligations. If you can extend your own fuel capacity to five days you'll give yourself more flexibility in the event of a disaster.

The regional loss of power is almost inevitable in the event of a disaster such as Hurricane Sandy. Power lines will collapse and the utility company will cut power to protect substations. You should ensure access to local power generation facilities. Perhaps your own standby generator or maybe a facility shared between local businesses. The standby facility should be regularly tested with total power down scenario's. You should walk through what would happen if the basement of your building becomes flooded. A co-location hosting centre discovered this to their cost. In some cases heroic efforts such as a human bucket chain may be necessary to keep the generators running.

Lloyds - the insurance people; have written a telling paper on energy security paper. It is worth visiting their site to download a copy to read.

Flooding can strike most businesses. You may be in a low lying coastal area, a river flood plain. You may have a major water main pipe running under the street outside of your front door with the risk of pipe burst. In storm conditions damaged roofing can allow rain water leakage. If there is risk you should look at: 
  • the arrangements you have to prevent the water from reaching your key machinery; this might include high sill doorways, sealing of leakage points, tanking of the room floor and walls.
  • the arrangements you have to quickly drain any flooding in your building. It might include pre-laid pipes, drainage sumps in equipment rooms and submersible drainage pumps. 
  • leak detection equipment and alerts;
  • overhead drip trays which can help prevent overhead water from leaking on to sensitive equipment. 
They plan for these type of flooding incidents in Australia. In New York a substantial about of phone/data cabling was destroyed by salt water flooding in a Verizon below ground level cable vault (90,000 cubic feet). The compressed air pumps, which help to keep the copper data cables dry, failed allowing salt water to saturate the insulation and destroy the data communications. Most of the old cable will have to be chopped away and replaced with optical fibre.

No comments:

Post a Comment