If one system fails in a data centre, the knock-on effect can be huge. Queen’s University, Ontario, can attest to this. What began as an air-conditioning system failure ended in their data centre being offline for about 20 hours.
In late July last year the air-conditioning system used in the Queen’s University Data Centre began to leak. This malfunction caused mineral oil from the AC compressor to be sprayed into the room. The oil created a fine mist as it was sprayed; when this mist came into contact with sensors for the fire suppression system it was mistakenly interpreted as smoke. The fire suppression system was activated, the data centre was shut down and the fire services were called.
IT staff at the university worked extremely hard and were able to bring the data centre back online for the beginning of the working week. However, the damage had been done. Many students were unable to use the university’s online learning management system and were unable to register for courses. The fire detection sensors also suffered damage as a result of contact with the mineral oil so needed to be replaced.
Whilst the sequence of events that occurred at Queen’s University were highly unusual, it is not uncommon for an AC failure to cause water damage and issues as a result of overheating. However, by closely monitoring for water leaks and unusual temperature spikes it is possible to react to potential threats before disaster strikes! The cost of network downtime and hardware replacement means that these monitoring systems often pay for themselves after just one disaster averted.
References: The Whig