The Least of Their Worries

With a monster storm on their hands, the folks at Acadian Ambulance had plenty to worry about in the final days of August and the first few of September 2005. In the wake of Hurricane Katrina, the coastal areas of southeastern Louisiana and Mississippi were shattered, New Orleans was flooded and the company's employees were scrambling to navigate stricken areas and get the sick and injured out and to safety.

     The last thing company officials needed to worry about was keeping an eye peeled for trouble with their computer-aided dispatch system.

     But then, they didn't have to. Their CAD vendor, San Diego-based TriTech Software Systems, was watching from afar.

     "When the hurricane hit, we were constantly monitoring their systems," says TriTech President/CEO Chris Maloney. "We have different levels of support we can offer to clients, and when we know a client is going through a major event, we escalate their support level and go into proactive monitoring. We dial in to their system periodically and check to make sure it's healthy. We monitor internal systems to see if there are any issues, or potential issues, with the database, the software, the hardware or the networking."

     Under normal circumstances, this monitoring happens on an as-needed basis--basically when problems are detected. Proactive monitoring entails TriTech staff regularly dialing in to a user's system to make sure everything's functioning as it should.

     "We have about 20 different checklist items that we go through," says Maloney. "We have software on the system that constantly monitors the system's health, and then about once an hour, we dial in and check on those software components to find out whether everything's running properly or there are indications that the system is going through some sort of degradation. If, for example, a system starts having performance problems because of increased load, we can see that, to a certain extent, proactively. And then we can advise the customer of any repairs or preventive measures they need to take to keep the system operational."

     Acadian's system functioned smoothly during Katrina and its prolonged aftermath, but at one point, TriTech did detect a possible problem.

     "They saw we were throwing some errors on one of our servers," says John Zuschlag, Acadian's senior vice president of support services. "It was something that wasn't actually affecting the system, but when we got into a lull--which actually did happen, believe it or not--we rebooted that particular server, and it cleared up the whole issue."

     "We saw something that indicated the system was becoming sluggish, because of the increased call load," says Maloney. "We were able to diagnose it, call the customer before the customer even knew there was a problem, advise them of the situation and tell them what we thought they needed to do about it."

     That call load was a dynamic challenge for Acadian--which serves virtually the entire southern half of Louisiana and some areas in Mississippi--throughout the Katrina ordeal. It spiked for several days before the storm, dropped during and immediately after it, then soared again in the devastation that followed.

     With a storm of such magnitude, the company's communications capability didn't escape entirely unscathed. With towers down across the region, Acadian's hardened communications center in Lafayette temporarily lost voice connectivity with providers in the New Orleans area, although mobile data links to the field (also a TriTech system) were preserved. In the parts of Mississippi it serves, satellite backup kept everyone connected.

     But the CAD system itself held up admirably, even with extra call-takers plugged in to handle the call volume.

     "I never even considered that the CAD might fail," says Zuschlag. "Once they called and told me they were monitoring the system, it made me realize how fortunate we were that we didn't have to worry about the CAD product."