Help! We'Ve Lost Our Hard Drive!

HELP! WE'VE LOST OUR HARD DRIVE!

by Jeane Massey

May 11^th was a typical busy Monday in our office. Our 20 staff members were using our system constantly throughout the morning; at noon we did our normal mid-day floppy disk backup of the Pioneer III accounting files. (I always breathe a sigh of relief after completing a backup because I know that, as of that moment, our work is safe.)

As our staff members returned from lunch, they once again put the system to extensive use. The day ended at 5:30 p.m., and I entered the command for the streaming tape device to back up the entire system at 11 p.m. that night. This is the second part of our normal backup procedure-floppy disk backup of accounting files at mid day and full system backup at night, when the system is not in use.

When we came to work on Tuesday morning, May 12, the first users trying to log on to the system were unable to do so. I immediately checked my workstation, where our tape backup software is installed, to discover- much to my dismay-that the backup scheduled to begin at 11 p.m. the night before had not taken place. When the backup program attempted to access the system, no system was available. The message on my screen screamed: 'HARD DRIVE FAILURE!'

We spent most of the morning talking with (or waiting to talk with!) the customer service folks at Agency Management Services (AMS). We tried all of their suggestions, but nothing worked: We could not resurrect the hard drive. Finally, we called our local computer- maintenance firm. They confirmed that the hard drive had failed and would have to be replaced. We will always wonder but probably never know what might have caused this failure. The message that appeared on the file-server monitor was not noted or recorded, because someone other than our system coordinator tried to reboot the system, losing the original failure message.

I knew we would be inconvenienced by this failure, but I was confident that we would be back up and running without a long delay. After all, we had a current backup. Doing a daily backup is somewhat like purchasing a snake-bite kit: You sincerely hope you will never have to use it; but in case of need, you're more than happy to have it. This was true with us. However, as careful as we had always been about daily backups, we discovered that we were not prepared for what needed to be done to restore our system completely.

A new hard drive must be configured the same as the hard drive it replaces, so that all programs you restore will operate as they were originally set up. Do you know how your agency's hard drive and network program are set up? I did not. As a result, it took many phone calls and a great deal of time working with the original installer to determine how the new drive should be configured.

Once the hard drive was finally up and running, we had to load the tape-backup software so we could restore the last backup tape. Are you familiar with your backup software? Do you know how to install and configure this program for your particular situation? I did not. We spent about two hours figuring out what we needed to do. This lack of 'set-up' knowledge contributed to our being down for two and one-half days.

When we were finally back up and running, we knew we had lost some of our work. I have always been quite concerned about the two daily four-hour periods before the next backup is run. If our hard disk had failed in the early morning hours after the late-night backup, we would not have lost anything. However this failure occurred after the mid day floppy disk backup, but before the late-night full system backup. So we had a half day's work in jeopardy.

At the close of the business day, we always print a daily transaction log and take it off premises, together with the latest tape backup. However, the transaction log shows only accounting transactions. So when our system died, we lost all non-accounting transactions made after the last full-system backup because we had no record of these entries. These transactions included name and address additions or changes (including profiling), daily and claims entries, company and producer additions or corrections, rating proposals, and word processing. Using the transaction log, we re-entered the missing accounting transactions.

All of us, however, continue to worry about what important transactions we may have entered between backups (especially in diary) that are lost. This worry has prompted us to begin a procedure that will help if we should ever lose our system again. Each staff member is encouraged to keep a log showing the account number of all files worked between backups. In this way, if we should face the loss of our system again, we will know what files were worked. We can then recreate all transactions-accounting and nonaccounting-that occurred between backups.

We use our system to the fullest, utilizing all options included with our accounting program. And we have Personal Lines and Commercial Lines rating programs loaded, as well as word processing, a proposal program, and a spreadsheet program. This means we have many nonaccounting transactions stored on our file server.

We now know it is vital to know what computer-stored files may have been added or changed since the last backup. We like to think the 'be prepared' lesson we learned after losing our hard drive will be helpful to others who face the same possibility. We encourage every agency to take the following precautions:

In spite of the confusion and frustration we experienced when our hard drive died, we felt we were very lucky. We had good, solid backup procedures in place, and they worked. However, if this ever happens to us again, we are now much better prepared. And this preparation will reduce our down time. We encourage you to learn from our experience. Who knows, you may one day soon have to put your snake-bite kit to use.