Meeting Date:†††††††††† February 19, 2004††††††††††††††††††††††††††††† Budgeted:No

Program Line Item No.:Contingency

Staff Person:††††††††††††† Rick Dickhaut††††††††††††††††††††††††††††††††††† Cost Estimate: $ 12,514

††††††††††††††††††††††††††††††††††† Inder M Osahan

General Counsel Approval: N/A

Committee Recommendation: The Administrative Committee reviewed this item on February 10, 2004 and recommended approval.

CEQA Compliance: N/A



SUMMARY:The District suffered a hardware failure on January 26, 2004. Staff worked with tools installed and available on-site, along with equipment manufacturers, to restore the data files stored on the system but was unable to do so.Due to the severity of the failure, data stored on the damaged hard disk drives was recovered by retaining services of Ontrack Data Recovery, Inc.Ontrack recovered all of the lost data and staff has been able to conduct business as usual since that time.Utilizing the authority granted in Resolution No. 2002-08, which increases the General Managerís authority to approve contracts from $2,000 to $20,000 in emergency situations, the General Manager approved the expenditure of $12,514 for this purpose.


RECOMMENDATION:Staff recommends Board ratification of the expenditure of contingency funds in the amount of $12,514 to Ontrack Data Recovery, Inc.The Administrative Committee considered this matter at their February 10, 2004 meeting and voted 3 to 0 to recommend approval.The Committee also recommended that staff attempt to recoup the cost from the manufacturer of the failed equipment.


BACKGROUND:On Monday, January 26, 2004, the Districtís electronic file storage server suffered a severe hardware failure.Files stored on the system were inaccessible for the day while staff worked with the equipment manufacturer to try to restore the system.†† In addition to the hard disk failure, the current backup system for the District had also failed due to faulty hardware that went undetected by the backup system.The data restore process was unsuccessful and the District retained Ontrack Data Recovery, Inc. to help restore the data.The data recovered by Ontrack has been restored to the production servers and is available to staff to conduct regular business.The data recovery was 100 percent successful and has saved considerable staff time and resources.Information gathered from eight staff members indicated that over $30,000 in staff time would have had to be expended just to recreate their own data.It is estimated that the cost of time expended by the entire staff would have totaled almost $100,000.


During the time the regular server was unavailable, staff deployed a standby server to conduct business as usual while the data was being recovered from the damaged hard disk drives.To avoid having a reoccurrence of a disaster of this magnitude, staff has implemented redundancy not only for the file server, but also for system monitoring and backups.