Disaster Recovery Plan - Term Paper Example

DISASTER RECOVERY PLAN “Information as Power” in the 21st century has been well recognized by the Adelphi (AU) - a private university located in the state of New Jersey, USA. AU has therefore chosen to establish state of art Telecommunication and Information Systems for its administration and regular operations. Sophisticated systems carry with them, the inherent risks of malfunction or serious damage, either by chance or by intent, at times by known sources and more often from unknown sources. Minor glitches can be managed, but the need for a well thought out ‘Troubleshooting Plan’ hardly needs any over emphasis; particularly, when confronted with serious I T system problems which have the potential of bringing the entire operations to a grinding halt – a virtual disaster, which any organization can ill afford. The ultimate aim of this project is to protect the principal business functions and assets, and suggest a back up strategy to successfully bail out AU in the event of disasters. The project shall attempt to examine all the relevant issues connected with identifying all the assets of AU and the risks associated with them, together with their linkages in relation to a wide variety of likely disasters, concluding with the assembling of a disaster recovery team. This effort at compilation of a dynamic Disaster Recovery Plan is to address the pertinent issues by utilizing the famed “5 W’s & H  What, Where, Which, When, Who and How” approach, by providing convincing answers to the six core questions spread out in the six sections that follow! In an ever changing world, organizations should be wary of natural or manmade disasters that could disrupt business processes. Loss of customers apart, millions of dollars could go down the drain and never recovered if business processes are disrupted and IT systems do not recover fast enough to normalcy within the optimum response time. The Business Continuity Plan is intended to resume business processes whereas the restoration of the IT systems is by the Disaster Recovery Plan. The objective of the latter is to restore the operability of systems that support mission-critical and critical business processes to normal operation in the quickest time possible. Business continuity plan is an amalgam of the business resumption plan, incumbent emergency plan, incident management plan, continuity of operations plan, and disaster recovery plan, all rolled into one. The following treatise presents an overview of a disaster management recovery exercise. 1) The first question to be asked is: “WHAT are the principal assets of AU which need to be protected against disasters?” An asset is defined as a tangible or intangible entity that has value or utility to an organization in relation to its business operations for uninterrupted working with protection. The most crucial asset in an organization is information, and all other important assets linked to it, including people as a human resource, processes, records and facilities which are of no less value. Barring goodwill, reputation and intellectual property which are the intangible assets of the university, all other assets possibly can be categorized as tangible. The onus is therefore on the organization to devise ways and means of not only protecting their key people but also the integrity, confidentiality and continuous availability of its records, processes and facilities constituting the backbone of its I T system for its success and sustained growth. Asset identification and valuation forms part of a risk assessment exercise which logically precedes its protection. In its bid to devise a foolproof disaster recovery plan, AU has therefore categorized its principal assets as follows: Personnel: Students’ ID and academic data pertaining to faculty and other staff members including department wise directories are stored under this category for ease of access and retrieval. Operations: The core cluster of information systems aiding the principal business functions get listed under this heading. Facilities: Infrastructure items like laboratories, auditoriums, sport fields, buildings and administration facilities with dedicated support for generation of payrolls, academic and budgetary reports et al are covered under it. Hardware / Equipment: Hardware components like Servers and Routers, network devices, hard drives, desktops, laptops, modems, printers, scanners, fax and photocopying machines, projectors, telecommunication systems et al constitute some of the physical assets of AU which get listed in this category. Software: Software applications and utilities, software facilities and packages stored in CD’s, DVD’s et al are included under this head. Databases: Departmental payroll records, personnel and academic databases, data recording files, educational books, confidential documents, printed records et al come under this category. The assets identified above should be protected from unauthorized access, theft, modification, disclosure, technical failures and destruction as they have the potential to cause disruption of business activity if exposed to emergencies and disasters, not to mention terrorism and campus shoot-outs of recent origin. The mission of AU is to consistently drive business, regardless of unforeseen interruptions, natural or manmade to cause loss, disruption or damage to its information systems. It has necessitated the enunciation of an appropriate disaster recovery plan to trigger a business continuity plan to guarantee functional stability of the organization. It envisages suitable control and recovery strategies for those interruptions whose impact is assessed to be high against non-availability of one or more of these prime assets. Outlining of procedures for quick retrieval of the data and restoration of the systems is foreseen to minimize the impact and duration of an unexpected outage which can lead to blockage of access to these systems and the critical data stored therein. AU has made special efforts in collaboration with its information security personnel to chalk out the protocols to be observed for recovery of critical data during service interruptions, based upon ‘criticality information’ evaluated and furnished by the respective data owners. 2) The second question to be answered is: “WHERE” are all the risks likely to be encountered? Acceptance of the fact that business assets are vulnerable to varying degrees of risks in relation to their associated value is the first step towards effective decision making for their protection. The ability to foresee and initiate proactive measures against exposures to assets that can have a debilitating effect on an organization is called ‘risk management’. Dissemination of the information related to risks and integration into the business process for wide circulation in its affiliated colleges and divisions, ensures a general awareness about the matter for timely corrective action. In the context of AU and its assets, comprehending and isolating the identified risks for reducing or transplanting them by proper analysis and control assumes utmost importance. Sizing up of the potential threats emanating from within or from outside the organization and its preparedness in meeting them effectively is in effect, a risk analysis exercise necessary to be carried out for assessing and evaluating the adequacy of the physical and environmental security measures and controls in vogue. Without being unduly bogged down by the details of the potential nature of disasters and their after effects which an organization is likely to be confronted with, a comprehensive risk assessment encompassing a realistic threat analysis serving the complementary role of aiding the implementation of preventive measures is deemed absolutely necessary. Success of a disaster recovery plan in the AU environment is dependent on the factors elaborated hereunder: Ability to isolate every potential event that can jeopardize the infrastructure and data that an organization depends upon – commonly known security and network threats like viruses, worms, Trojans et al called malicious attacks notwithstanding, it is imperative that the geography of the organization be elaborately studied for external threats to it from nature like earthquakes, tornadoes, floods or frequent power failures caused by storms or rolling blackouts to confront them with confidence and preparedness. Advance planning to cross-train personnel in disaster recovery and business continuity – A geographically dispersed pool of cross-trained personnel in the domains of Disaster Recovery and Business Continuity available readily to combat emergency situations, would go a long way in responding favorably to large environmental disasters engulfing a large local population. Activation of the alternative mode of internal communication when the standard communication infrastructure is paralyzed – Tie-up with a third party service provider for triggering the alternative mode of communication in the event of say, a power outage for activating a core group of trained personnel to execute the disaster recovery and business continuity plan. Adequate backup power facility provision – Investment in the Uninterrupted Power Supply Source (UPS) with the longest life available would be the most prudent and sensible decision in combating widespread environmental disruptions, leading to extended periods of non-availability of power. Giving credence to the fact that loss of important data can also mean a break of business, adequate reserves of fuel to run the back-up generators should be stockpiled, to at least make the establishment run for the minimum period of a week. Ability to prioritize the sequence in which resources need to be reactivated – First things should always come first! Confidential data restoration should preferably precede the generic data, with dedicated backup systems earmarked for the former. It might be prudent to restore the email application, say, in preference to the departmental file servers. There could be some applications which could wait for a day or two without drastically affecting the business. For ensuring clarity of purpose of ‘intervention sequencing’, a company certified queue card mechanism derived after ABC analysis, is deemed mandatory to avoid confusion in the event of a disaster. ‘Ease of Access’ to physical documentation of the Disaster Recovery and Business Continuity Plan – In order that the Plan document be deemed complete, it should not only contain precise instructions about execution of the Plan detailing all the incumbent processes, but also the details of locations of the system resources needed to execute the recovery; the multiple locations where the document in the form of manuals shall be available for ready reference, with special emphasis on ease of access to the key personnel. The Plan document being available for use at the right place and at the right time to the right people in the event of emergencies is of utmost importance. Validation of the adequacy of the backup systems – An out of date or corrupted Disaster Recovery and Business Continuity Plan is as good as not having it at all; worse, if it were to also fall victim to the disaster. It is therefore mandatory that protection of information integrity is accorded top priority by subjecting the data storage system to regular backups at predefined intervals, for both onsite and offsite applications, with penal provisions if felt necessary to demonstrate rigorous compliance. Periodic testing of the Disaster Recovery and Business Continuity Plan - Analogous to a typical fire drill, an organization need to be doubly sure that the recovery plan actually works when it is intended to work. “Data fire drills” must be conducted at predefined regular intervals to test the Plan in real time situations ranging from basic power outages to major catastrophic events which could have the potential of months of agonizing wait. Apt Password Management – It is always sensible to store the system passwords at more than one geographically separated location with a core group of key professionals having access to the passwords and the codes, in tune with the current norms of data security with password protection. Key personnel quitting the organization should automatically prompt password and code changes. The Disaster Recovery and Business Continuity Plan as a ‘dynamic’ document- An internal organizational protocol should be worked out which establishes and documents a list of trigger points that should invoke changes to the Plan and its constituents inclusive of personnel, equipment, location and application changes et al to render it ‘dynamic’. Periodic testing, evaluating, refining and regularly updating the Plan to keep it on course with the organization’s stated goals and objectives, should always be an in-house and on-going exercise in consonance with changing times. Revisiting the Plan once every quarter should be a must. AU functions from its two geographic locations, viz. Teaneck and Madison and the risks of data collection, data inconsistency, categorization, manageability and maintenance are common to both. Consolidation and integration of information poses problems, particularly for sourcing external data from remote file storage locations because of the data being dispersed. The process of normalization of the data involving compliance to predefined rules and categories to maintain consistency further becomes cumbersome. In order to ensure accuracy and consistency of the data, periodic cross-check at every step becomes unavoidable. Risk analysis need be done, keeping in view the AU ground realities of the various facets of its working, encompassing its personnel, student services, faculty services, knowledge assets, data, equipment, communication networks, buildings and finance, to just name a few critical constituents, which have a bearing on its overall functionality. In the above scenario, AU’s Business Continuity Planning strategy, implementation and maintenance needs are to be assessed holistically - all possibilities of functional paralysis in the event of exposure to a variety of known and unknown risks should be examined. Business Continuity Plan constituents have been detailed below for each AU administered college and division, based on the above facts and criteria: Prescribing maximum permissible IT & Communications’ System Downtime Threats’ Identification having implications of loss of goodwill and reputation Program or Data Failure Hardware Failure Electrical Power Failure leading to data loss Fire Contingency Plans’ formulation for restoration of services within the prescribed time, paying heed to the risk management and contingency planning recommendations given by the Security and Disaster Recovery group. 3) The third question to be addressed is: “WHICH” are the assets threatened by risks? The mission of AU for delivery of essential business functions is critical. It stands committed to ensure the continuity of operation of essential business functions with its supporting electronic information systems despite damage, loss or disruption due to an emergency or disaster. Notwithstanding the fact that a wide variety of incidents can render these systems and data inaccessible, the need for procedures for quick recovery of the data and restoration of the systems to minimize the impact and duration of the unforeseen outage is unequivocal. Personnel, Data, Hardware, Software, Facilities and Operations are the some of the principal assets of AU. The likely impact on the University’s functional operations has been assessed based upon the non-availability of the essential system driven services. AU in consultation with the principal data owners and its information security personnel after an elaborate evaluation has firmed up the ‘criticality’ aspect of its data and information resources. A 3-tiered approach has been followed for preservation of AU assets by classification according to their assessed risks, with special focus on the potential impact of an outage on its operations as detailed below: Tier 1: Critical assets which are vital for the daily operations of the University are categorized under the first tier. Most critical among the University assets comprising of personnel, hardware, software, and databases which can cause disruption of the academic programs of the University find mention here. Tier 2: Very important assets which permit flexibility to the University to keep its functions operational despite the non-availability of the others are included in this category. Piecemeal job works like financial services and departmental payroll services, printing and data recording files, software packages, educational books, sundry files et al constitute this tier. Reinstatement of these services in the subsequent increment time period immediately following a disruption ensures continuity. Tier 3: Important assets which provide the leeway necessary for the University to maintain skeleton services for an extended period of time without the other assets and afford the luxury of a waiting period of up to a week for recovery are included in this last tier. All such actual physical services which are not required immediately following a disruption due to a disaster like printers, scanners, projectors, fax machines, modems, hard drives et al get featured in this category. A ‘Risk Severity’ matrix shown below highlights the AU evaluation criteria for each of the three tiers vis-à-vis the services on offer, in relation to their functionalities: Loss or damage to systems of any of the three tiers entails the following risks on AU’s unhindered operations: Adverse effect on individuals, and AU’s assets and operations Adverse effect on research, and thereby on the institution Financial implications to the institution Legal hassles for the University Contractual obligations for the University Loss of strategic advantage to the University A genuine appreciation of the asset-risk linkage is thus crucial for assuring the longevity and protection of all the AU assets. 4) The next question to be asked is: “WHY” do you need to do what you are doing – viz. identifying and prioritizing business functions? AU is functioning in a ‘campus business’ environment utilizing standard business continuity principles and processes to fulfill its primary mission of facilitating research, teaching and public services involving students, staff and other support personnel in the domain of education; the other functional components of campus operations being its infrastructural assets like buildings, water, power, sewer, communication systems, access and IT framework et al to name a few. In this unambiguous endeavor, its principal goal is to reduce risk and minimize disruption of campus research and academic programs. Efforts made at identifying specific risks to ensure the campus continuity planning to be both manageable and meaningful thus make enormous sense. AU advocating ‘Business Impact Analysis’ is aimed at identification and evaluation of all the vital business functions and systems with a view to assess the impact of loss or disruption of every function. It thus becomes an effective tool for categorization according to the time frames required for recovery of each function. Its purpose is also to quantify the consequences of disruption in terms of financial loss, additional expenses, and political fallout, and to identify other intangible factors. The maximum duration for which an organization can withstand disruption indicating its ‘endurance capability’ for driving the recovery priorities for business systems and functions is also the parameter under evaluation by the above analysis. Prioritization of mission-critical and administrative functions for business resumption entails categorization of the various IT, data processing and financial systems according to their degree of importance and the pre-designated order of restoration. The University business function has chosen to categorize the various systems as critical, vital, sensitive or non-critical, depending on the sequence and time frames of restoration of data, applications and systems, given the fact that some systems have a high cost of interruption implying a low business tolerance for their interruption during disasters. Health, safety, general voice and data communication facilities and emergency situations believed to cause destruction to life and property are identified as critical top priority functions. Capable of being restored as soon as possible not exceeding 24 hours, these critical and important functions are the lifeline of the business for it to survive. A tabular representation appears as shown below: Sl. No Function  Critical / Priority 1 Moderate / Priority 2 General Data Processing & Communication Systems’ functions University Business & Academic functions Miscellaneous 1 On campus network services Personnel Services Operating System Reception at every Department 2 On campus voice communication Financial Records System Information Desk 3 Off campus voice communication Payroll System Admission Desk 4 Special circuits (911; Public Safety – 2222; Tech Support – 8282 etc.) Student Information System Staff Time Reporting System 5 Voicemail including information & broadcast mailboxes Student and Advisor Link Financial / Budget History 6 AU webpage - www.AU.edu Employee Link - 7 Email and server systems Student & Extended Visitor Information System (SEVIS) - 8 Network Authentication Service (NETID) University Information System - Critical enough for the organization to be run profitably and in a successful manner and restorable within a week, the University Business and Academic functions are assigned the moderate, second level priority. Primarily conceptualized as a system to support the priority systems 1 and 2, the last ‘General’ category encompasses the other miscellaneous functions of the University, the prolonged absence of which are not likely to hamper its normal operations with a restoration period pegged at 30 days. It does make business sense to do what has been contemplated above. 5) The penultimate question to be asked is: “HOW is the backup strategy going to be developed and implemented”? In addition to maintenance and upkeep of multiple generations of operating systems and applications for all data storage facilities, data backup with good security features is an absolute must for disaster recovery. Considering the higher cost of re-creating lost data, how, when and how frequently data should be backed up will form the basis of any investment. Among a host of options, a remote data backup facility for off-site storage on tape or disk seems viable, given the AU constraints of geography, infrastructure and budget, and the degree of protection needed in relation to the prioritized resources required to enable perform critical functions. A secure, climate controlled, fireproof media vault at a storage facility under the control of a commercial media storage service provider might be another viable option. Designed to transmit the on-site backup data overnight or at prescheduled times, the predefined off-site copy authentication procedure should be in conformity with all AU outsourced elements of the continuity solution. Transmission, storage and maintenance of confidential and critical data should take place with an appropriate level of security. ‘Mirroring’ appears to be the most suited technique for data redundancy in preference over ‘striping’ and ‘parity’, because of the low fault tolerance capability of the latter, although faster backing up to disks is in effect, not really data mirroring. Data written simultaneously to separate hard drives or drive arrays can preferably be stored in CD’s, DVD’s, disks or tapes depending upon criticality, unlike in the ‘parity’ technique, wherein overwritten or lost data gets revealed without the need for storage. Conventionally, data written to disk using backup software must be written back to a host in its native format before it can be accessed again. Despite the disadvantage of both drives and disk arrays processing in the write-to-disk function mode hindering system performance, mirroring presents the option of making the system operate from the working hard drive or disk array, if one hard drive array fails; it also permits one disk to process a read request and the other for different processing request. Its distinct advantages are simple data recovery, minimal downtime and increased performance in reading the disk. Every single function in AU being dependent on their data storage in one way or the other, it becomes imperative that media is retrieved on a regular basis from the off-site storage and tested to verify proper functioning of the backups, apart from the primary responsibility of backing up computer systems and application software for recovery in the event of a disruption. Whether the teaching functions craving for course management systems to provide grade books, teaching materials and portfolios or the researchers seeking domain specific information from the data stored, services should always be available at their beck and call. A backup protocol meeting the following criteria has been suggested, with the premise that AU critical data would be backed-up at least once every working day at prefixed timings: Enable recovery to at least the start of business on any week day of a failure. Provide at least one more level of backup to cover the case of the failure of the primary backup media. Ensure off-site storage of backup media to enable a full data recovery, not exceeding the preceding week. Conduct half yearly audit of backup media without fail. The AU regimen suggested for performing backup of their generic data with a built-in stagger vis-à-vis the critical data driven by its security policy, is detailed below: Daily backup of deemed important data during off-peak hours in the night. Static data backup at less frequent intervals. Surprise checks to monitor compliance. Depending on the assessed overall volume and quantum of data storage, real time replication might be a better solution in preference to tape backup, saddled with its inherent insufficiencies. Replication solutions vary in cost and complexity and allow for immediate system and data availability for recovery and user access with practically near zero data loss. Synchronous and asynchronous replication is feasible with host-based or hardware-based options. Excellent disaster recovery and high availability in a heterogeneous, applications-integrated storage environment is assured with the most flexible and cost effective host-based replication option. An ideal scenario would be that of a judicious mix of backup, recovery and tape management tools in conjunction with data replication and mirroring tools and their attendant benefits aiding automation tools to provide the capability of achieving security at both the system and applications level. 6) The final question to be asked is: “WHO are all the key people who will play the pivotal roles in ensuring recovery from disasters? Disasters are an integral part of life. Prudence lies in combating them with fortitude. The human element is the key determining factor in recovery from such disasters. AU being a huge organization with several departments engaged in imparting the best quality education to its students, it has realized the importance of not only putting a disaster recovery team into place but also spelt out their specific roles for implementation to clockwork precision. Special efforts have gone into the selection of the right persons for the right slots to make the task of recovery easy and foolproof. It has been ensured that all the team members have the thorough knowledge about the business, technology, security and the other basic requirements to resume the business after a disaster, regardless of their presence at the time of implementation of the disaster recovery plan. AU has strategically chosen to disperse these key personnel across its establishment to help catalyze the other people into action for activation of different plans specific to each department, to enforce recovery efforts according to guidelines laid down by the university. The details of the team composition are described below: 1. Risk Management Team: It is a core management team intended to administer and monitor the activities immediately following a disaster and provide the necessary guidance for recovery planning, execution, sustenance and suggesting improvements for future development and maintenance, based on experience gained. A Disaster Recovery Coordinator directs the decentralized recovery teams and their subgroups in accomplishing the tasks expected of them in reporting relationship with the Executive Director. 2. Executive Director: Vested with the discretionary powers to provide both vertical and lateral support, advice and counsel to recovery teams by keeping communication channels open, the Executive Director is the lead person to provide assistance to the campus crisis management team in handling all information security and confidentiality issues with utmost caution in accordance with university and federal rules and procedures. He is the contact person for all aspects relating to insurance, payroll, external auditor liaison, buildings and other fixed assets of the university in the event of a disaster. 3. Campus Crisis Management Team (CCMT): CCMT is an administrative decision making group constituted for responding promptly to campus wide emergencies categorized as critical incidents that threaten life or property, or those which impact a sizeable population of the university community, including and not limited to major fires, civil disturbances and natural disasters. Enhancing operational response capabilities, facilitating flow of communication and quick decision making are its principal functions. 4. Facilities Management Team: All aspects of the AU campus facilities including and not limited to power and water systems, buildings and space needs, cabling and instrumentation et al, which have bearing on their present or future performance at all levels of disaster recovery planning come under its ambit. 5. Communications Team: The AU Communications team deemed to be the PR wing of the IT Systems infrastructure, is the vital nerve center of activity for inter and intra-communication within the campus community. It handles issues related to mobilization of team members in case of emergencies and organizing group discussions for brain storming in seeking viable quick fix solutions. In normal times, it is responsible for notifying staff and other affiliated groups regarding problems encountered, solutions administered, and reports generated apart from the routine function of releasing communiqués and raising alerts within the team members, whenever needed. 6. Administrative Team: Conceptualized as a back end support function, its team members provide logistical business and financial support services for matters pertaining to acquisition of inventories and components required for backup framework, budgets and aiding departments in developing their recovery plans. 7. Recovery Coordinator: A designated team leader for corporate-wide Disaster Recovery Planning (DRP), risk assessment and strategy development for all time readiness, Recovery Coordinator is the key person responsible for training and supervising the junior members of the DRP staff. As a resource person expected to interact with technical experts, vendors and government functionaries amongst several others, his principal responsibility is to design, develop, integrate and test the efficacy of DRP procedure of the organization on real time basis for recommending appropriate recovery options. However grandiose the Disaster Recovery Plan, it is eventually the key trained personnel of AU who ‘man’ and maintain it that matter to, MAKE IT HAPPEN – hence, the relevance of the concluding question; and, the answer addressing the core theme of this project! Works Cited 1) CISCO, White Paper on Disaster Recovery: Best Practices, 2008, Online, http://www.cisco.com/en/US/technologies/collateral/tk869/tk769/white_paper_c11-453495.pdf (Accessed December 3, 2011) 2) Bahan, Chad, The SANS Disaster Recovery Plan, 2003, Online, http://www.sans.org/reading_room/whitepapers/recovery/disaster-recovery-plan_1164 (Accessed December 3, 2011) Read More