FEMA Needs to Improve Management of Its Flood Mapping Programs OFFICE OF INSPECTOR GENERAL September 27, 2017 OIG-17-110 DHS OIG HIGHLIGHTS FEMA Needs to Improve Management of Its Flood Mapping Programs September 27, 2017 Why We Did This Audit Flood hazard identification and mapping is an integral part of the National Flood Insurance Program (NFIP) as it creates the foundation for floodplain management, flood insurance, and mitigation. We sought to determine whether the Federal Emergency Management Agency’s (FEMA) Risk Mapping, Assessment and Planning Program (Risk MAP) resulted in the production of timely and accurate flood maps in accordance with NFIP requirements. What We Recommend We made four recommendations designed to improve FEMA’s management and oversight of its flood mapping programs. For Further Information: Contact our Office of Public Affairs at (202) 254-4100, or email us at DHS-OIG.OfficePublicAffairs@oig.dhs.gov www.oig.dhs.gov What We Found FEMA is unable to assess flood hazard miles to meet its program goal and is not ensuring mapping partner quality reviews are completed in accordance with applicable guidance. FEMA needs to improve its management and oversight of flood mapping projects to achieve or reassess its program goals and ensure the production of accurate and timely flood maps. Specifically, FEMA – x needs to improve its financial management of flood map projects to achieve or to reassess its program goal of 80 percent New, Valid, or Updated Engineering program miles; x has not updated its Risk MAP life cycle cost estimate to inform critical decision making; x lacks uniform, centralized policies and procedures for projects placed on hold; and x is not performing adequate oversight to ensure mapping partner quality reviews comply with requirements set forth in applicable guidance. Without accurate floodplain identification and mapping processes, management, and oversight, FEMA cannot provide members of the public with a reliable rendering of their true flood vulnerability or ensure that NFIP rates reflect the real risk of flooding. FEMA’s Response FEMA noted that it has made significant progress developing and implementing several flood-related policy and program changes since the Office of Inspector General began this audit. FEMA remains strongly committed to providing credible risk information to communities so that they can make informed flood risk management decisions. The report contained four recommendations with which FEMA concurs. OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Washington, DC 20528 / www.oig.dhs.gov September 27, 2017 MEMORANDUM FOR: Angela Gladwell Risk Management Deputy Assistant Administrator Federal Insurance and Mitigation Administration Federal Emergency Management Agency FROM: John V. Kelly _..., Deputy Inspector General SUBJECT: FEMA Needs to Improve Management of its Flood Mapping Programs Attached for your action is our final report, FEMA Needs to Improve Management of its Flood Mapping Programs. We incorporated the formal comments provided by your office. The report contains four recommendations aimed at improving FEMA's management and oversight of its flood mapping programs. Your office concurred with all four recommendations. Based on the information provided in your response to the draft report, we consider recommendations 1 and 2 open and resolved. Once your office has fully implemented the recommendations, please submit a formal closeout letter to us within 30 days so that we may close the recommendations. The memorandum should be accompanied by evidence of completion of agreed upon corrective actions and the disposition of monetary amounts. We consider recommendations 3 and 4 open and unresolved. As prescribed by the Department of Homeland Security Directive 077-01, Follow-Up and Resolutions for the Office ofInspector General Report Recommendations, within 90 days of the date of this memorandum, please provide our office with a written response that includes your target completion date for each recommendation. Please send your response or closure request to OIGAudit sFollowup@oig.dhs.gov. Consistent with our responsibility under the Inspector General Act, we will provide copies of our report to appropriate congressional committees with oversight and appropriation responsibility over the Department of Homeland Security. We will post the report on our website for public dissemination. Please call me with any questions, or your staff may contact Maureen Duddy, Deputy Assistant Inspector General for Audits, at (617) 565-8723. Attachment OFFICE OF INSPECTOR GENERAL Department of Homeland Security Table of Contents Background .................................................................................................... 1 Results of Audit .............................................................................................. 3 Management of Flood Mapping Projects Needs Improvement .................. 3 FEMA Should Enhance Its Oversight of Mapping Partner Quality Reviews .......................................................... 7 Conclusion .......................................................................................... 10 Recommendations ......................................................................................... 11 Appendixes Appendix A: Objective, Scope, and Methodology ................................. 14 Appendix B: FEMA Comments to the Draft Report ............................... 17 Appendix C: Report Distribution .......................................................... 21 Abbreviations ECD FEMA FIRM or flood map LCCE MIP Needs Database NFIP NVUE OIG QR Risk MAP RMD verification check www.oig.dhs.gov estimated completion date Federal Emergency Management Agency Flood Insurance Rate Map life cycle cost estimate Mapping Information Platform Coordinated Needs Management Strategy Database National Flood Insurance Program New, Valid, or Updated Engineering Office of Inspector General quality review Risk Mapping, Assessment and Planning Program Risk Management Directorate Database Verification Tool OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Background Flooding has been, and continues to be, a serious risk in the United States. To reduce this risk, Congress established the National Flood Insurance Program (NFIP) with the passage of the National Flood Insurance Act of 1968. Although originally intended to be funded with policyholder premiums, NFIP, in part due to natural disasters in 2005 and 2012, has proven to be actuarially unsound. For example, as of March 2016, the Federal Emergency Management Agency (FEMA) owed the U.S. Treasury $23 billion, up from $20 billion in November 2012, to pay flood claims. FEMA administers NFIP through the Federal Insurance and Mitigation Administration. NFIP’s three-fold mission is to – x x x make federally backed flood insurance available to home and business owners and renters; minimize the economic impact of flood events through floodplain management, which incorporates mitigation efforts and communityadopted floodplain ordinances; and identify and map community areas subject to flooding. Flood hazard identification and mapping is an integral part of NFIP as it creates the foundation for floodplain management, flood insurance, and mitigation. A Flood Insurance Rate Map (FIRM or flood map) includes statistical information such as data for river flow, storm tides, hydrologic and hydraulic analyses, and rainfall and topographic surveys. Flood maps provide risk information and enable communities to make informed decisions, e.g., to adopt and enforce minimum floodplain management regulations that reduce the loss of life and property. Flood studies are conducted to assess the following conditions: x x x x riverine flooding of rivers, streams, or other waterways; flooding of lakes and ponds; coastal flooding caused by hurricanes or severe storms; and shallow flooding, ponding, and sheet flow. FEMA manages its mapping needs through the Coordinated Needs Management Strategy Database (Needs Database), a resource that includes processes and data for tracking flood hazard studies. FEMA uses the Needs Database assessment process to report the percentage of updated and valid flood maps. FEMA expresses this metric as the New, Valid, or Updated Engineering (NVUE) percent attained. www.oig.dhs.gov 1 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security The NVUE percent attained is a ratio of all NVUE study miles divided by the total miles in FEMA’s mapped inventory. NVUE metrics distinguish between engineering studies that adequately identify the level of flood risk (known as Valid) from those that are in need of restudy (known as Unverified). Changes in topography, hydrology, and/or land development are evaluated as part of this validation process. When a floodplain needs to be studied or is being studied, it is labeled as Unverified in the Need Database. Unverified miles can only become NVUE compliant through a new flood study. According to the National Flood Insurance Reform Act of 1994, FEMA must assess the need to revise and update all floodplain areas and flood risk zones identified once during each 5-year period. Thus, valid miles will expire every five years if not assessed. Failure to assess an NVUE compliant mile within the 5-year window will result in the mile being re-categorized as “Unknown” in the Needs Database. Unknown miles have not been subjected to the validation process to determine whether they reflect the current flood risk or are in need of restudy. In 2009, FEMA set a goal to attain 80 percent NVUE by the end of fiscal year 2014. FEMA Headquarters and regions select flood mapping projects based on the highest risk, need, and available funding. FEMA relies on mapping partners such as production and technical services contractors, which are private engineering firms, as well as state and local governments or regional agencies participating in FEMA’s Cooperating Technical Partners program for riverine, levee, and coastal flood studies. FEMA uses the Mapping Information Platform (MIP) as the system of record to manage the production of flood maps. MIP integrates program and project management, data storage and retrieval, standardized quality control reviews, and tracking of flood map production into one web-based application. This review focuses on riverine flooding, which occurs in waterways subject to overbank flooding, flash floods, and urban drainage system flooding. www.oig.dhs.gov 2 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Results of Audit FEMA is unable to assess flood hazard miles to meet its program goal and is not ensuring mapping partner quality reviews are completed in accordance with applicable guidance. FEMA needs to improve its management and oversight of flood mapping projects to achieve or reassess its program goals and ensure the production of timely and accurate flood maps. Specifically, FEMA – x needs to improve its financial management of flood map projects to achieve or to reassess its program goal of 80 percent NVUE program miles; x has not updated its Risk Mapping, Assessment and Planning Program (Risk MAP) life cycle cost estimate (LCCE) to inform critical decision making; x lacks uniform, centralized policies and procedures for projects placed on hold; and x lacks oversight to ensure mapping partner quality reviews (QR) comply with requirements set forth in FEMA’s quality review guidance. Without accurate floodplain identification and mapping processes, management, and oversight, FEMA cannot provide members of the public with a reliable rendering of their true flood vulnerability or ensure that NFIP rates reflect the real risk of flooding. Management of Flood Mapping Projects Needs Improvement FEMA is not meeting its program goal to ensure flood hazards are current with NVUE miles. In 2009, FEMA set a performance goal to attain 80 percent NVUE by the end of FY 2014. However, in February 2014, FEMA notified the Department of its anticipated breach of that goal and set an approved revised baseline of 64 percent NVUE. Despite adjusting its performance goal, FEMA only attained 49 percent NVUE at the end of FY 2014 (see figure 1). As of December 2016, FEMA’s NVUE was at 42 percent, meaning more than half of the Needs Database inventory of miles either required a re-study or have not yet been assessed through the validation process. www.oig.dhs.gov 3 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Figure 1: National NVUE Attained through FY17 Quarter 1 % NVUE Attained 60.0% 40.0% 20.0% 32.5% 49.3% 45.0% 45.7% 46.7% 48.0% 49.2% 42.0% 40.9% 42.4% 42.4% 40.2% 41.8% 0.0% Source: FEMA This was caused in part because FEMA (1) does not have an appropriate system for tracking obligations and expenditures for regions at the task order level; (2) has not updated its Risk MAP life cycle cost estimate to inform critical decision making; and (3) did not always effectively oversee the progress and completion of flood map projects. FEMA needs to improve its management of flood map projects to achieve or to reassess its 80 percent NVUE program miles goal. Financial Management Oversight to Achieve Program Goal FEMA needs to improve its financial management of flood map projects to achieve or to reassess its program goal of 80 percent NVUE. FEMA attributed the reduction in NVUE compliance to a decrease in program funding. FEMA’s funding for FYs 2010 to 2014 was expected to total $1.47 billion, but FEMA received just $1.23 billion during this period. We identified $194 million unexpended funds from FYs 2010 to 2014 that could have been used for additional projects. FEMA officials stated that they were unable to spend the $194 million due to mapping projects lasting long periods or projects that were not closed. Although we identified two regions with an in-house mechanism to track and trace commitments, we were unable to reconcile these unexpended funds to mapping studies because FEMA does not have a financial system that enables them to track obligations and expenditures for regions at the task order level. Tracking obligations and expenditures would enable FEMA to better manage its resources and give them a means for monitoring and assessing the validity of its program goal. The Risk MAP Life Cycle Cost Estimate FEMA has not updated its Risk MAP life cycle cost estimate (LCCE) to inform critical decision making. FEMA created the original Risk MAP LCCE in 2010 and updated it in 2012 to project costs through 2014. Following NVUE performance goal breach and the award of two major production and technical www.oig.dhs.gov 4 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security service contracts in 2014, FEMA drafted an LCCE to estimate the total cost of Risk MAP over the life of the program (FY2010 through FY2020). This document was in draft form at the conclusion of our review. According to one FEMA Program Specialist, the LCCE remains in draft because of their impending new governance structure and uncertainty regarding future congressional appropriations. We believe that finalizing the Risk MAP LCCE would help FEMA estimate the likely cost of flood hazard studies and contribute to establishing a baseline for NVUE compliance over the next 5 years. Projects Placed on Hold FEMA did not always effectively oversee the progress and completion of flood map projects. A FEMA region may place a flood map project on hold to temporarily suspend its assessment of schedule and cost performance (earned value calculation) when it cannot progress due to uncontrollable circumstances. Schedule and cost performance is calculated by earned value at the project level. Earned value calculations of time, budget, and performance better enable FEMA to manage and assess the health of a flood map project. FEMA may place a project on hold pending the receipt of data from an external source or when progress cannot be made until critical information such as new levee guidance has been received. Although the intent of placing projects on hold is supposed to be temporary to suspend earned value calculations, FEMA lacks centralized policies and procedures for expediting removal of projects from on hold status. Specifically, 65 of the 88 projects we tested were on hold for more than a year. When flood map projects are placed on hold for long periods, FEMA cannot provide members of the public with a timely understanding of their true flood vulnerability or ensure that NFIP rates reflect the real risk of flooding. We identified 240 of 1,947 (12 percent) active flood map projects that were on hold during of our review. We focused our review on 88 of the 240 on hold projects that were riverine flood map projects. Of the 88 riverine projects tested: x 28 (or 32 percent) were on hold for an average of 2.9 years because they were impacted by new levee accreditation guidance; x 21 (or 24 percent) were on hold for an average of 2.6 years because their period of performance for the mapping contract expired. These projects could remain on hold indefinitely as FEMA may or may not fund to continue the project in the subsequent years; and www.oig.dhs.gov 5 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security x 13 (or 15 percent) were on hold for an average of 2.9 years because the project was in the post or revised preliminary mapping process. When a project needs significant modification, it may be necessary to issue a revised set of preliminary maps. The revised preliminary process usually requires a contract modification in order to change the project scope, period of performance, and funding. See figure 2 for the range of reasons FEMA placed 88 projects on hold. Figure 2: Duration and Reason Risk MAP Projects Placed on Hold for Regions Combined Duration in Years (Average) Reason Levee Analysis and Mapping Procedures Expired Period of Performance Revised Preliminary Phase Completed; Future Work Anticipated Awaiting Completion of Countywide Project No Additional Funding Awaiting LiDAR Data Post Preliminary Phase Change Request Other* 1 # 28 21 11 9 3 3 2 2 1 8 88 2 3 4 2.9 2.6 2.8 1.1 1.9 2.9 0.8 3.4 1.3 2.4 Source: Office of Inspector General (OIG) analysis of FEMA data *Other: Projects that have been combined with other flood mapping projects or are placeholders in the MIP. The Regional Project Management Lead or the Regional Project Manager is responsible for tracking the status of projects and placing them on hold with FEMA Headquarters approval. FEMA region officials reported that they would like more input or guidance from Headquarters on how they should complete or prioritize on hold projects. For example, while one FEMA region prioritizes the funding of on hold projects during their annual planning phase, another region pools projects regardless of their status. Also, while one FEMA region plans and sets aside funding to cover expenses associated with community appeals, another does not. Because FEMA allows its 10 regions to independently manage and oversee their flood mapping projects, at a minimum, FEMA should set a standard or benchmark for the completion of on hold projects and improve communication and coordination among Regional Project Managers to develop and share best practices. www.oig.dhs.gov 6 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security FEMA Should Improve Its Oversight of Mapping Partner Quality Reviews FEMA lacks oversight to ensure that mapping partner QRs comply with applicable guidance. Specifically, FEMA did not ensure that mapping partners completed and uploaded required checklists and self-certifications into MIP. FEMA also did not always verify that flood map project data passed all required Database Verification Tool checks (verification check) prior to issuing a map or ensure that documentation was kept on file to support bypassing failed verification checks. In 2013, FEMA revised its QR requirements that mapping partners must comply with during the course of developing flood maps. FEMA’s guidance requires that mapping partners use FEMA’s three mandatory standardized checklists during the course of developing a flood map. These checklists document whether project data complies with FEMA’s standards. Once the mapping partner makes any necessary corrections and completes the checklist, the mapping partner uploads them into MIP and self-certifies the flood map’s compliance with FEMA’s standards. Additionally, data used to develop a flood map must pass automated verification checks in order for the project to advance to the next step in the development of the flood map. Verification checks determine whether project data used in developing a flood map complies with FEMA’s Flood Map Database standards. Mapping partners upload data into MIP and use the verification checks to test the quality of draft, preliminary, and final flood map database submittals. If the MIP verification check fails, the data is returned to the mapping partner for correction. In some instances, the mapping partner can request a verification check bypass and explain the reason. All quality compliance check issues noted during the QR process must be fully addressed, documented, resolved, and included in MIP. However, FEMA’s guidance does not assign responsibilities to FEMA personnel to ensure that mapping partners – x x x complete and upload the required checklists and self-certifications into MIP; validate that flood map project data passes all verification check prior to issuance of a flood map; and upload documentation of verification check bypasses in the MIP. FEMA regions reported a general lack of understanding as to who should oversee QRs performed by mapping partners. According to one FEMA Program Specialist, FEMA has not provided any training to Regional Program Managers www.oig.dhs.gov 7 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security regarding the QR process. Additionally, this specialist was unaware of any FEMA region that confirms the uploading of required checklists into MIP. We held discussions with five Regional Program Managers and they offered the following feedback about oversight uncertainty. Different managers indicated that they – x x x x x x x did not know who was responsible for overseeing the quality review process; were not required to upload quality review checklists to the MIP prior to 2015; did not have access to all MIP files; believed the QR process was not mandatory; stated the regions and Headquarters shared responsibility for overseeing QRs; did not always know whether mapping partners completed and uploaded QR checklists to MIP (based on the manager’s tendency not to use MIP); and did not know where QRs are housed in MIP. During the course of our review, we also learned that a Regional Program Manager may be responsible for up to 50 ongoing flood mapping projects in addition to first responder duties for disaster-related events. We judgmentally selected nine flood map projects completed between May 2014 and July 2016 to substantiate the lack of internal controls identified during the course of our review. None of nine flood map projects selected for our review complied with FEMA’s QR guidance. Specifically, all nine projects either did not have FEMA’s mandatory checklists or the required self-certification uploaded in MIP.1 Eight of the nine projects had a failed verification check for a draft, preliminary, final, or test after final flood map submittal. The results of our analysis follow. Quality Review Checks FEMA is not supervising contractor performance to ensure that required QR checklists and self-certifications are completed and uploaded into MIP. All nine projects selected for our review either did not have FEMA’s mandatory QR checklists or the required self-certification uploaded in MIP. Specifically, 19 out of 27 QR checklists (70 percent) and 8 out of 9 self-certifications (89 percent) were not in MIP at the time of our review. See table 1 for the results of our review by region. We selected one project from each of FEMA’s 10 Regions except Region 8, where we could not find any flood map projects completed during our test period. www.oig.dhs.gov 8 OIG-17-110 1 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Table 1: Total Number of Quality Review Checklists and Self Certifications Missing in the MIP by Region Required Quality Review Checklists in MIP? Region Case # 1 2 3 4 5 6 7 9 10 11-01-0889S 13-02-0708S 10-03-0355S 09-04-8601S 11-05-1556S 10-06-1115S 11-07-2747S 11-09-0920S 10-10-0993S Total Quality Review 3 Yes No No No No No No No Yes 7 of 9 Quality Review 7 Yes No No No No No No No No 8 of 9 19 of 27 Quality Review 8 Yes Yes No Yes No No No Yes Yes 4 of 9 Self – certifications in MIP? No No No No No No No No Yes 8 of 9 Source: OIG Analysis of MIP data For QR checklists not included in MIP, we contacted FEMA to verify whether mapping partners had in fact completed them. FEMA subsequently received from mapping partners 9 of the 19 missing checklists and 3 of the 8 missing self-certifications. Database Verification Tool Checks FEMA is not supervising contractor performance to ensure flood map project data passed all required verification checks. Additionally, FEMA did not ensure documentation was kept on file to support bypassing failed verification checks. Eight of the nine projects selected for our review had a failed verification check for either a draft, preliminary, final, or test after final flood map submittal. Specifically, 14 of 27 (55 percent) required verification checks failed and were bypassed, but there was no evidence of a bypass explanation in MIP. See table 2 for the results of our review by Region. www.oig.dhs.gov 9 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Table 2: Number of Database Verification Tool Bypasses Not on MIP Region Case # 1 2 3 4 5 6 7 9 10 11-01-0889S 13-02-0708S 10-03-0355S 09-04-8601S 11-05-1556S 10-06-1115S 11-07-2747S 11-09-0920S 10-10-0993S Total Number of Database Verification Tool Bypasses Not on MIP 1 1 2 2 0 2 2 3 1 14 Type of Submittal Final Final* Draft* and Preliminary* Draft and Final Draft and Preliminary Draft and Preliminary Draft, Preliminary, and Final Final Source: OIG analysis of MIP data *Provided by mapping partner To determine whether mapping partners completed bypass explanations, we contacted FEMA personnel who coordinated with mapping partners for bypass documentation. Mapping partners could not provide documentation for 11 of the 14 bypasses. We also found four instances where a verification check failed but the project advanced to the next phase without a bypass request and the map was issued. A FEMA official was unable to explain the reason FEMA advanced three of the four projects, and did not provide documentation to support map issuance. Without adequate oversight to ensure that mapping partner QRs comply with applicable guidance, FEMA cannot provide the public with a reliable rendering of their true flood vulnerability or ensure that NFIP rates reflect the real risk of flooding. Conclusion FEMA needs to improve its management and oversight of flood mapping projects to achieve program goals and ensure the production of accurate and timely flood maps. Flood hazard identification and mapping is an integral part of NFIP. It creates the foundation for floodplain management, flood insurance, and mitigation. Flood maps provide risk information and enable communities to make informed decisions such as adopting and enforcing minimum floodplain management regulations that reduce the loss of life and property. FEMA lacks uniform, centralized policies and procedures for projects placed on hold and is not adequately overseeing mapping partner quality reviews. Without accurate floodplain identification and mapping processes, management, and oversight, FEMA cannot provide members of the public with a reliable rendering of their true flood vulnerability or ensure that NFIP rates reflect the real risk of flooding. www.oig.dhs.gov 10 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Recommendations We recommend that the Federal Insurance and Mitigation Administration, Risk Management Deputy Assistant Administrator, Federal Emergency Management Agency: Recommendation 1: Implement a system of tracking regional cost allocations for flood map projects including studies and other investments, as well as the status of funds (expended and unexpended) and where they are in their lifecycle. Recommendation 2: Coordinate with the FEMA Chief Acquisition Executive Office to finalize and approve the Risk MAP LCCE. Recommendation 3: Develop an effective oversight strategy and implementation plan including necessary policies and procedures to enable mapping projects to move forward on a timely and consistent basis. Recommendation 4: Revise policies and procedures to require Regional Project Managers to – x x x ensure QR checklists and self-certifications are completed and uploaded into MIP; verify project data passes all Data Verification Tool checks prior to issuance of a flood map; and ensure documentation of all Data Verification Tool bypasses is uploaded in MIP. FEMA Comments and OIG Analysis FEMA concurs with all of the recommendations. In its comments, FEMA noted that it has made significant progress developing and implementing several flood-related policy and program changes since OIG began this audit. These changes include: (1) development and finalization of a Risk Mapping Assessment and Planning (Risk MAP) Program Lifecycle Cost Estimate (LCCE); (2) development and implementation of the Risk Management Directorate (RMD) Centric Budget Matrix Tool; and (3) updating the Risk MAP Quality Assurance Management Plan. We have included a copy of the management comments in their entirety in appendix B. A summary of FEMA’s response and our analysis follows. Response to Recommendation #1: Concur. FEMA’s RMD is developing budget tracking procedures with action steps for regional program components www.oig.dhs.gov 11 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security and staff. These steps include: (1) aligning cost tracking to production milestones, 2) working with regional staff to develop cost tracking guidance, and 3) implementing a review of the cost tracking procedures as part of the quarterly occurring joint performance reviews with the Regions to assess operations during FY 2018. Estimated completion date (ECD): September 30, 2018. OIG Analysis: FEMA’s corrective action is responsive to the recommendation. The recommendation will remain open and resolved until FEMA provides evidence to support that corrective actions are completed. Response to Recommendation #2: Concur. FEMA’s RMD initiated an update of the Risk MAP LCCE, led by the Risk MAP Program Manager, during the audit period. The LCCE is now in final routing for approval by FEMA Office of the Chief Financial Officer, Office of the Chief Information Officer, and the Chief Acquisition Executive. ECD: October 31, 2017. OIG Analysis: FEMA’s corrective action is responsive to the recommendation. The recommendation will remain open and resolved until FEMA provides evidence of the final approved version of the LCCE. Response to Recommendation #3: Concur. FEMA is developing a strategy to monitor the status of projects currently on hold. The strategy will include quarterly joint performance review meetings that will introduce performance metrics to ensure regions continue to maintain progress in moving projects forward to completion. Monthly cost and schedule performance reports will be reviewed during these meetings and corrective action will be provided as needed. FEMA will also develop a national implementation plan to categorize all on hold projects and will provide regions with guidance to move these projects forward in a consistent manner. ECD: to be determined. OIG Analysis: FEMA’s corrective action is responsive to the recommendation but will remain open and unresolved pending receipt of ECD and evidence of its strategy and implementation plan to ensure mapping projects currently placed on hold move forward in a timely manner. Response to Recommendation #4: Concur. FEMA‘s Quality Working Group finalized the Quality Assurance Management Plan requiring regions verify their project data passes all Data Verification Tool checks prior to issuance of a flood map. FEMA also update the Mapping Information Platform system to include quality review (QR) tasks and validation tasks. The QR task will be the repository for the upload of the QR checklists, Production and Technical Services self-certification documentation, and Data Verification Tool check information, to include results and bypasses, should they exist. The validations tasks will allow the Regions to validate that correct information is uploaded for www.oig.dhs.gov 12 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security the QR tasks and the project has met the quality requirements. A report of the data validation tasks and documentation checks will be developed and provided to all study managers on a quarterly basis to provide an additional level of visibility on the process and data collection. ECD: to be determined. OIG Analysis: FEMA’s corrective action is responsive to the recommendation but will remain open and unresolved pending receipt of ECD and until we have reviewed FEMA’s Quality Assurance Management Plan. www.oig.dhs.gov 13 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Appendix A Objective, Scope, and Methodology The Department of Homeland Security (DHS) Office of Inspector General (OIG) was established by the Homeland Security Act of 2002 (Public Law 107ï296) by amendment to the Inspector General Act of 1978. We conducted this audit to determine whether Federal Emergency Management Agency’s (FEMA) Risk Mapping, Assessment and Planning Program (Risk MAP) assessed and updated flood maps timely and accurately in accordance with National Flood Insurance Program (NFIP) requirements. Specifically, we reviewed whether FEMA is meeting its program goal to ensure that 80 percent of flood hazard miles are current with New, Valid, or Updated Engineering (NVUE); oversaw the progress and completion of riverine flood map projects placed on hold; and ensured mapping partner quality reviews comply with requirements set forth in FEMA’s quality review guidance. To accomplish our objective, we obtained and reviewed pertinent Federal and FEMA specific regulations, policies, procedures, and guidance relevant to flood mapping. We interviewed FEMA officials and staff responsible for the management and oversight of flood mapping projects. To determine whether FEMA met its program goal of 80 percent NVUE miles, we obtained FEMA’s report titled Overview of Risk MAP Coordination Needs Management Strategy and NVUE Status issued to Congress on September 18, 2015. We compared the data in this report to reports from FEMA’s Coordination Needs Management Strategy to verify accuracy. We obtained additional fiscal year data from the Coordination Needs Management Strategy system to present the current year NVUE attained percentage presented in figure 2 of this report. For the purpose of this report we determined that the data was sufficiently reliable. To identify unexpended obligations, we obtained a listing of financial transactions from FEMA’s Integrated Financial Management Information System (IFMIS) for fiscal years 2009 through 2016. We subtracted the total expenditures and adjustments from total obligations included in the listing of IFMIS transactions. We were unable to test the data reliability of unexpended funds because FEMA does not track obligations and expenditures at the task order level for legacy contracts. To assess the process of assuring the timely production of flood maps, we identified the number of all active and on hold projects between April and November 2016 (1,947 projects) from FEMA’s Mapping Information Platform (MIP) and FEMA regions. We limited the scope of our review to riverine mapping www.oig.dhs.gov 14 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security projects that were categorized as “on-hold” in MIP as riverine projects are the largest program cost within Risk MAP. In total we identified 240 “on-hold” projects with expenditures totaling $41 million. We developed and sent out a questionnaire to seven FEMA regions to understand how each manages, tracks, and processes on-hold projects. Based on responses received, we conducted interviews with seven regional officials who manage flood map projects. We judgmentally selected these regions based on criteria such as the number of on-hold projects at the time of our review and type of responses given to our questionnaire to obtain a broad range of view on how projects are managed. To assess the reliability of the on-hold data, we compared the data generated from MIP to each region’s responses and discussed the on-hold project data with regional officials. We also “spot checked” for completeness and accuracy by reviewing the information provided by the regions to MIP. We found inconsistencies in some of the projects status during our interview with one region and also found some data was missing because the case numbers were not added to MIP. However, for the purpose of this report we determined that the data was sufficiently reliable. To determine whether quality reviews were conducted in accordance with FEMA’s guidance, we verified whether (1) mandatory QR checklists and selfcertifications were completed and uploaded in MIP; (2) there was documentation of a bypass request in MIP when project data failed a Data Verification Tool check; and (3) projects passed all required Data Verification Tool checks prior to FEMA issuing a flood map. We judgmentally selected 9 of 1,894 coastal and riverine projects from FEMA’s Risk MAP Progress website on July 7, 2016, that were prepared by production and technical services mapping partners and completed between May 2014 and July 2016. Production and technical services contractors and Cooperative Technical Partners (CTP) completed a total of 1,894 projects. However, our scope does not include projects completed by CTPs. The focus of our review was on riverine flooding; however, we included one coastal project in our sample as it was reviewed while developing our understanding of FEMA’s quality review process during our survey phaseǤ We selected one project for each of FEMA’s 10 regions except for Region 8, as we were unable to find in that region a production and technical services project that was completed during our test period. Additionally, we relied on the data provided in the FEMA’s Risk MAP Progress website and did not conduct further reliability tests of the information. We did not select more than nine projects because FEMA personnel stated that they were not overseeing the quality review process conducted by mapping partners. www.oig.dhs.gov 15 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security We conducted this performance audit between March and December 2016 pursuant to the Inspector General Act of 1978, as amended, and according to generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based upon our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based upon our audit objectives. The Office of Audits major contributors to this report are: Maureen Duddy, Deputy Assistant Inspector General for Audits; Carolyn Hicks, Director; Paul Exarchos, Audit Manager; Andrew Smith, Audit Manager; Jeff Mun, co-Auditor in-charge; Roger Thoet, co-Auditor in-charge, Corneliu Buzesan, Program Analyst; Steffanie Moore, Program Analyst; Aneet Marwaha, Program Analyst; Ellen Gallagher, Communications Analyst; and Juan Santana, Independent Referencer. www.oig.dhs.gov 16 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Appendix B FEMA Comments to the Draft Report www.oig.dhs.gov 17 OIG-17-110 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Attachment: DHS Management Response to Recommendations Contained in The OIG recommended that the Federal Insurance and Mitigation Administration, Risk Management Deputy Assistant Administrator, Federal Emergency Management Agency: Recommendation 1: Implement a system of tracking regional cost allocations for ?ood map project including studies and other investments, as well as the status of funds (expended and unexpended) and where they are in their lifecycle. Response: Concur. During the audit period, RMD developed budget tracking procedures that generally enhanced effectiveness in managing funding allocations. RMD is now focused on expanding enhancement efforts with action steps for regional program components and staff. These steps include: - Aligning cost tracking to production milestones. Since the audit, FEMA has updated the Mapping Information Platform (MIP) to improve tracking of production through the phases bounded by the Key Decision Point process. FEMA will leverage this new feature to provide a framework for cost tracking procedures. Estimated Completion Date (ECD): December 31, 2017. 0 Using this new framework, FEMA will work with our Regional staff to develop cost tracking guidance, which will integrate cost tracking into MIP production phases. In this phase, FEMA will also establish monitoring mechanisms to assess the effectiveness of these new procedures. ECD: March 31, 2018. 0 Implement a review of the cost tracking procedures as part of the quarterly occurring Joint Performance Reviews (JPRs) with the Regions to assess operations during FY 2018. ECD: September 30, 2018. Overall ECD: September 30, 2018. Recommendation 2: Coordinate with the FEMA Chief Acquisition Executive staff to ?nalize and approve the Risk MAP LCCE. Response: Concur. RMD initiated an update of the Risk MAPLCCE, led by the Risk MAP Program Manager, during the audit period. The LCCE update was coordinated with Component Acquisition Executive Staff, ?nalized, and is now in ?nal routing for approval by FEMA Of?ce of the Chief Financial Of?cer, Of?ce of the Chief Information Of?cer, and the CAB. ECD: October 31, 2017. Recommendation 3: Develop an effective oversight strategy and implementation plan including necessary policies and procedures to enable mapping projects to move forward on a timely and consistent basis. Response: Concur. RMD recognizes the need to develop an oversight strategy and implement a plan to ensure mapping projects currently placed on hold move forward in a timely manner. 18 17? 1 1 OFFICE OF INSPECTOR GENERAL Department of Homeland Security To ensure adequate oversight, FEMA will develop a strategy to monitor the status of projects currently on hold by April 30, 2018. This strategy will include the institution of quarterly PR meetings with FEMA Regions and the introduction of performance metrics to ensure Regions continue to maintain progress in moving these projects forward to completion. FEMA will also leverage annual planning and funding memos to require Regions to advance projects. FEMA will develop a national implementation plan to categorize all on hold projects and will provide Regions with guidance to move these projects forward in a consistent manner. The timing to implement this plan is dependent upon available budget and resources due to the need to re-procure work for expired contracts to meet the most current standards and guidance. While additional focus has been on closing out on hold projects, all active projects are monitored via cost and schedule performance metrics using an Earned Value Management (EVM) system to track project status against cost and schedule baselines. cost and schedule performance reports are produced via the Map Information Platform (MIP). When JPR quarterly meetings are held, this information will be reviewed and corrective action provided as needed. Overall ECD: To Be Determined (TBD) Recommendation 4: Revise policies and procedures to require Regional Project Managers to: 0 ensure Quality Review (QR) checklists and self-certi?cations are completed and uploaded into - verify project data passes all Data Veri?cation Tool (DVT) checks prior to issuance of a ?ood map', and a ensure documentation of all DVT bypasses is uploaded in MIP. Response: Concur. The Risk MAP Quality Working Group (QWG) is a FEMA internal working group composed of FEMA staff and Risk MAP Program contractors. During the audit period, the QWG ?nalized the QAMP which has been reviewed by all Risk MAP contractor teams and signed off on by the Risk MAP Program Manager. The QAMP is the guiding document that sets the criteria for each provider?s more specific Quality Management Plan. Further, the QAMP enforces the requirement for compliance to ensure that QR checklists and self-certi?cations are uploaded to the MIP for compliance and to align with Risk MAP Program Standard ID (SID) 508. The updated QAMP also states the requirements outlined in SID 508, indicating that Regions must verify their project data passes all DVT checks prior to issuance of a ?ood map. In June 2017, mapping project management tool the MIP) was updated to include QR tasks and validation tasks. The QR task will be the repository for the upload of the QR checklists, Production and Technical Services self-certi?cation documentation, and DVT check information, to include results and bypasses, should they exist. The validations tasks will allow the Regions to validate the correct information is uploaded to the QR tasks and the project has met the quality requirements. These new tasks will ensure l9 17? OFFICE OF INSPECTOR GENERAL Department of Homeland Security the appropriate documentation is uploaded with the study and the quality requirements have been met, since the project will not be able to move forward without passing these validation steps. A report out of the data validation tasks and documentation checks will be developed and provided to all study managers on a quarterly basis to provide an additional level of visibility on the process and data collection. The formalization and finalization of the QAMP better aligns Risk MAP processes with SID 508 and Recommendation 4. The QAMP will continue to be a living document and the QWG will evaluate the QAMP every six months to ensure alignment with Program changes. A critical component to ensuring the success and socialization of the QAMP and setting the expectation to comply with SID 508 is the identification and creation of trainings related to quality. These trainings will bolster the programmatic knowledge of FEMA Headquarters and Regional staff and our mapping partners. These trainings will also ensure that the Regions have the proper familiarity with the processes, roles, and responsibilities outlined in the QAMP and how it relates to SID 508. Trainings will be identi?ed and created by February 28, 2018, with the ?rst training delivered by August 30, 2018. The Risk MAP QWG chair at FEMA Headquarters will be responsible for the delivery of these trainings. During the next several months by December 30, 2017), socialization of the QAMP to the FEMA Regions will continue to occur. Many stakeholders in the Regions were not part of the QAMP update process, so this socialization effort will ensure these stakeholders receive the updated QAMP and are able to ask questions related to the update. It is important to note, however, that while overall socialization to Regions is essential, both PTS contractors that perform QRs for the Regions have been part of the development of the QAMP and are currently aware of the expectations that the QAMP outlines. This continued socialization will be the responsibility of the Risk MAP QWG chair at FEMA Headquarters. Overall ECD: TBD 20 17? 1 1 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Appendix C Report Distribution Department of Homeland Security Secretary Deputy Secretary Chief of Staff General Counsel Executive Secretary Director, GAO/OIG Liaison Office Assistant Secretary for Office of Policy Assistant Secretary for Office of Public Affairs Assistant Secretary for Office of Legislative Affairs Office of Management and Budget Chief, Homeland Security Branch DHS OIG Budget Examiner Congress Congressional Oversight and Appropriations Committees www.oig.dhs.gov 21 OIG-17-110 ADDITIONAL INFORMATION AND COPIES To view this and any of our other reports, please visit our website at: www.oig.dhs.gov. For further information or questions, please contact Office of Inspector General Public Affairs at: DHS-OIG.OfficePublicAffairs@oig.dhs.gov. Follow us on Twitter at: @dhsoig. OIG HOTLINE To report fraud, waste, or abuse, visit our website at www.oig.dhs.gov and click on the red "Hotline" tab. If you cannot access our website, call our hotline at (800) 323-8603, fax our hotline at (202) 254-4297, or write to us at: Department of Homeland Security Office of Inspector General, Mail Stop 0305 Attention: Hotline 245 Murray Drive, SW Washington, DC 20528-0305