OFFICE OF THE CITY AUDITOR / City. and County of Honolulu State of Hawai’i Audit of the Honolulu Authority Fo~ Rapid Transportation (HART) COUNCIL COM. 10 Report No. 16-03/April 2016 Audit of the Honolulu Authority For Rapid Transportation (HART) A Report to the Mayor and the City Council of Honolulu Submitted by THE CITY AUDITOR CITY AND COUNTY OF HONOLULU STATE OF HAWAI`I Report No. 16-03 April 2016 OFFICE OF THE CITY AUDITOR CITY AND COUNTY OF HONOLULU 1001 KAMDKILA BOILEVAHD. 216. KAPDLEI HAWAII PHONE (608} 768-3134 (008) TBS-3135 EDWIN S.W. YOUNG CITY AUDITOR April 15, 2016 The Honorable Ernest Y. Martin, Chair and Members Honolulu City Council 530 South King Street, Room 202 Honolulu, Hawaii 96813 Dear Chair Martin and Councilmembers: A copy of our ?nal report on the Audit of the Honolulu Authority for Rapid Transportation (HART) is attached. City Council Resolution 15-90 requested a performance audit of HART to determine the adequacy of the HART's processes to ensure that the rail project is constructed and completed economically, effectively, and ef?ciently. Per the resolution, the audit sub-objectives were to examine the HART ?nancial and contingency plans; contract awards, expenditures, and subcontractor costs; and evaluate the project cost increases and shortfalls. Other sub-objectives were to review rail project revenues and expenditures from 2007 through 2014; determine when HART was aware of the ?nancial shortfalls; and to assess the potential for additional cost overruns. The resolution requested our of?ce to evaluate the potential continuing costs after the project is completed and operational. This audit was performed in accordance with generally accepted government auditing standards from May 2015 to April 2016. Background The city voters approved a ?xed guideway system for the City and County of Honolulu in 2008 and approved an amendment to the city charter in the 2010 general election that established the Honolulu Authority for Rapid Transportation. HART was formed on July 1. 2011. HART is a semi-autonomous agency of the City and County of Honolulu government. HART is responsible for completing a 20-mile ?xed rail system that will run from Kapolei in the Western and of the city to Ala Moana Shopping Center. Despite having a goal of completing the project on time and on budget. HART project costs have increased $1.3 billion (25 percent) from the original estimate of $5.2 billion to an estimated $6.5 billion. Project construction started in 2012. The project was originally to be completed by January 2020 with interim rail service beginning in June 2016 and full service starting in March 2019. Project delays and recent estimates indicate rail service will not begin on time and may be delayed to late 2018 for interim service and late 2021 for full service operations. The Honorable Ernest Y. Martin, Chair and Members April 15, 2016 Page 2 of 3 Audit Results HART's processes can be improved to construct and complete the project more economically, effectively, and ef?ciently. Speci?cally, we found that HART's ?nancial and subsidiary plans are not reliable or current, and HART's ?nancial plan has not been updated to re?ect the rail project's most current ?nancial condition in spite of the signi?cant cost increases. HART needs to strengthen its controls over ?nancial information reporting to ensure data is complete and readily available from its Contract Management System delay claims are adequately tracked, monitored, and reported; and pending utility agreements, contingency allowance ?gures, and general excise tax (GET) county surcharge receipts are complete and accurate. Project management and contract administration controls can be improved. More speci?cally, HART's Project Management Plan (PMP) and Operations and Maintenance Plan (0MP) are outdated and unreliable as decision-making tools. HART made concessions to a single contractor; did not perform quantitative analysis to justify a major decision to repackage a bid for nine stations in the Westside Stations Group; and paid $1.5 million in stipends to unsuccessful bidders without knowing the bidders' actual costs. HART is also paying for vacant of?ce space. Contract administration controls need to address invoice payments, procurement ?le documentation, and prevent improper payments. Better planning is needed to address and manage future rail project costs. Subsidies will be needed to fund rail operations and maintenance costs after the rail is constructed. HART needs to improve planning and oversight to effectively address and manage future operations and maintenance needs; maximize fare box recoveries and ridership; and minimize city subsidies. Absent the improvements, we anticipate additional cost overruns will occur. More speci?cally, project cost estimates, details, methodologies, and cost assumptions are unsubstantiated and project managers are not managing actual costs against their budgets. Management Response HART is staffed with many dedicated individuals who are striving to successfully complete the rail project. We are therefore disappointed by the HART responses which mischaracterize and misrepresent the audit discussions and the history of the audit. HART had over ?ve weeks to respond to the draft reports. Distribution of the con?dential drafts was limited to only authorized HART and city of?cials. HART requested and received early distribution of the drafts and conceded it distributed the con?dential drafts within its organization without our permission. HART's attempts to discredit the audit work and attempts to intimidate the auditors were unprofessional. In its response to the management discussion draft, HART disagreed with the audit ?ndings and recommendations and provided additional documents and data which we closely examined. Based on our analysis and veri?cation of the additional information, we modi?ed the report and updated the ?ndings. However, our analysis and veri?cation of the additional information reaf?rrned our audit results. Based on our audit work and supporting work papers, we stand by our audit ?ndings, conclusions, and recommendations. It is our hope that HART will reconsider and implement the recommendations needed to improve its ?nancial management, planning, project management, contract administration and other operations to ensure the rail project costs are minimized. The Honorable Ernest Y. Martin, Chair and Members April 15, 2016 Page 3 of 3 We express our appreciation for the c00peration and assistance provided us by the staffs of HART, its consultants, and the many others who provided inputs for this review. If you have any questions regarding the audit report, please call the auditor-in-charge, Amy Cheung, at 768-9233 or me at 768-3130. Sincerely, Edwin S. W. Young City Auditor c: Dan Grabauskas, Executive Director and CEO, HART Kirk Caldwell, Mayor Roy Amemiya, Jr., Managing Director Diane Arakaki, Program Administrator, HART Nelson Koyanagi, Jr., Director, Department of Budget and Fiscal Services HART Board of Directors This page intentionally left blank. Table of Contents Chapter 1 Introduction and Background Introduction............................................................................................................................1 Background.............................................................................................................................1 The Honolulu Rail Transit Project.......................................................................................2 Project Schedule.....................................................................................................................3 Funding Sources.....................................................................................................................4 Chapter 2 HART Needs to Improve Financial Management and Planning Summary.................................................................................................................................5 Background.............................................................................................................................5 PMOC Expressed Issues and Concerns in 2012................................................................7 HART’s Processes Can Be Improved to More Economically, Effectively, and Efficiently Report Project Costs.....................................................................................10 Project Cost Estimates Lacked Supporting Documentation..........................................15 Improved Financial Management and Planning Are Needed......................................21 Recommendations...............................................................................................................23 Chapter 3 HART Needs to Improve Project Management and Contract Administration Summary...............................................................................................................................25 Background...........................................................................................................................25 Construction Cost Overruns Are Not Unusual...............................................................26 Additional HART Rail Project Cost Overruns and Shortfalls Are Likely ..................27 Contract Administration Needs to Be Improved............................................................39 Recommendations...............................................................................................................43 Chapter 4 Better Planning Is Needed to Address and Manage Future Rail Project Costs Summary...............................................................................................................................45 Background...........................................................................................................................45 HART Needs to Plan for Annual Operations and Maintenance of the Rail System...............................................................................................................................46 Other Rail Systems Indicate Annual and Ongoing Operations and Maintenance Costs Must Be Addressed..............................................................................................46 Subsidies Will Be Needed to Fund Rail Operations and Maintenance Costs.............47 HART Needs to Improve Planning to Maximize Fare Box Recoveries.......................48 HART Needs to Improve Planning to Maximize Ridership..........................................50 Recommendations...............................................................................................................52 i Chapter 5 Conclusions and Recommendations Recommendations...............................................................................................................54 Management Response.......................................................................................................56 Appendices Appendix A Appendix B Appendix C Appendix D Appendix E Audit Objectives, Scope, and Methodology.....................................................................59 Glossary of Acronyms.........................................................................................................63 Glossary of Terms and Definitions....................................................................................65 Contingency Balance Comparison (by Month), (Dollars in Millions)..........................69 General Excise Tax (GET) Receipts Comparisons (by Quarter), (Dollars in Millions)........................................................................................................71 Appendix F Procurement Documentation File......................................................................................73 Appendix G City Council Resolution 15-90............................................................................................75 Appendix H Management Response........................................................................................................79 List of Exhibits Exhibit 1.1 Exhibit 1.2 Exhibit 2.1 Exhibit 2.2 Exhibit 2.3 Exhibit 2.4 Exhibit 2.5 Exhibit 2.6 Exhibit 2.7 Exhibit 2.8 Exhibit 2.9 Exhibit 2.10 Exhibit 3.1 Exhibit 3.2 Exhibit 3.3 Exhibit 3.4 Exhibit 3.5 Exhibit 3.6 Exhibit 3.7 Exhibit 3.8 Exhibit 3.9 Exhibit 4.1 Exhibit 4.2 Exhibit 4.3 ii HART Organization Chart....................................................................................................2 Map of the Honolulu Fixed Rail System.............................................................................3 New Starts Grant Allocation (by Federal Fiscal Year)1 ....................................................6 Estimated Project Cost Increases and Revenue Shortfalls (Dollars in Thousands)......8 Funding Milestones...............................................................................................................9 Project Cost Estimates (Dollars in Thousands) (As of June 2012, December 2014, and October 2015*)..........................................................................................................12 Contract Balance Comparison (Dollars in Millions).......................................................14 Incomplete Delay Claims Summary (Dollars in Thousands)........................................15 Reporting Comparison of Delay Claim Costs .................................................................16 Utility Costs Increase (Dollars in Millions)......................................................................18 Project Enhancement Costs Increase (Dollars in Millions)............................................19 Escalation Costs Increase (Dollars in Millions)................................................................21 Factors that Contributed to the Project Cost Increase and Revenue Shortfall (Dollars in Thousands)...................................................................................................26 Capital Costs Comparison of Heavy and Light Rail Projects........................................27 Contingency Reserves Increase (Dollars in Thousands)................................................31 Key Management Plans.......................................................................................................33 Westside Stations Group Bid Cancellation Timeline......................................................35 Stipend Payments.................................................................................................................37 Office and Workstation Vacancy Rates.............................................................................38 Photos of Vacant Offices and Workstations.....................................................................39 Summary of Contract Types Reviewed............................................................................40 Project Schedule....................................................................................................................45 Subsidy Comparisons (Dollars in Millions).....................................................................46 Sources of Operating Funds by Transit Agency*............................................................48 Exhibit 4.4 Exhibit 4.5 Rail Farebox Recovery Rates Comparison........................................................................49 Ridership Forecast vs. Actual.............................................................................................51 iii This page intentionally left blank. Chapter 1: Introduction and Background Chapter 1 Introduction and Background Introduction This audit was conducted pursuant to City Council Resolution 15-90, Requesting the City Auditor to Conduct a Performance Audit of the Honolulu Rail Transit Project, to determine the adequacy of the Honolulu Authority for Rapid Transportation’s (HART) processes to ensure that the rail project is constructed and completed economically, effectively and efficiently. Per the resolution, the audit sub-objectives were to examine the HART financial and contingency plans; contract awards, expenditures, and subcontractor costs; and evaluate the project cost increases and shortfalls. Other sub-objectives were to review rail project revenues and expenditures from 2007 through 2014; determine when HART was aware of the financial shortfalls; and to assess the potential for additional cost overruns. The resolution requested our office to evaluate the potential continuing costs after the project is completed and operational. Background HART is a semi-autonomous local government agency established in July 2011 by a charter amendment, after voters approved a fixed guideway system in 2010. HART’s mission is to plan, design, construct, operate, and maintain Honolulu’s high-capacity, fixed guideway rapid transit system. HART is governed by a 10-member board consisting of 3 members appointed by the mayor, 3 members appointed by the city council, 3 ex-officio members, and 1 member appointed by the board. Board members serve five-year staggered terms and are not compensated during their terms. The board meets monthly and establishes HART’s overall policy for the development, operation, and maintenance of the public transit system. In 2012, the board appointed a chief executive officer (CEO) to manage and provide leadership to HART. 1 Chapter 1: Introduction and Background Exhibit 1.1 HART Organization Chart HART BOARD ADMINISTRATOR STATE OVERSIGHT PROGRAM MANAGER EXECUTIVE MANAGEMENT LEGAL COUNSEL CORPORATION COUNSEL SENIOR CLERK/ RECEPTIONIST PRIVATE SECRETARY DEPUTY EXECUTIVE DIRECTOR PROJECT DIRECTOR PRIVATE SECRETARY SECRETARY DESIGN AND CONSTRUCTION DIRECTOR ADMINISTRATIVE SERVICES DIRECTOR PROJECT CONTROLS MANAGER DIRECTOR OF SPECIAL PROJECTS CONTRACT ADMINISTRATION AND CHANGE DIRECTOR BUDGET AND FINANCE CHIEF FINANCIAL OFFICER PLANNING, PERMITTING AND RIGHT OF WAY DIRECTOR CIVIL RIGHTS CIVIL RIGHTS OFFICER PROCUREMENT AND CONTRACTS DIRECTOR GOVERNMENT RELATIONS DIRECTOR OPERATIONS AND MAINTENANCE DIRECTOR FARE SYSTEM PROJECT MANAGER PUBLIC INFORMATION DIRECTOR OF COMMUNICATIONS QUALITY ASSURANCE DIRECTOR SYSTEM SAFETY AND SECURITY CHIEF SAFETY AND SECURITY OFFICER Source: OCA based on HART information as of December 11, 2015. The Honolulu Rail Transit Project 2 The $5.2 billion Honolulu Rail Transit Project (rail project) is the largest, most expensive public works project in the State of Hawai`i’s history. The project is intended to mitigate traffic congestion between O`ahu’s east to west transportation corridors, and will consist of 21 stations that will connect the East Kapolei station to the Ala Moana Center station. Chapter 1: Introduction and Background Exhibit 1.2 Map of the Honolulu Fixed Rail System Source: HART Project Schedule The project was originally to be completed by January 2020 with interim rail service beginning in June 2016 and full service projected to start in March 2019. Project delays and recent estimates indicate rail service will not begin on time and may be delayed to late 2018 for interim service and late 2021 for full service operations. 3 Chapter 1: Introduction and Background Funding Sources HART’s operational and capital costs for the project are funded through various federal, state, and local monies. • Federal funding comes from a $1.55 billion grant from the Federal Transit Administration (FTA) Section 5309 New Starts Fund1. As of November 27, 2015, HART received $472.5 million of the federal grant. • The State of Hawai`i imposed a half percent (0.5 percent) General Excise Tax (GET) surcharge for the City and County of Honolulu to fund the project. The county surcharge was to expire on December 31, 2022. It has since been extended to 2027. • The City and County of Honolulu will issue general obligation bonds (debt financing) to fund construction of the rail project. In November 2015, the city council approved legislation that enabled the city to issue up to $350 million in general obligation commercial paper to fund the public improvements and equipment related to the rail project. The Federal Transit Administration New Starts program is the federal government’s primary financial resource for locally planned, implemented, and operated major transit capital investments. The New Starts program funds new projects and extensions to existing fixed guideway transit systems in every area of the country. 1 4 Chapter 2: HART Needs to Improve Financial Management and Planning Chapter 2 HART Needs to Improve Financial Management and Planning Summary Despite having a goal of completing the project on time and on budget, Honolulu Authority for Rapid Transportation’s (HART) project costs have increased $1.3 billion (25 percent) from the original estimate of $5.2 billion to an estimated $6.5 billion. HART’s processes can be improved to construct and complete the project more economically, effectively, and efficiently. Specifically, we found that HART’s financial and operating plans are not reliable or current; and HART’s financial plan has not been updated to reflect the rail project’s most current financial condition in spite of the significant cost increases. HART needs to strengthen its controls over financial information reporting to ensure data is complete and readily available from its Contract Management System (CMS); delay claims are adequately tracked, monitored, and reported; and pending utility agreements, contingency allowance figures, and general excise tax (GET) county surcharge forecasts are accurately reported. Absent the improvements, we anticipate additional shortfalls and cost overruns will occur. Background HART’s mission is to plan, design, construct, operate, and maintain Honolulu’s high-capacity, fixed guideway rapid transit system. To fulfill its mission, HART is responsible for completing the project on time and within budget and ensuring the design and actual construction of the project will facilitate the delivery of a safe, high quality, and cost-efficient service in the future. HART is also responsible for maintaining public trust through prudent and transparent use of financial, human, and environmental resources. Other HART responsibilities are to support the creation of mixed use, pedestrian-friendly, compact development along the rail line; pursuing partnerships with the private sector to create economic opportunities and generate income and cost savings for the rail transit system; and fostering an organization that is open, accountable, inclusive, and delivers better than promised results. As of November 2015, HART received $472.5 million of the $1.5 billion federal grant for the rail project. Per the 2012 Full Funding 5 Chapter 2: HART Needs to Improve Financial Management and Planning Grant Agreement (FFGA), HART and the city had to comply with the grant terms to receive the remaining balance of $1 billion. The exhibit below details the allocations by federal fiscal year. Exhibit 2.1 New Starts Grant Allocation (by Federal Fiscal Year)1 Federal Fiscal Year Allocation 2008 FTA Obligated Amounts $ 15,190,000 Actual Drawdown Amounts to Date $ 15,190,000 Available Balance $ - 2009 19,800,000 19,800,000 - 2010 30,000,000 30,000,000 - 2011 55,000,000 55,000,000 - 2012 200,000,000 200,000,000 - 2013 236,277,358 152,519,166 2014 250,000,000 83,758,192 - 250,000,000 2015 250,000,000 - 250,000,000 2016 250,000,000 - 250,000,000 2017 Total 243,732,642 $ 1,550,000,000 $ 472,509,166 243,732,642 $ 1,077,490,834 Source: HART data as of November 27, 2015 The federal fiscal year runs from October 1 to September 30. The table covers federal funding only. Total rail project funding includes federal, state, and local funding sources. 1 Federal funding includes $4 million of the American Recovery and Reinvestment Act (ARRA) funds out of $214 million of FTA Section 5307 Formula. State funding includes the state half percent (0.5 percent) General Excise Tax (GET) county surcharge to fund the rail project. The GET county surcharge originally was to expire December 31, 2022, but was extended through December 31, 2027 to cover the additional project cost increases and revenue shortfall. HART estimates the five-year extension will generate revenue in the range of $1.2 billion to $1.8 billion. GET collections from FY 2007 to FY 2015 totaled $1.522 billion. City funding includes general obligation bonds (debt financing) to fund construction of the rail project. In November 2015, the city council approved legislation that allowed the city to issue up to $350 million in general obligation commercial paper to fund rail project related improvements and equipment. HART also anticipates using the funds to cover its short-term cash flow needs. In 2015, city council resolution 15-18 eliminated the use of $210 million of FTA Section 5307 grant monies in the project’s financial plan to ensure the funds were only used for city transportation services (i.e. TheBus and Handi-Van services). 6 Chapter 2: HART Needs to Improve Financial Management and Planning PMOC Expressed Issues and Concerns in 2012 The FTA hired an independent consultant to monitor the construction project, identify problems, and to report deficiencies or concerns. The HART project management oversight consultant (PMOC) monthly reports discussed issues and concerns over the viability of HART’s operations. In the monthly reports, the PMOC questioned the following: • The adequacy of HART’s ability to “forecast costs for the existing design-build contracts.” It emphasized that it is critical that this issue be quickly corrected to demonstrate that the grantee has the technical capacity and capability going forward.2 • The lack of “technical capacity and capability specific to project controls.3” • The sufficiency of contingency reserves; and • The need to develop, update, and implement secondary risk mitigation measures. In December 2014, the HART’s chief executive officer (CEO) stated that the agency was facing a $600 million cost overrun and a $310 million revenue shortfall. The public statement notified the city council that project costs had increased and revenues were less than projected. Actions to resolve the shortfalls: In January 2015, the Hawai`i State Legislature introduced bills to extend the rail project GET county surcharge from December 2022 to December 2027. HART and city officials lobbied in support of the GET extension and the legislature passed the bill in May 2015. In the October 2012 monthly report, the PMOC noted that HART and the PMOC have held monthly breakout sessions to review the status of the forecast costs, schedule management, risk management, and cost containment measures. The report noted that these breakout sessions have resulted in increased confidence by the PMOC of the grantee’s ability to manage the project budget and schedule. 3 Project Controls are acts of project management staff in all aspects of cost, schedule, contract administration, and configuration management. In the February 2013 PMOC monthly report, HART acknowledged the situation and hired a new Project Controls Manager. 2 7 Chapter 2: HART Needs to Improve Financial Management and Planning After the governor signed the bill in July 2015, HART reported to the HART Board of Directors on October 15, 2015 that the cost overruns had increased an additional $714 million, for a total of $1.3 billion. The exhibit below details the changes in estimated project costs and revenues. Exhibit 2.2 Estimated Project Cost Increases and Revenue Shortfalls (Dollars in Thousands) Cost Estimates Description December 2014 HART Board Meeting Change October 2015 HART Board Meeting Estimated Costs Increase: Lawsuits and Delay Claims Utility Relocations Project Enhancements Cost Escalation Allocated Contingency Unallocated Contingency Debt Financing Costs Total Estimated Costs Increase $190,000 50,000 75,000 45,000 240,000 --$600,000 $190,000 120,000 130,000 240,000 240,000 299,000 95,000 $1,314,000 Estimated Revenue Shortfall: GET Projected Shortfall FTA 5307 Fund Reduction Total Estimated Revenue Shortfall $100,000 210,000 $310,000 $100,000 210,000 $310,000 $910,000 $1,624,000 Total $ % $714,000 54% $714,000 44% Source: Office of the City Auditor (OCA) analysis based on HART Project Risks Update, December 18, 2014 and Project Cost Update, October 15, 2015. Impact of shortfalls: HART subsequently came under increasing scrutiny by policy makers, the local media, and the public. Throughout the project, local news reports drew public attention on the credibility of HART’s project cost information, and policy makers expressed concerns over the lack of detailed financial information provided by HART for decision-making purposes. The city council also expressed concerns related to the HART data, financial management and planning, decision making, contract administration, and post-construction costs. As a result, the city council delayed approving the GET surcharge extension. 8 Chapter 2: HART Needs to Improve Financial Management and Planning After the Federal Transit Administration (FTA) issued a warning related to the lack of project funding, members of the city council approved the county surcharge in January 2016. The FTA stated, in a letter to the mayor, the next $250 million federal installment will not be released until the city and HART provided a revised cost estimate and schedule, an updated financial plan, and a commitment of local funds to cover the increased cost estimates. The exhibit below details the timeline. Exhibit 2.3 Funding Milestones • December 2014 HART announced a $600 million projected capital costs increase and a $310 million revenue shortfall. • January 2015 Hawai`i State Legislature circulated senate and house bills regarding the GET county surcharge extension. • April 2015 FTA confirmed the need to repay funds spent in full to the federal government if the rail project is cancelled. • May 2015 Hawai`i State Legislature passed House Bill 134, extending the GET county surcharge extension from December 31, 2022 to December 31, 2027. • July 2015 Governor signed GET county surcharge extension bill into law (Act 240). Approval to adopt the ordinance (Bill 23) by the City County must be decided by June 30, 2016. • October 2015 HART projected an additional $714 million in project cost increases. • November 2015 FTA issued a letter expressing concern that the city has not yet completed the actions needed to extend the GET surcharge that is critical to completing the project. • January 2016 City Council adopted city ordinance 16-1 supporting the 5-year extension of the GET county surcharge. • February 2016 Mayor signed bill into law. Source: OCA analysis based on various sources 9 Chapter 2: HART Needs to Improve Financial Management and Planning HART’s Processes Can Be Improved to More Economically, Effectively, and Efficiently Report Project Costs Regular reporting provides management with information necessary to make sound decisions and to be transparent and accountable to key stakeholders and the public. Our review found that HART can improve its financial management and planning by retaining and providing reliable project cost information to policy makers and decision makers. Despite significant changes, HART has not regularly updated and reported accurate and reliable project cost information. As a result, HART reports contain inconsistent project cost data which limit the overall usefulness of its financial planning, project cost, and funding information. More specifically, HART needs to: • Regularly update financial and operating information and plans; • Provide reliable and consistent project cost information; • Effectively track, monitor, and report on delay claims and related costs; • Document and support utility cost increases and estimated cost overruns for project enhancements; • Use specific and consistent factors in calculating and estimating escalation costs; and • Properly report on GET county surcharge forecasts. Financial and operating plans are not regularly updated 10 An FTA grantee must demonstrate financial management and capacity to match and manage FTA grant funds and to cover cost increases and operating deficits. Chapter 2: HART Needs to Improve Financial Management and Planning To ensure compliance with the FTA requirements, HART should follow best practices that ensure its financial and operating plans are regularly updated and are accurately reflected in its rail project financial reports. In the FTA 2015 triennial review4, the FTA reported deficiencies in the project’s financial management and capacity. Specifically, HART’s financial plan did not demonstrate sufficient financial capacity to complete the project as currently planned. HART did not update its financial plans in light of the recent cost projections and current shortfall of GET surcharge receipts. More specifically, updating the financial plan was not identified until April 2014. HART delayed communicating the potential cost increases to the city council until March 2015 after HART’s CEO announced a $910 million project deficit to the board in December 2014. The $910 million project deficit consisted of $600 million in increased costs, a $210 million reallocation of federal FTA Section 53075 funding to TheBus operations, and a $100 million GET county surcharge revenue shortfall. While there were indicators that led to the project deficit, project managers and staff in key positions stated they were unaware of HART’s fiscal condition until the December 2014 public announcement. As a result, corrective actions were not taken to ensure the FTA financial management and capacity concerns were satisfied. The United States Code, Chapter 53 of Title 49, requires the FTA to perform reviews and evaluations of Urbanized Area Formula Grant activities at least every three years. The site visit to the city occurred February 2 through 5, 2015. The final report was issued on April 9, 2015. As a corrective action to the finding, the FTA requested an updated financial plan by July 13, 2015. The plan should identify all funding sources for funding the HART project through completion within the FFGA scope and budget. HART submitted a draft financial plan to the FTA on August 14, 2015. The FTA closed the outstanding finding under the triennial review cycle on October 20, 2015. According to HART, the financial plan is being revised in light of the GET surcharge extension as of April 2016. 5 Federal Section 5307 (49 U.S.C. § 5307) is a formula grant program for urbanized areas that provides capital, operating, and planning assistance for mass transportation. This program was initiated by the Surface Transportation Act of 1982 and became FTA’s primary transit assistance program. The federal funds are apportioned to urbanized areas utilizing a formula based on population, population density, and other factors associated with transit service and ridership. Section 5307 is funded from both federal general revenues and trust funds, and is available for transit improvements for urbanized areas. 4 11 Chapter 2: HART Needs to Improve Financial Management and Planning At the October 2015 board meeting, HART reported the project cost overrun had increased to $1.3 billion. The cost overruns are detailed in the exhibit below. Exhibit 2.4 Project Cost Estimates (Dollars in Thousands) (As of June 2012, December 2014, and October 2015*) Description Project Capital Costs Original Estimates Cost Estimates June 2012 FFGA Financial Plan December 2014 HART October 2015 HART Board Meeting Board Meeting Change from Original Estimates $4,949,000 $4,948,000 $4,948,000 Lawsuits and Delay Claims -- 190,000 190,000 Utility Relocations -- 50,000 120,000 Project Enhancements -- 75,000 130,000 Cost Escalation -- 45,000 240,000 Allocated Contngency -- 240,000 240,000 Unallocated Contingency -- -- 299,000 $4,949,000 $5,548,000 $6,167,000 Total Project Capital Costs Debt Financing Costs Total Project Costs** 215,000 215,000 310,000 $5,163,000 $5,763,000 $6,477,000 $ % $1,218,000 25% $95,000 44% $1,314,000 25% (*) This table excludes revenue shortfall of $210 million in New Starts Fund reduction and shortfall of $100 million in general excise tax (GET) county surcharge receipts. (**) Numbers may not add up due to rounding. Source: OCA analysis based on the FFGA financial plan, June 2012; HART Project Risks Update, December 18, 2014 and Project Cost Update, October 15, 2015. Our analysis indicates the latest cost overrun figures are not reliable and will likely increase because the HART financial data and plans have not been updated to reflect the changes in the project costs. 12 Chapter 2: HART Needs to Improve Financial Management and Planning Reliable and consistent project cost information is lacking Reporting inconsistent information can cause confusion for policy makers and the public. During our review of HART reports, we found reports that contained missing, outdated, and incomplete information. According to HART, not all contract information is populated in the HART contract management system (CMS). As a result, HART management and staff relied on CMS reports6 that were missing project cost information. For example, we found: • Contract numbers in different reports contained different amounts. For example, in Exhibit 2.5, the contract balances in four different reports ranged from $2.6 million to $3.5 million. • CMS data was inaccurate. For example, HART executed a $100,000 professional services contract, but the expenditure report we reviewed indicated HART paid over $146,000 under the contract. HART staff later confirmed that there was a CMS error which excluded two contract amendments that totaled $250,000 from the CMS report. • Invoice data was incomplete. For instance, we found a missing invoice for $11,344 was not properly uploaded into CMS. • Delay claim data was incomplete. More specifically, delay claims totaling nearly $64.2 million were not reported. Most notably, the $8.7 million delay claim7 filed by Ansaldo Honolulu Joint Venture8 (eventually settled in October 2015) was not reported. Other delay claim data was outdated and not updated for two months. (See Exhibit 2.6, pending and possible changes). HART information system staff reported that the CMS is a virtual, real time system, and reports extracted from the CMS will never be the same because the database contains real time changes. HART does not have any policies or practices to ensure HART data are consistent in all reports or that data are reconciled. 7 According to HART, the $8.7 million delay claim filed by Ansaldo Honolulu Joint Venture (See Exhibit 2.6, Core Systems Design Build O&M Contract pending claim) was included in the $10-$20 million escalation costs increase (See Exhibit 2.7). However, HART was unable to provide details to support the $10-$20 million cost estimate. 8 AnsaldoBreda and Ansaldo STS became a part of the Hitachi Group Company on November 2, 2015 and November 3, 2015, respectively. AnsaldoBreda is now Hitachi Rail Italy. 6 13 Chapter 2: HART Needs to Improve Financial Management and Planning • The number of executed and pending utility agreements were inconsistent and unreconciled. More specifically, the CMS report showed a total of 48 utility contracts. In contrast, the tracking spreadsheet identified 54 contracts (including 40 active utility contracts, 7 closed contracts, and 7 pending contracts). • Six utility agreements were missing in a key report. Six pending contracts valued at $107.9 million were excluded from the HART CMS forecast report. • Differences in the contingency balance did not match external PMOC monthly reports. HART balances were higher than the figures reported by the PMOC. Appendix D compares the differences between HART’s contingency balances with the figures reported by the PMOC. The differences between the HART balances and the PMOC balances ranged from $149 million to $254 million. • State of Hawai`i GET balances did not match PMOC monthly reports. Appendix E compares the differences between the actual quarterly GET receipts with amounts reported by the PMOC. Although there may have been a timing difference between collection and reporting, the variances we found were significant. Reporting differences between the HART GET receipts and the PMOC reports ranged from $25 million to $492 million. Exhibit 2.5 Contract Balance Comparison (Dollars in Millions) List of Awarded Contracts Summary as of June 22, 2015 Description Total Contract Award Total Executed Changes Total Contract Amount Run Date: 8/13/15 $3,506 391 $3,897 Run Date: 11/5/2015 $3,496 356 $3,852 Forecast Report for June 2015 $2,620 376 $2,995 HART Facts as of June 30, 2015 $3,083 N/A $3,083 Source: OCA analysis of HART’s List of Awarded Contracts Summary as of June 22, 2015, Forecast Report for June 2015, and HART Facts as of June 30, 2015. 14 Chapter 2: HART Needs to Improve Financial Management and Planning Exhibit 2.6 Incomplete Delay Claims Summary (Dollars in Thousands) Contract West O`ahu/Farrington Highway (WOFH) Guideway Design-Build (DB) Contract WOFH Guideway DB Contract WOFH Guideway DB Contract Maintenance & Storage Facility DB Description Delay of Notice to Proceed (NTP) 2, 3, and 4 – CMC Escalation LCC Delay-Time Related Overhead Utility Delays Archaeological Inventory Survey (AIS) Suspension Part 2 Delay to Method Shaft 6 Kamehameha Highway Guideway (KHG) DB Contract KHG Guideway DB Contract Delay, Right-of-Way (ROW) KHG Guideway DB Contract Escalation Due to Schedule Impacts Airport Section Utilities Construction Time Impact Delay – Navy ROW Design-Bid-Build Contract Core Systems DB O&M Contract 9-Month Delay Claim Farrington Stations Group Farrington Stations Group NTP 1B Delay – Duration, Station Module Design NTP 2 Delay – Project & Interface Management Costs Total* Executed Claims Pending Changes Possible Changes $6,228 $8,000 5,275 3,000 121 25,000 4,500 802 $8,700 2,207 400 $11,957 $8,700 $43,577 (*) Numbers may not add up due to rounding. Source: OCA analysis based on HART Project Delays Cost Summary Spreadsheet, August 28, 2015, June 26, 2015, January 31, 2015, and December 18, 2014; CMS forecast report, September 25, 2015. Project Cost Estimates Lacked Supporting Documentation Cost estimating, by nature, is imprecise. Therefore, it is important to develop cost estimating methodologies and document key assumptions for the estimates. The Guide to the Project Management Body of Knowledge (PMBOK Guide) states that supporting documentation should provide a clear and complete understanding of how the cost estimate was derived. Supporting detail should include: • A description of the project’s scope of work; • Documentation of the basis of the estimate (i.e., how it was developed); • Documentation of all assumptions made; • Documentation of any known constraints; and • An indication of the range of possible estimates. 15 Chapter 2: HART Needs to Improve Financial Management and Planning We did not find documentary evidence to support $450 million of the additional cost increases as it relates to the above basic requirements. HART cannot demonstrate it has an effective method of tracking, monitoring, and reporting on delay claim costs In December 2014, HART stated delay claims totaled $190 million and were attributed to lawsuits, escalation costs, and other delays. Of the $190 million, $146 million were executed through change orders funded by the project contingency reserves. We found that the remaining costs of approximately $44 million were either unsupported or the claim amount changed because HART does not have an effective method in tracking, monitoring, and reporting on delay claim costs. Exhibit 2.7 quantifies the delay claim costs. Exhibit 2.7 Reporting Comparison of Delay Claim Costs December 2014 and October 2015 HART HART Board Project Delays Cost Meetings Summary $45,902,918 $39,039,250 Total Legal Delay Costs $77,126,198 $77,126,198 Total Other Delay Costs $49,106,403 $30,078,065 Escalation (Combination of Notice to Proceed & Legal) Total Delay Claims Cost $172,135,519 $146,243,513 Plus Several Remaining Claims and Escalation Actuals Estimated Total Impact Up To $10-20 million $190 million $0 $146,243,513 Difference ($6,863,668) $0 ($19,028,338) ($25,892,006) $10-20 million $44 million Source: OCA analysis based on HART Project Risks Update, December 18, 2014; Project Cost Update, October 15, 2015; and HART Project Delays Cost Summary, August 2015 In the December 2014 board meeting, HART provided a onepage Project Delays Cost Summary (spreadsheet) in support of the $190 million in delay claim costs. As of October 2015, HART’s spreadsheet remained unchanged. However, we identified changes that were not updated because HART does not separately track and monitor delay claim costs. By updating the claim information, we found: • $12.2 million in additional potential delay costs related to the West O`ahu/Farrington Highway Guideway DesignBuild contract9 that increased the delay claim from $6.8 million to $19 million. West Oahu/Farrington Highway Guideway Design-Build contract (Delay of NTP 2, 3, & 4 – Escalation Costs) 9 16 Chapter 2: HART Needs to Improve Financial Management and Planning • $1 million in additional potential delay costs related to the Kamehameha Highway Guideway Design Build contract (Escalation Due to Schedule Impacts) that increased the delay claim from $3.5 million to $4.5 million. • $825,000 in overstated delay claims related to the CMS forecast report. The report removed an $825,000 delay escalation claim for the maintenance and storage facility, but the update was not reflected in the tracking spreadsheet and not entered for two months. • $670,184 in overstated delay claims in the tracking spreadsheet. The tracking spreadsheet identified a $7.5 million delay escalation claim for the West O`ahu Farrington Highway Guideway Design-Build contract, but the forecast report showed $6,829,816. • We also did not find details to support litigation costs of approximately $6.9 million because HART tracks them separately as ineligible project costs. Despite our finding, the manager of Project Controls stated that the spreadsheet was never intended to be used as a management tracking tool. The Project Controls manager also stated that HART tracks and monitors delay claim costs using CMS and C2HERPS. HART provided CMS forecast reports to show how delay claims are tracked and monitored. We found that these reports are inclusive of all contracts and change orders. In one report, we identified over 490 line items that consisted of contracts with executed change orders; pending, probable, and possible changes; and claims in dispute. We question how HART can accurately identify, track, monitor, and report on total delay claim costs when these reports do not track them as separate costs. HART lacked adequate support for $120 million in utility costs Utility relocation costs were not included in the $910 million project deficit reported by HART in December 2014. In October 2015, however, HART provided project cost updates that showed utility costs increased from $50 million to $120 million. See Exhibit 2.8. 17 Chapter 2: HART Needs to Improve Financial Management and Planning Exhibit 2.8 Utility Costs Increase (Dollars in Millions) Cost Estimates Utility Work Description Additional Clearance Conflicts with HEI High Voltage Lines HEI Utility Relocations in the City Center Segment Service Connection for Permanent Power Total December 2014 HART Board Meeting $20 25 5 $50 October 2015 HART Board Meeting $88 25 7 $120 Change $ $68 0 2 $70 % 77% 0% 29% 58% Source: OCA analysis based on HART Project Risks Update, December 18, 2014 and Project Cost Update, October 15, 2015. HART explained that they used the Rough Order of Magnitude (ROM) 10 methodology to value the utility cost estimates. Although projects in the early phase generally have limited information to produce quality cost estimates, at a minimum, we expected a description of the estimate, scope and assumptions, data sources, estimating methodology and rationale, risk analysis results, and a conclusion about whether the cost estimate was reasonable. When we requested supporting documentation for how HART reached the total cost estimates, we were told that detailed estimates were still being developed and were not available for review. HART eventually provided a draft three-page document that listed five options that totaled $99 million. The cost estimates did not provide detailed documentation describing how it was derived; showed no evidence of any review or approval; and did not identify the factors used to estimate the $120 million in utility costs. As a result, we question the credibility of the estimate. According to HART’s Cost Estimating Procedure, a Rough Order of Magnitude Estimate (ROM) is an estimate developed to facilitate project budget and feasibility determinations. The order of magnitude estimate information is based on parametric units (e.g. route feet, lane miles, gross square feet, number of parking stalls) and other quantifiable data. Pricing is based on historical cost caps that are adjusted for project location, size or capacity differences, and cost escalations. The GAO Cost Estimating and Assessment Guide states that a ROM is developed when a quick estimate is needed and few details are available. Based on historical information, it is typically developed to support what-if-analyses, and can be developed for a particular phase or portion of an estimate to the entire cost estimate. The analysis is helpful for examining differences in alternatives to see which are the most feasible. Because it is developed from limited data and in a short time, a ROM analysis should never be considered a budget-quality cost estimate. 10 18 Chapter 2: HART Needs to Improve Financial Management and Planning We subsequently discovered the project manager responsible for the utility contracts did not know about the $120 million cost increase. $46 million in estimated cost overruns for project enhancements were not supported Like the utility relocation costs, project enhancements were not discussed when HART reported the $910 million cost overrun and revenue shortfall in December 2014. In October 2015, however, HART reported project enhancements costs that increased from $75 million to $130 million. The additional $55 million cost estimate increase included $35 million in public highway improvements and $20 million in additional escalators. The HART project controls manager told us that project enhancements were created to present change orders differently to stakeholders and the public. (See Exhibit 2.9 below.) Exhibit 2.9 Project Enhancement Costs Increase (Dollars in Millions) Description Platform Safety Gates Additional Seats Fare Collection Systems Emergency Backup Generators Public Highway Improvements Additional Escalators Total Project Status Executed Change Order Executed Change Order Solicitation Solicitation Preparation Change Orders Preparation Solicitation Preparation Cost Estimates December 2014 October 2015 HART Board HART Board Meeting Meeting $28 $28 2 2 15 15 15 15 15 50 0 20 $75 $130 Change $ ----35 20 $55 % ----233% 100% 73% Source: OCA analysis based on HART Project Risks Update, December 18, 2014; Project Cost Update, October 15, 2015; and CMS forecast report, September 2015. We found that HART could not support $46 million of the $130 million in project enhancement cost estimates. Discrepancies existed between the cost estimates presented to the HART board and the documentation supporting the estimates because HART did not follow its cost estimating procedures. For example, we did not find detailed descriptions or support for: • $18.1 million for public highway improvements • $5.2 million for the emergency backup generators • $2.7 million for fare collection 19 Chapter 2: HART Needs to Improve Financial Management and Planning We questioned the estimated $20 million for additional escalators. HART provided a handwritten proposed costs document that showed two estimates which varied by roughly $8 million with a low of $17 million and a high of $25 million. Higher estimates can overstate the total project costs while lower estimates can potentially result in cost overruns. According to the GAO Cost Estimating and Assessment Guide, it is imperative that all assumptions are documented so that management fully understands the conditions the cost estimate was structured on. The GAO further states that failing to do so can lead to overly optimistic assumptions that heavily influence the overall cost estimate, to cost overruns, and to inaccurate estimates and budgets. While cost estimates are only estimates, absent detailed information, we were unable to identify all the cost elements included in the total amount. More importantly, we could not assess the reasonableness and appropriateness of the methodology and assumptions used to develop some of the cost estimates. Escalation cost11 estimates were not calculated by using a specific factor HART increased its escalation cost estimates by $195 million, from $45 million in December 2014 to $240 million in October 2015. HART attributed the increase to extraordinary market conditions. During the audit, HART staff was unable to explain the methodology used to support these cost estimates. We were told that HART did not use a specific factor to calculate the $240 million in escalation costs and that these costs are subsets of the total escalation within the project costs. HART claims that it has procedures and methodologies in place to forecast escalation that is based upon historical data as well as using the latest bid results, other agency procurement results, market studies, and independent economic reports, etc. When we asked for evidence to substantiate the $240 million escalation cost estimates, however, there was no documentation to support how the $240 million was derived. HART was also unable to provide a detailed breakdown of escalation costs for the total $6.5 billion project cost. 11 20 Escalation costs represent cost increases projected by a contractor or HART when estimating work to be completed at a time in the future. Chapter 2: HART Needs to Improve Financial Management and Planning Consequently, we were unable to verify HART’s methodology to forecast escalation costs and the reasonableness of the additional escalation costs because the amounts were not calculated by using a consistent and specific factor. The exhibit below shows the unsupported cost increase. Exhibit 2.10 Escalation Costs Increase (Dollars in Millions) Cost Estimates Description Change December 2014 HART October 2015 HART Board Meeting Board Meeting Cost Escalation $45 $240 $ % $195 433% Source: OCA analysis based on HART Project Risk Update, December 18, 2014; and Project Cost Update, October 15, 2015. HART underreported GET county surcharge forecasts In 2014, HART projected a GET revenue shortfall in the range of $80 to $100 million and attributed the shortfall to a coding error by the State of Hawai`i Department of Taxation. The error resulted in a $9.9 million distribution error which was compounded annually over 10 years for a total of $100 million. We requested information related to the $100 million GET revenue shortfall and analyzed the spreadsheets provided by HART. The HART spreadsheets showed how GET county surcharge receipts were tracked and how revenues were projected. We found that HART’s revenue forecasts were higher than reported. Consequently, HART underreported its projected shortfall amount by approximately $41 million. HART executive management knew the amount was higher than the $100 million reported, but did not report its projections accurately to the board in December 2014. Improved Financial Management and Planning Are Needed While the FTA does not require submittal of updated financial and operating plans after the award of a full funding grant agreement, it does retain the right to ask for updated financial and operating plans if any significant changes to the project occur after the funding grant agreement is signed. 21 Chapter 2: HART Needs to Improve Financial Management and Planning According to the FTA Guidance for Transit Financial Plans, sound financial planning ensures the financial health of transit agencies and affects the quality of service provided. Financial and operating plans serve as a fundamental tool for management and policy makers to make critical decisions, especially for a project of this magnitude. Consistent with best practices, the plans should therefore be regularly updated to reflect the most current financial condition of the project. A HART executive stated that the outdated plans are not an impediment to HART operations. According to the executive, HART is fulfilling its reporting requirements; updating the financial and operating plans are contingent upon the passage of the GET county surcharge extension12 by the city; and that FTA has not established a specific timeframe requirement for the financial updates. Another HART executive stated a formal update requires HART to go through a lengthy process that is subject to the review and approval of key stakeholders, including the HART Board of Directors, the project management oversight consultant, and the FTA.13 We believe this reasoning should not delay HART’s efforts to update its financial and operating plans. Without current financial and operating plans, HART management, policy makers, and decision makers will be unable to make cost-effective decisions to ensure the project is completed efficiently, effectively, and economically. At the time of our interview, the GET county surcharge extension was still subject to the adoption of bill 23 by the city council. 13 According to HART, the FTA instructed the agency to submit a revised financial plan because of a deficiency related to the lack of projected funds. In order to revise the plan, HART noted that it sought to extend the GET surcharge as a viable finding source. HART also noted that the FTA held off any further action until HART could demonstrate that it had the financial capacity to complete the project. As a result of the GET extension, the FTA has scheduled a full budget review of HART’s updated financial plan at the next risk refresh meeting on March 30, 2016. 12 22 Chapter 2: HART Needs to Improve Financial Management and Planning Recommendations HART should: 1. Increase efforts to regularly update its financial plan. The cost changes and adjustments are necessary to reflect the current financial condition of the project. Updates should be supported by detailed, source documentation. 2. Update its Operations and Maintenance Plan (OMP) to address funding, management, and other transit needs. 3. Consistently and accurately report on project cost information, identify and explain variances if internal and external reports are intended to be different so that policy makers and the public receive consistent and reliable project cost information. 4. Develop methods to ensure data used in HART, PMOC, and other reports are consistent, accurate, reliable, and can be reconciled among all the reports using the data. 5. Develop a process for tracking and monitoring all costs, including the status of delay claim costs. 6. Support its cost estimates with consistent, reliable and sufficient information. To do so, HART should thoroughly document details, including any forecasting methodology and assumptions made to support its cost estimates. 7. Replace the contract management system (CMS) with a system that is more user friendly and more appropriate to managing the HART construction project. If the CMS system is retained, HART should define which CMS data elements, data fields, and functions should be used and which parts should be deactivated or eliminated. 8. Use the city’s C2HERPS enterprise resource planning system to develop, monitor, track, and report budget, financial, and accounting data. The CMS system should not be used for these purposes. 9. Develop a forecasting model to best predict escalation costs and support it with documentation. 23 Chapter 2: HART Needs to Improve Financial Management and Planning This page intentionally left blank. 24 Chapter 3: HART Needs to Improve Project Management and Contract Administration Chapter 3 HART Needs to Improve Project Management and Contract Administration Summary Project management and contract administration controls can be improved. More specifically, Honolulu Authority for Rapid Transportation’s (HART) Project Management Plan (PMP) and its subsidiary plans are outdated and unreliable as decisionmaking tools. HART made concessions to a single contractor; did not perform quantitative analysis to justify a major decision to repackage a bid for nine stations in the Westside Stations Group; and paid $1.5 million in stipends to unsuccessful bidders without knowing the bidders’ actual costs. HART is also paying for vacant office space. Contract administration controls need to address invoice payments, procurement file documentation, and prevent improper payments. Background On December 18, 2014, HART’s chief executive officer (CEO) reported to the HART Board of Directors that the total project costs will increase to $5.8 billion, an increase of $600 million in additional costs and a $310 million funding shortfall. HART attributed project cost increases to three separate events: lawsuits that resulted in delay claims, higher than expected bid for the construction of the nine Westside stations, and the unfavorable general excise tax (GET) county surcharge revenue receipts. Ten months later, on October 15, 2015, total project costs increased to $6.5 billion. Exhibit 3.1 provides details. 25 Chapter 3: HART Needs to Improve Project Management and Contract Administration Exhibit 3.1 Factors that Contributed to the Project Cost Increase and Revenue Shortfall (Dollars in Thousands) Original Estimates June 2012 FFGA Financial Plan Description Project Capital Costs $4,949,000 Estimated Costs Increase: Lawsuits and Delay Claims Utility Relocations Project Enhancements Cost Escalation Allocated Contingency Unallocated Contingency Debt Financing Costs Total Estimated Costs Increase* Estimated Revenue Shortfall: GET Projected Shortfall FTA 5307 Fund Reduction Total Estimated Revenue Shortfall ------- 215,000 $5,163,000 --- $0 Cost Estimates December 2014 HART Board October 2015 HART Meeting Board Meeting $4,948,000 $4,948,000 $190,000 50,000 75,000 45,000 240,000 215,000 $5,763,000 $190,000 120,000 130,000 240,000 240,000 299,000 310,000 $6,477,000 $100,000 210,000 $310,000 $100,000 210,000 $310,000 -- (*) Numbers may not add up due to rounding Source: Office of the City Auditor (OCA) analysis based on HART Project Risks Update, December 18, 2014 and Project Cost Update, October 15, 2015. Construction Cost Overruns Are Not Unusual Our comparison of construction costs for other capital rail projects indicated cost overruns frequently occur. The following data table features a comparison of capital costs among 20 heavy and light rail projects. • The first 8 services (Atlanta, Baltimore, Buffalo, Miami, Pittsburgh, Portland, Sacramento, and Washington, D.C.) were rail projects that initially established heavy and light rail services in the respective urban areas. While one project was completed under budget, the others had cost overruns that ranged from 13 percent to 83.1 percent. • The remaining 12 heavy and light rail projects were extensions and built to integrate the new projects into already established heavy rail transit services. One heavy rail project was completed under budget. The other 11 projects had cost overruns that ranged from 3.3 percent to 35.8 percent. 26 Chapter 3: HART Needs to Improve Project Management and Contract Administration Exhibit 3.2 Capital Costs Comparison of Heavy and Light Rail Projects Project Year Project Completed Atlanta Baltimore Miami Washington, D.C. 1979 1983 1984 1976 Buffalo Pittsburgh Portland Sacramento 1984 1984 1986 1987 Baltimore-Johns Hopkins Chicago-SW Transitway San Francisco-Colma 1994 1993 1996 Baltimore-BWI Hunt Valley Portland-Westside Hillsboro St. Louis-MetroLink 1997 1998 1993 Chicago-Douglas Branch San Francisco-SFO Washington-Largo 2005 2003 2004 Minneapolis-Hiawatha Pittsburgh-Stage II Reconstruction San Diego-Mission Valley East 2004 Total Capital Cost (dollars in millions)* Original FFGA At Completion 1990 FTA Report** Heavy Rail $1,723 $2,720 $804 $1,289 $1,008 $1,341 $4,352 $7,968 Light Rail $478 $722 $699 $622 $172 $266 $165 $188 2003 FTA Report Heavy Rail $311 $353 $438 $522 $172 $180 Light Rail $110 $116 $887 $964 $456 $464 2007 FTA Report Heavy Rail $473 $441 $1,186 $1,552 $413 $426 Light Rail $513 $697 Cost Overruns (%) 57.9% 60.3% 33.0% 83.1% 51.0% -11.0% 55.0% 13.0% 13.7% 19.1% 4.9% 6.2% 8.7% 1.8% -6.8% 30.9% 3.3% 35.8% 2004 $363 $385 6.0% 2005 $427 $506 18.7% (*) All dollar amounts shown represent adjusted real values. (**) The FFGA program was established after the projects featured in the 1990 study. Source: Office of the City Auditor (OCA) analysis based on information obtained from the Urban Transit Rail Projects: Forecast Versus Actual Ridership and Cost, 1990, prepared by Dr. Don H. Pickrell for the Urban Mass Transit Administration and the FTA’s Predicted and Actual Impacts of New Starts Projects, 2003 and 2007. Additional HART Rail Project Cost Overruns and Shortfalls Are Likely One of HART’s stated goals is to preserve the stewardship of resources by maintaining public trust through the prudent and transparent use of financial, human, and environmental resources. HART can demonstrate good public stewardship by wellsupporting its management decisions. In addition to deficiencies in financial management and planning, we found shortcomings that could impact the project’s longterm financial viability, increase the likelihood of additional 27 Chapter 3: HART Needs to Improve Project Management and Contract Administration cost overruns, and reduce the funds available to the rail project. Specifically, we found: • HART does not have an adequate contingency reserve. • Cost controls were insufficient to control cost increases. • Updating the project management plan, including the subsidiary plans, was never made a priority. • Inappropriate concession (retainage payments) with a contractor was made. • Cost estimates and potential savings were lacking in the decision to repackage the Westside Stations Group. • HART lacks policy and procedures on administering stipends and has paid $1.5 million without documentation of unsuccessful bidders’ actual costs. • HART is paying for vacant office space. • Contract administration needs to be improved. The project management oversight consultant (PMOC) meets with HART management and staff monthly to discuss increased costs and to ensure that cost issues are proactively addressed. In December 2014, the PMOC reported it provided numerous cost mitigation recommendations that HART should consider implementing. Despite the recommendations, PMOC noted that minimal cost containment measures had been accomplished by HART and the trend of minimal cost containment was alarming. 28 Chapter 3: HART Needs to Improve Project Management and Contract Administration HART does not have an adequate contingency reserve Normally, the amount of contingency required for a project decreases with the project’s progress. Over time, as costs become more definitive, the contingency amount should decrease. However, HART increased its allocated and unallocated1 contingency funds by $539.4 million because it does not have an adequate contingency reserve. • In June 2012, HART reduced its original $866 million contingency by $222 million to $644 million after it submitted its 2012 Full Funding Grant Agreement (FFGA) financial plan to the Federal Transit Administration (FTA). • In July 2012, HART established a $76.1 million Known Changes2 contingency account to separately fund costs that would have been covered by unallocated contingency reserves. • In October 2012, the PMOC noted that “significant contingency reduction occurred, to a point where contingency was below accepted control levels” (vs. HART’s submittal of the updated cost estimate in 2012 to support the FFGA application). Subsequently, the PMOC acknowledged that HART “has implemented efforts to recover contingency levels through cost reduction measures, value engineering, and revised project delivery strategies.” • From May 2013 through July 2013, the PMOC expressed concerns “with the adequacy of the remaining contingency given the anticipated costs due to the project delays.” In August 2013, the PMOC noted a concern on “whether there is sufficient contingency remaining, given the status of the project.” Unallocated contingency provides a funding source to cover unknown but anticipated additional project execution costs and uncertainty due to risk factors such as unresolved design issues, market fluctuations, unanticipated site conditions and change orders. It also covers unforeseen expenses and variances between estimates and actual costs. 2 HART tracks Known Changes separately from the Project contingency established under the FFGA. Known Changes are executed through budget transfers. According to HART, Known Changes are recognized as project scope and not contingency. HART management explained that the Known Changes were identified as pending changes that were subject to final negotiations with contractors. Upon our review of Known Changes, we found that HART used these reserves to fund change orders that included a $20.1 million delay claim, $6.8 million in non-rail escalation and rail mark-up costs, and $2.6 million budget transfer into allocated contingency, contrary to the intended use of this fund. 1 29 Chapter 3: HART Needs to Improve Project Management and Contract Administration • In the September 2014 PMOC monthly report, the PMOC recommended that “strong controls must be put in place immediately to avoid future rapid contingency reductions.” It also added “the frequency and the levels of project management to which these statistics are reported should be improved and monitored monthly.” • In the December 2015 PMOC monthly report, the PMOC estimates $303 million remaining in total contingency. HART’s forecast report, as of December 2015, identifies $330 million in executed change orders and projects $301 million in pending, probable, and possible changes. HART did not communicate the need for additional contingency until December 2014. In December 2014, HART increased the underfunded contingency reserves to $884 million. In October 2015, HART increased its reserves to $1.18 billion. The HART contingency increases from $644 million (13 percent) in June 2012 to $884 million (15.9 percent) in December 2014 to $1.18 billion (19.2 percent) in October 2015 were part of the reported project shortfalls (See Exhibit 3.3). The last increase of $539.4 million (allocated and unallocated) was more than the allocated contingency under the final FFGA, and may have been excessive. 30 Chapter 3: HART Needs to Improve Project Management and Contract Administration Exhibit 3.3 Contingency Reserves Increase (Dollars in Thousands) Original Estimates Description Allocated Contingency Unallocated Contingency Allowance Total Contingency* June 2012 FFGA Financial Plan $540,100 $101,900 $1,600 $643,600 Contingency Percentage of Total Project Capital Costs 13.0% Cost Estimates December 2014 October 2015 HART Board HART Board Meeting Meeting $782,000 $782,000 $102,000 $401,000 --$884,000 $1,183,000 15.9% Change from Original Estimates $ $539,400 % 84% 19.2% (*) Numbers may not add up due to rounding Source: OCA analysis based on the FFGA financial plan, June 2012; HART Project Risks Update, December 18, 2014; and Project Cost Update, October 15, 2015; PMOC Monthly Report, May 2013. Cost controls were insufficient to control cost increases We found deficiencies related to HART’s cost controls3 that, in our opinion, partly contributed to the significant cost increases. In March 2014, the PMOC performed an initial review of information provided by HART and suggested a possible recommendation for HART to develop aggressive cost containment measures. In December 2014, HART announced the $910 million project cost overrun and revenue shortfall. The PMOC reported that, in February 2015, HART started implementing some cost containment measures. Although HART claims that evaluating and developing cost containment opportunities and cost reduction strategies were ongoing activities, we believe that HART could have taken a more proactive approach in implementing cost containment measures. Instead, HART reacted by requesting more funding. According to the PMBOK guide, control costs is the process of monitoring the status of the project, to update the project budget, and managing changes to the cost baseline. 3 31 Chapter 3: HART Needs to Improve Project Management and Contract Administration We also found that not all project managers compare actual costs against their budgets even though this comparison is an important cost control mechanism. One project manager said that she relies on project controls to monitor her contract costs. The project is at risk of additional cost overruns when project managers do not pay attention to project costs and budgets. In our opinion, project managers should continuously compare actual costs against budget amounts, should analyze any variances, and take corrective actions before costs go higher than expected. Updating the project management plan, including subsidiary plans, was never made a priority A Project Management Plan (PMP) is a formal, approved document that guides how a project is to be executed, monitored, and controlled. It includes subsidiary plans that provide guidance and direction for cost management planning and control. According to the FTA Project and Construction Management Guidelines, a PMP is required by statute for major capital projects, provides a functional, financial, and procedural route map for the grantee to effectively and efficiently manage on-time, withinbudget, and at the highest quality level in its unique project environment. The FTA requires the PMP to be submitted prior to preliminary engineering and updated through subsequent project phases. Moreover, best practice recommends continuous updates to the PMP because it can provide greater precision with respect to schedule, costs, and resource requirements to meet the defined project scope. We found that HART has not updated its PMP, including the subsidiary plans, despite considerable changes in project schedule, costs, and staffing since the federal FFGA was issued by the FTA in December 2012. Updating the PMP and its subsidiary plans4 has not been a high priority for HART. According to the PMOC, the PMP update has been in progress prior to March 2013. The PMOC indicated that it was critical for HART to update the PMP and its subsidiary plans. We believe that an outdated PMP and its subsidiary plans could hinder management’s ability to effectively guide the project to completion in an economical, effective, and efficient manner. Subsidiary plans include the quality management plan; real estate acquisition and management plan; bus fleet management plan; rail fleet management plan; safety and security management plan; safety and security certification plan; configuration management plan; staffing and succession plan; risk and contingency management plan; operating plan; force account plan; mitigation monitoring program; interface management plan; contract packaging plan; claims avoidance plan; construction management plan; contract resident engineer manuals; and project procedures. 4 32 Chapter 3: HART Needs to Improve Project Management and Contract Administration Exhibit 3.4 Key Management Plans Plan Contract Packaging Plan (CPP) Financial Plan Description The CPP describes each third party contract which will be undertaken by HART to implement the Project. This document provides a summary of the capital costs and funding sources associated with both the Project and the city's ongoing capital needs for its existing public transportation system, including the results of three sensitivity analyses and potential mitigation strategies. It also describes the city's plan to fund the operations and maintenance costs associated with the Project, TheBus, and TheHandi-Van services. Master Project Schedule (MPS) The MPS is the primary schedule developed by the Project team which includes and coordinates the work of the various project segments and contractors. Operations and Maintenance The OMP presents the capacity of the city to Plan (OMP) operate and maintain the integrated transit system. Project Management Plan The PMP establishes the framework for (PMP) administering implementation of the Project. It describes and documents the overall management approach for the Project and is used both as a management tool to guide HART and as an informational overview for project participants and interested parties. Risk and Contingency The RCMP describes the approach that the Management Plan (RCMP) project will adapt to identify risk, assign the lielihood of occurrence of each risk, and quantify the associated potential impact on project delivery objectives if it occurs. It provides senior management with a systematic process for identifying, assessing, evaluating, managing, and documenting risks that could jeopardize the success of the project. Orginal Issue or Last Date Identified as an Update Update Item Status as of December 2015 Approved for FFGA (2012) August 2014 Updating June 2012 August 2014 Updating Approved for FFGA (2012) August 2014 Updating June 2012 August 2014 Updating July 17, 2012 Not Identified Updating September 26, 2011 September 2013 Updating Source: HART Standard Terms Definitions and Acronyms, April 19, 2012; HART Monthly Progress Report, December 2015; and PMOC Monthly Report, December 2015. Waiver concessions were made Pursuant to the Hawai`i Revised Statutes §103-32.1, city contracts allow HART and the city to withhold up to 5 percent of a contract amount to ensure that the contractor’s performance is satisfactory and acceptable. Once the city is satisfied with the project or is satisfied with any re-work the contractor is asked to do, the city will release the retained amount (retainage) to the contractor. 33 Chapter 3: HART Needs to Improve Project Management and Contract Administration For one contract, the rail prime contractor, Kiewit, requested a partial release of the retained amount. On March 6, 2013, HART waived the five percent retainage for three construction contracts with Kiewit5. This concession was made for three specific change orders totaling $26 million and the total amount waived was about $1.3 million. Actual expenditures totaled $23 million, of which the total amount actually waived was $1.1 million. According to HART managers, they approved the waiver as an incentive for Kiewit to stay on the job. The approval reduced Kiewit costs during the period when the rail project work was suspended. The suspension occurred after an August 2012 Hawaii Supreme Court decision caused the project work to halt so archaeological surveys could be completed. In September 2013, project work restarted.6 By waiving the 5 percent retainage, HART and the city lost some leverage to ensure Kiewit performed satisfactorily under its contracts. The city also risked being accused of favoritism or bias towards one contractor. Cost estimates and potential savings were lacking in the decision to repackage the Westside Stations Group7 HART was unable to demonstrate that it prepared a reasonable level of cost-benefit analysis to justify its decision to repackage the Westside Stations Group prior to the rebid. Our review of the Westside Stations Group repackage strategy found that HART lacked sufficient documentation to quantify the expected costs and potential savings when the decision was made to cancel the original bid. Consequently, HART assumed significant risks that could have driven contract costs higher and made future savings unattainable. The contract documents indicate Kiewit requested a partial release of retainage on only one contract prior to HART issuing the approval memorandum for all 3 Kiewit contracts. 6 According to HART managers, the term waived was an error and the wording should have been “retention will not be withheld.” 7 The Westside Stations Group consists of the first nine stations along the rail route. HART repackaged the Westside Stations Group into three rail station groups: Farrington Highway Stations, West O`ahu Stations and Kamehameha Highway Stations. 5 34 Chapter 3: HART Needs to Improve Project Management and Contract Administration During our audit, we asked HART to provide cost estimates and to identify its expected savings from the repackage. After opening the packages of the three station bids, HART executive management claimed a potential cost savings of $31 million on the decision to divide the nine stations into three packages of three stations. According to HART, the sum of the independent cost estimates totaled $263 million, $31 million less than the original low three bid of $294 million for the nine station package. We found that HART lacked sufficient time to fully evaluate and quantify any potential savings. On September 9, 2014, HART’s CEO, who also serves as the chief procurement officer, authorized HART staff to cancel the nine stations group construction bid prior to the receipt of all independent cost estimates for the three rail station packages, including a $5.2 million contract for a H2R2 ramp that was originally part of the Kamehameha Highway Stations Group. See Exhibit 3.5 for the timeline. Exhibit 3.5 Westside Stations Group Bid Cancellation Timeline August 13, 2014 Original Bid Open Date September 9, 2014 Original Bid Cancellation Date 2014 2015 December 10, 2014 Independent Cost Estimate Date Farrington Highway Stations Group 2016 August 17, 2015 Independent Cost Estimate Date Kamehameha Highway Stations Group April 2, 2015 Independent Cost Estimate Date West O`ahu Stations Group Source: OCA analysis of independent cost estimates and HART Monthly Progress Report, November 2015. 35 Chapter 3: HART Needs to Improve Project Management and Contract Administration According to HART, its consultants performed quantitative analysis to support its decision to repackage the Westside Stations Group bids and provided two whitepapers to substantiate its cost-benefit analysis. While there were discussions of market factors and different procurement strategies, we did not find any quantitative analysis to show the potential costs or savings for the decision to rebid. One of the whitepapers included several pages on value engineering, scope reduction and scope transfer ideas. For the first three stations, a Rough Order of Magnitude (ROM) for cost savings were notated next to each idea. These ROM for cost savings were extrapolated to the remaining six stations. HART was unable to provide how the ROM figures were derived. The independent cost estimates for the three packages totaled $263 million which was $79 million higher than the engineer’s original estimates. Nevertheless, HART reported in its August 2014 Monthly Report that the overall cost of the $5.2 billion project would not change and that additional costs could be covered using a combination of contingency funds and adjusting the contract scope to reduce costs. Contrary to the August 2014 Monthly Report, HART announced in December 2014 that the cost for all the stations to be constructed and the remaining guideway will exceed the contingency reserves, off-sets, and other funds available by several hundreds of millions of dollars. Fortunately, the actual outcome reduced the original, nine station group bid by $31 million. If the strategy had failed, the decision could have driven contract costs higher than the original nine station bid. Because of the fiscal situation confronting the agency, the CEO told us that he had to make it work. He relied on his professional judgment and consultant’s opinions when he canceled the nine station bid and repackaged the contract into three rail station groups. While professional judgment is important, critical decisions should be supplemented by quantitative analysis and documented analysis, as well as past experience, and current or historical data. HART paid $1.5 million to unsuccessful bidders without documenting their actual costs 36 According to a HART officer, issuing stipends to unsuccessful bidders is a common practice, is allowed under state law, and is accepted by the Federal Transit Administration (FTA). According to the FTA, this practice gives the grantee title to the proposed Chapter 3: HART Needs to Improve Project Management and Contract Administration design concepts. The stipend payment covers all or part of the cost of preparing bid proposals and encourages competition although the practice is an added cost to the grantee. While there is no specific evidence that firms would not submit bids if no stipend was provided, HART offered $3.5 million to compensate unsuccessful bidders for three design-build contracts8. The stipends were to be divided equally and not to exceed $500,000 for each unsuccessful bidder. HART’s records show that $1.5 million9 was paid to three unsuccessful bidders as of June 30, 2015. Exhibit 3.6 Stipend Payments Date Contract West O`ahu/Farrington Highway Guideway West O`ahu/Farrington Highway Guideway Kamehameha Highway Guideway Total Stipend Payment Offered Amount 02/11/10 $1,000,000 $500,000 05/26/10 $1,500,000 $500,000 12/07/11 $1,000,000 $3,500,000 $500,000 $1,500,000 Source: OCA analysis based on C2HERPS data provided by the HART. We were, consequently, unable to assess whether the stipend payments were excessive or if the stipends covered the unsuccessful bidder’s actual costs because HART had not established any written policy or procedures related to administering the stipend payments. Effective policy and procedures provide staff guidance for issuing proposal stipends to unsuccessful bidders. Without any policies or procedures, it is unclear how HART was able to determine the compensation amount for each of the unsuccessful bidders; assess the documentation needed to support their actual costs; or determine the value or usefulness of the unsuccessful bidders’ proposals. West O`ahu/Farrington Highway Guideway, Kamehameha Highway Guideway, and Airport Guideway and Stations Design-Build Contracts. 9 According to HART, all 3 stipends were paid on city procured contracts. In connection with the Kamehameha Highway contract, the stipend payment was issued subsequent to the establishment of HART. According to a HART executive, HART plans to continue the practice of issuing stipend payments. 8 37 Chapter 3: HART Needs to Improve Project Management and Contract Administration HART leases four floors of office space. During our on-site visit at HART, we identified one leased floor of 16,182 rentable square feet that had 12 vacant offices and 15 empty workstations. About 41 percent of the offices and workstations were unoccupied on this floor when we conducted a physical inventory count in July 2015. HART is paying for vacant office space Although HART stated it has plans to occupy the empty space, the offices and workstations sat vacant throughout our six-month audit. While it was empty, we requested a current inventory listing of its leased offices and workstations. It took HART staff five weeks to provide us this information. Review of its inventory listing revealed that the vacancy rate increased to 44 percent. As of April 2016, HART reported a vacancy rate of 27 percent on the floor in question. Upon verification, we found that the vacancy rate is closer to 32 percent10. HART should evaluate and document its office space requirements and minimize HART operating costs by subletting its surplus office space or renegotiating its leases so that it only pays for space that is needed. Exhibit 3.7 Office and Workstation Vacancy Rates Count as of July 2015 Total Offices and Workstations Offices Workstations Total Count as of November 2015 Occupied Unoccupied Vacancy Rate Occupied Unoccupied Vacancy Rate 34 5 39 12 15 27 26% 75% 41% 34 3 37 12 17 29 26% 85% 44% 46 20 66 Source: OCA physical count of leased office space and HART Work Space Inventory Listing, November 30, 2015 As of April 2016, HART reported an aggregate occupancy rate of 88 percent. The aggregate amount included three floors that were 93 percent to 96 percent occupied and one floor that was 73 percent occupied. The audit discusses the 23rd floor that is currently 32 percent vacant. 10 38 Chapter 3: HART Needs to Improve Project Management and Contract Administration Exhibit 3.8 Photos of Vacant Offices and Workstations Source: OCA Contract Administration Needs to Be Improved Current and complete policies and procedures are necessary to provide clear and effective guidance to staff regarding contract management. Strong recordkeeping practices serve to ensure compliance with federal, state, and local laws. We sampled 25 contracts during our audit and found several deficiencies. We found that HART has not developed written policies related to contract administration and invoice payment practices, and procedures were incomplete and not regularly updated. These deficiencies could lead to noncompliant and questionable practices. 39 Chapter 3: HART Needs to Improve Project Management and Contract Administration Exhibit 3.9 Summary of Contract Types Reviewed Contract Type Firm Fixed Price Cost Reimbursement Time and Materials Legal Contract Lease Agreement Cost Plus Fixed Fee Combination Firm Fixed Price & Time and Materials Intergovernmental Agreement Task Order Time and Materials Total No of Contracts Reviewed 8 4 3 3 2 2 1 1 1 25 Source: OCA sample selection Contract administration policies and procedures were incomplete and outdated We found HART has not developed written policies related to contract administration and invoice payment practices. Policies and procedures were also needed for capital project monitoring and reporting. HART contract administration procedures were last updated in 2012 and need to be updated to reflect current policies and procedures. Out-of-date policies and procedures increase the risk of contract mismanagement if guidance is incorrect. Contract administration roles and responsibilities were confusing We found that HART’s contract administration procedures defined confusing roles and responsibilities for its project managers, contract managers, and contract administrators. HART’s project managers acted as contract managers until August 2015. In a separate updated procedure manual, HART delineated the positions into two distinct roles. In addition, the terms contract administrator and contract manager are used interchangeably at HART even though procedures define them differently. As a result, changes to contract administration procedures are confusing. Until August 2015, the project manager filled both the project manager and the contract manager roles. The dual role increased the risk of contract mismanagement between 2012 and 2015. Other contract file deficiencies 40 For the sample of 25 contract files, we found: • Contract files had missing documentation. Documentation deficiencies included no complete listings of contract modifications and supporting documentation in the files; Chapter 3: HART Needs to Improve Project Management and Contract Administration 8 files which grant HART access to only prime contractor records; no files with access to sub-contractor records; 3 contract files that did not include the Scope of Work11; and 2 contract files missing a Letter or Notice of Award. • Financial disclosures and conflicts of interest certifications were missing. We found no evidence to show that HART required financial disclosures for prime contractors or subcontractors; 7 contract files where the prime contractors did not certify they had no real or apparent conflicts of interest12, and no evidence of subcontractor’s conflicts of interest certifications. HART maintained subcontractors were required to file conflicts of interest statements as part of the solicitation process. • Excluded contractor checks were missing. Two contract files lacked evidence that an Excluded Parties List System (EPLS)/System for Award Management (SAM) check was performed prior to award to ensure the contractor was not suspended or debarred by the Federal government. These checks may have been made, but were not documented in the procurement files. Although HART maintains that EPLS/SAM checks were not put into routine practice at HART until 2012, our review of contract files suggests these checks were being performed as early as 2007. The compliance requirement with 2 CFR 180.300 was met because HART demonstrated that it had obtained certification from these 2 respective contractors regarding debarment, suspension and other ineligibility and voluntary exclusion from transactions financed in part by the U.S. Government. Subsequent to our review, HART presented Statements of Work for 2 contract files. HART noted that the third had been electronically filed, but would be included as a hard copy in the contract file. 12 Subsequent to our review, HART presented Conflict of Interest certifications for 4 contract files. HART contends that one contract was procured prior to its inception, and that the two remaining contracts were funded by HART, but were procured by Corporation Counsel. 11 41 Chapter 3: HART Needs to Improve Project Management and Contract Administration • No evidence that contract managers conducted performance reviews; 22 contract files did not have designated contract managers; 1 contract file lacked the independent evaluations and scores of the evaluation committee related to the contract award13; and 3 files had no evidence of a cost or price analysis by HART for the intended award14. A cost or price analysis should be performed for every contract so that the essential objective of a reasonable price is assured. Invoice payment procedures do not address all contract types We found that HART’s invoice payment procedures do not address all contract types, such as cost reimbursement, and time and materials contracts, because there were no defined policies and only one procedure15 related to contract payments. HART’s invoice payment procedure was developed in 2012 and was limited in scope to only Firm Fixed Price contracts which uses a Schedule of Milestones to determine monthly progress payments by milestone achieved. Because HART has a variety of contracts and invoices which are not paid based on milestones, we believe that this procedure is insufficient to fully support HART’s invoice payment process. Furthermore, we found that HART’s procedure had not been updated to reflect its current practices with respect to invoices. If practices for paying invoices for contracts and goods and services are not adequately conveyed in guidance, policies, and procedures, it can lead to invoices being paid in spite of insufficient support and questionable expenses. Invoice payments had unsupported and unallowable costs A strong invoice payment process prevents improper payments from being made. HART risks making improper payments when there is a lack of proper review and documentation to support the work or services billed. Our review of 50 HART invoices revealed instances of incomplete and improper payments or authorizations. Subsequent to our review, HART presented independent evaluations and scores of the evaluation committee related to this contract award. 14 Subsequent to our review, HART presented cost or price analysis for 2 contract files. HART produced a waiver for cost or price analysis being performed for the third contract; however, the waiver states that no prices had been submitted. No further analysis was performed. 15 Contractor Progress Payments, 5. CA-03, Rev. 1.0 - April 19, 2012. 13 42 Chapter 3: HART Needs to Improve Project Management and Contract Administration We also found unsupported and unallowable costs. Specifically, our review showed HART paid: • 3 invoices, valued at $8,670,112, where checklists and forms were being used by HART which were not described in the invoice payment procedure. No amounts were improperly paid. • 3 invoices that lacked the required payment review checklist. The invoices totaled $6,292,325. No amounts were improperly paid. • 2 invoices, valued at $18,607,656, had narrative descriptions attached to invoices which were incorrect. No amounts were improperly paid. • 2 invoices that had management approvals that totaled $23,288 although the work was performed prior to the execution of the contract agreements and constituted procurement violations. No amounts were improperly paid. • $11,344 for on-call contractor work performed for the Pig & the Lady restaurant (83 N. King Street), even though the work on the City Center had not officially started. HART contends the work was necessary, supported, and allowable under the terms of the contract. • $1,863 for unallowable travel agent fees. This was due to a conflict between HART and the City’s respective travel policies. • $740 for vacation travel expense paid for by HART. HART contends the work was allowable under the terms of the contract. Recommendations HART should: 10. Make it a priority to analyze significant changes to the project, determine how it will affect the project’s overall costs and schedule, and regularly update key management plans to reflect those changes to ensure that stakeholders and the public are informed of significant changes in a timely manner. 11. Ensure project managers prioritize budget management, compare actual costs to cost estimates, analyze any differences 43 Chapter 3: HART Needs to Improve Project Management and Contract Administration and make adjustments as necessary to prevent or minimize cost overruns. 12. Not make concessions on retainage to contractors, as it diminishes HART’s ability to ensure proper performance and could be misconstrued as favoritism or biased. 13. Document its cost-saving strategies and to the extent possible, quantify and document the amount of potential cost savings. 14. Develop and implement written internal policies and procedures that will address stipend payments, including requirements for supporting documentation of unsuccessful bidders’ actual costs, determination of compensated value, and limit payment to no more than the unsuccessful bidders’ actual costs or the stipend amount, whichever is less. 15. Better document its office space requirements and regularly review its office lease agreements to identify any unoccupied usable area. To reduce current operational costs so that it only pays for space that is needed and to find potential future savings, if space is unoccupied, HART should consider renegotiating the lease, subleasing the space, or allowing other city agencies to use or rent the space until HART actually needs the space. 16. Develop written policies and update procedures for contract administration. Clearly distinguish the roles and responsibilities of project managers, contract managers, and contract administrators in contract administration policies and procedures. Promote increased awareness of procurement and contract administration file recordkeeping by providing additional training to staff, and develop more robust guidance, policies, and procedures that address the variety of contracts and associated invoices HART receives in order to help standardize the invoice payment process and prevent improper payments. 44 Chapter 4: Better Planning Is Needed to Address and Manage Future Rail Project Costs Chapter 4 Better Planning Is Needed to Address and Manage Future Rail Project Costs Summary Once the rail system is completed and operational, other rail systems indicate annual and ongoing operations and maintenance costs must be addressed. Other rail systems in the nation indicate subsidies will be needed to fund rail operations and maintenance costs after the rail is constructed. HART has not planned for the operations and maintenance of the rail system or the costs of operating the system after it is completed. HART needs to improve planning and oversight to effectively address and manage future operations and maintenance needs; maximize fare box recoveries and ridership; minimize city subsidies; and fill operations and maintenance positions. Background Due to project delays, HART reports interim rail service will begin in late 2018 and full service operations are projected to begin in 2021. The original and updated time schedule is shown below. Exhibit 4.1 Project Schedule Source: Office of the City Auditor (OCA) analysis based on HART documents 45 Chapter 4: Better Planning Is Needed to Address and Manage Future Rail Project Costs HART Needs to Plan for Annual Operations and Maintenance of the Rail System HART reports it is only responsible for constructing the rail project and is not responsible for the ongoing, annual operations and maintenance of the rail system. As a result, we found HART plans were outdated and did not adequately address how to operate and maintain the rail system once it is completed. We also did not find plans that addressed the ongoing costs of operating the system. According to HART, however, the board and HART are committed to fulfilling its charter responsibilities for the operations and maintenance for the rail system. Other Rail Systems Indicate Annual and Ongoing Operations and Maintenance Costs Must Be Addressed Our city comparisons indicated that all rail services throughout the nation require some form of subsidy to make up for the costs of operating and maintaining the rail. Our comparisons showed that fares paid by riders were insufficient to cover the entire cost of operating and maintaining the rail systems. (See Exhibit 4.2) Exhibit 4.2 Subsidy Comparisons (Dollars in Millions) Rail Transit Service City Fare Revenues Atlanta Baltimore Boston Chicago Los Angeles Miami New York Philadelphia San Francisco Washington, D.C. $75.6 $12.9 $191.9 $278.2 $34.8 $22.8 $3,030.7 $95.7 $406.1 $605.5 Charlotte Cleveland Denver Houston Phoenix-Mesa Portland Sacramento Salt Lake Seattle-Puget Sound St. Louis $4.4 $3.0 $49.4 $4.5 $12.8 $46.4 $14.7 $19.0 $14.8 $18.6 Annual O&M Costs Heavy Rail $208.1 $51.7 $315.5 $513.6 $117.0 $77.7 $4,763.5 $186.7 $525.0 $909.5 Light Rail $13.1 $11.7 $87.1 $18.4 $28.7 $99.3 $50.0 $45.5 $52.9 $64.8 $ % $132.5 $38.8 $123.6 $235.4 $82.2 $54.9 $1,732.8 $91.0 $118.9 $304.0 63.67% 75.05% 39.18% 45.83% 70.26% 70.66% 36.38% 48.74% 22.65% 33.42% $8.7 $8.7 $37.7 $13.9 $15.9 $52.9 $35.3 $26.5 $38.1 $46.2 66.41% 74.36% 43.28% 75.54% 55.40% 53.27% 70.60% 58.24% 72.02% 71.30% Source: OCA analysis based on data obtained from the National Transit Database (NTD), 2013 46 Subsidy Chapter 4: Better Planning Is Needed to Address and Manage Future Rail Project Costs The comparisons indicated that fixed guideway projects often resulted in significant transit service realignments, such as the creation of a transit agency to oversee and administer the operations of rail and other modes of public transit. To ensure a smooth transition from construction to operations, HART needs to update its operations and maintenance plan, establish operations and maintenance policies, develop fare policy details, identify subsidy sources, determine ridership and sources of revenues, and fill critical positions related to the operations and maintenance of the system after it is completed. Subsidies Will Be Needed to Fund Rail Operations and Maintenance Costs Fare revenues are earned through carrying riders through regularly scheduled rail service. While fare revenues will cover a portion of the operations and maintenance costs, our comparison of other systems showed that the fare revenues will not be sufficient to fully support total operating and maintenance costs. According to HART’s chief executive officer (CEO), fare revenues will cover about 30 percent of the operations and maintenance costs. The remaining 70 percent will require subsidies from the city. Although city subsidies will be needed, HART has not clearly defined how rail operations and maintenance will be subsidized in its 2012 Full Funding Grant Agreement (FFGA) financial plan. Our comparison of other cities showed that other revenue sources1 are available to offset the cost of operating the rail system. For example, operating costs not funded by fare revenues can be supported by a combination of federal, state, and local government taxes. Exhibit 4.3 identifies state, federal assistance, and other fund sources for other cities. Excise taxes, special assessments for cities and towns, and property taxes. 1 47 Chapter 4: Better Planning Is Needed to Address and Manage Future Rail Project Costs Exhibit 4.3 Sources of Operating Funds by Transit Agency* (*) Percentages do not total 100 percent due to rounding. (**) Fare revenues based on the farebox recovery rate for entire transit agency in addition to rail service, often including but not limited to the following services: bus, commuter bus, commuter rail, light rail, heavy rail, ferry boat, and paratransit. Source: OCA analysis based on information from the NTD, 2013 Other forms of funding include parking fees, selling surplus land and property, retail space rental to vendors, utility company rentals of rights of ways, and advertising. Although these alternative sources of funds exist, HART has not identified other revenue sources for its rail operations and maintenance in its financial plan. As a result, HART and the city cannot ensure the city subsidies are minimized. HART Needs to Improve Planning to Maximize Fare Box Recoveries 48 The farebox recovery rate is the percentage by which the fare revenues collected account for the total operating costs of the service. It is calculated by dividing the total fare revenue by its total operating costs. Our city comparison of farebox recovery Chapter 4: Better Planning Is Needed to Address and Manage Future Rail Project Costs rates of other transit agencies showed that fare revenues alone will not be sufficient to fund all rail operations and maintenance costs. Exhibit 4.4 shows selected rail services across the metropolitan areas of the United States. All of these rail services had farebox recovery rates which varied greatly and were consistently less than 100 percent. Fares were either variable2 or flat rate3. Of the selected rail comparisons, farebox recovery rates ranged from 77 percent for San Francisco’s Bay Area Rapid Transit (BART) service to 24 percent for Houston. The average farebox recovery rate for all selected rail services for 2013 was 43 percent. Exhibit 4.4 Rail Farebox Recovery Rates Comparison Source: OCA analysis of farebox recovery rates based on information from the National Transit Database (NTD), 2013 A fare cost that varies in relation to the level of operational activity (time of day, distance travelled). 3 A fare cost that remains fixed irrespective to the level of operational activity. 2 49 Chapter 4: Better Planning Is Needed to Address and Manage Future Rail Project Costs Our comparison indicated HART needs to plan for potential fare revenues, farebox recovery rates, and anticipated ridership levels to ensure the rail operations and maintenance costs are covered. HART Needs to Improve Planning to Maximize Ridership 50 Our nationwide comparison of comparable rail systems indicated ridership levels have generally fallen short of forecasted levels. As part of the application process for the federal FFGA, HART developed ridership estimates and forecasts which were incorporated into the 2012 operations and maintenance plan (OMP). Our nationwide comparison indicated, however, that actual ridership fell short of the forecasted levels. (See Exhibit 4.5) Chapter 4: Better Planning Is Needed to Address and Manage Future Rail Project Costs Exhibit 4.5 Ridership Forecast vs. Actual Ridership City Baltimore Miami Washington Buffalo Portland Sacramento Atlanta North Line LA Red Line Chicago Orange Line Baltimore Johns Hopkins San Francisco Colma Dallas South Oak Cliff Denver Southwest Portland Westside Salt Lake South San Francisco SFO Washington Largo Chicago Douglas Branch Denver Southeast Portland Interstate MAX Sacramento South Forecasted Forecasted Year 1990 FTA Report* Heavy Rail 1987 103,000 1988 239,900 1986 569,600 Light Rail 1989 92,000 1989 42,500 1989 50,000 2003 FTA Report** Heavy Rail 2005 57,120 2000 297,733 2000 118,760 2005 13,600 2000 15,200 Light Rail 2005 34,170 2015 22,000 2005 49,448 2010 23,000 2007 FTA Report** Heavy Rail 2010 68,600 2020 14,270 2020 33,000 Light Rail 2020 38,100 2015 18,860 2015 12,550 Actual Percentage of Actual vs. Forecasted 42,600 35,400 411,600 41.36% 14.76% 72.26% 29,200 19,700 14,400 31.74% 46.35% 28.80% 20,878 134,555 54,986 10,128 13,060 36.55% 45.19% 46.30% 74.47% 85.92% 26,884 19,083 43,876 22,100 78.68% 86.74% 88.73% 96.09% 26,284 6,361 25,106 38.31% 44.58% 76.08% 22,545 12,785 8,734 59.17% 67.79% 69.59% (*) The federal FFGA program was established after the projects featured in the 1990 study. (**) As some forecasted years were beyond the scope of the 2003 and 2007 FTA reports, “actual ridership” figures represent most recent numbers at the time the report was conducted. Sources: OCA analysis based on information from the Urban Transit Rail Projects: Forecast Versus Actual Ridership and Cost (1990), prepared by Dr. Don H. Pickrell for the Urban Mass Transit Administration; Predicted and Actual Impacts of New Starts Projects: Capital Cost, Operating Cost, and Ridership Data (2003), prepared by the FTA; The Predicted and Actual Impacts of New Starts Projects: Capital Cost and Ridership (2007), prepared by the FTA. In our opinion, HART should prepare for a scenario in which actual ridership, or the number of passengers actually using rail, falls short of forecasted estimates. Its operations and maintenance plan is outdated and may result in inadequate revenues to cover the annual rail operating and maintenance costs. 51 Chapter 4: Better Planning Is Needed to Address and Manage Future Rail Project Costs HART Needs to Fill a Critical Operations and Maintenance Position HART’s Operations and Maintenance (O&M) director provides oversight related to the operations and maintenance of the rail system. The director is required to work closely with capital programs to provide operational and technical guidance during the design and construction phases. HART has not filled its O&M director position since the last occupant left the agency in August 2015. The city contract with Hitachi Rail Italy authorizes the company to operate and maintain the rail system and its passenger trains. According to HART, the Operations and Maintenance director will be responsible for administering the $1.4 billion operations and maintenance contract. The director will also be in charge of hiring a team of consultants to support the operations and maintenance function. Currently, no staff positions are assigned to support the operations and maintenance division. The vacant position4 and lack of support staff could leave HART unprepared when the rail becomes operational. Recommendations HART should: 17. Develop plans for annual and ongoing operations and maintenance of the rail system once it is completed and operational. The plan should address subsidies needed to fund rail operations and maintenance costs, maximize fare box recoveries and ridership; minimize city subsidies; address operations and maintenance (O&M) policies, subsidy sources, and alternative revenues. 18. Fill the operations and maintenance position and other key vacancies. The recruitment should include a succession plan to fill key positions in the event of any unexpected departures. 4 52 Subsequent to the audit, HART stated that it has filled this position. Chapter 5: Conclusions and Recommendations Chapter 5 Conclusions and Recommendations The Honolulu Authority for Rapid Transportation (HART) is a semi-autonomous local government agency established in July 2011 by charter amendment after voters approved a fixed guideway rail system. HART’s mission is to plan, design, construct, operate, and maintain Honolulu’s high-capacity, fixed guideway rapid transit system. HART’s processes have impeded its ability to construct and complete the project economically, effectively, and efficiently. Despite having a goal of completing the project on time and on budget, HART project costs have increased $1.3 billion, or 25 percent, from an estimated $5.2 billion at inception to an estimated $6.5 billion as of FY 2016. We found that the HART’s financial plan and Operations and Maintenance Plan (OMP) were not consistent, reliable or current. HART’s financial plan was not updated to reflect the rail project’s most current financial condition in spite of the significant cost changes. We found that HART’s Project Management Plan (PMP) and its subsidiary plans were also outdated, and unreliable as decision-making tools. HART needs to strengthen its controls over financial information reporting. For example, data was missing, incomplete, and not readily available from HART’s contract management system (CMS). Delay claims were inadequately tracked and monitored and may go unreported. We identified reporting discrepancies in pending utility agreements, differences in contingency allowance figures, and differences in general excise tax (GET) county surcharge receipts. In our opinion, additional cost overruns are likely. Rail project cost estimates consistently lacked sufficient detail and methodologies for underlying cost assumptions and were not documented. For example, delay claims were not updated to reflect the most current cost information, utility relocation costs increased by $120 million without this figure being fully supported, project enhancement costs of $46 million were not fully supported, and cost escalation increased by $240 million without using a specific factor. HART does not have adequate allocated and unallocated contingency reserves. HART underreported its projected shortfall 53 Chapter 5: Conclusions and Recommendations amount related to a $9.9 million GET coding error. We also found that project managers were not adequately managing their budget by comparing actual costs against their respective budgets. HART needs to improve project management and contract administration controls. We found HART made questionable concessions to a single contractor, and failed to perform quantitative analysis to justify a major decision to repackage a bid for 9 stations on the westernmost portion of the fixed guideway transit system (Westside Stations Group). HART lacked policies and procedures for approving and paying stipends to unsuccessful bidders and paid $1.5 million in legal stipends without knowing the actual costs incurred by the bidders. We also found that HART has been paying for vacant office space. Stronger controls over contract administration practices and invoice payment practices are needed. Specifically, guidance for contract administration is unclear, procurement file documentation controls had weaknesses, and the invoice payment process is inadequate to prevent improper payments from being made. We determined that HART needs better planning and oversight to effectively address and manage future needs. Subsidies will be needed to fund rail operations and maintenance costs after the rail is constructed. The amount of subsidy needed will depend on several variable factors, such as fares and the number of riders. Recommendations HART should: 1. Increase efforts to regularly update its financial plan. The cost changes and adjustments are necessary to reflect the current financial condition of the project. Updates should be supported by detailed, source documentation. 2. Update its Operations and Maintenance Plan (OMP) to address funding, management, and other transit needs. 3. Consistently and accurately report on project cost information, identify and explain variances if internal and external reports are intended to be different so that policy makers and the public receive consistent and reliable project cost information. 4. Develop methods to ensure data used in HART, PMOC, and other reports are consistent, accurate, reliable, and can be reconciled among all the reports using the data. 54 Chapter 5: Conclusions and Recommendations 5. Develop a process for tracking and monitoring all costs, including the status of delay claim costs. 6. Support its cost estimates with consistent, reliable and sufficient information. To do so, HART should thoroughly document details, including any forecasting methodology and assumptions made to support of its cost estimates. 7. Replace the contract management system (CMS) with a system that is more user friendly and more appropriate to managing the HART construction project. If the CMS system is retained, HART should define which CMS data elements, data fields, and functions should be used and which parts should be deactivated or eliminated. 8. Use the city’s C2HERPS enterprise resource planning system to develop, monitor, track, and report budget, financial, and accounting data. The CMS system should not be used for these purposes. 9. Develop a forecasting model to best predict escalation costs and support it with documentation. 10. Make it a priority to analyze significant changes to the project, determine how it will affect the project’s overall costs and schedule, and regularly update key management plans to reflect those changes to ensure that stakeholders and the public are informed of significant changes in a timely manner. 11. Ensure project managers prioritize budget management, compare actual costs to cost estimates, analyze any differences and make adjustments as necessary to prevent or minimize cost overruns. 12. Not make concessions on retainage to contractors, as it diminishes HART’s ability to ensure proper performance and could be misconstrued as favoritism or biased. 13. Document its cost-saving strategies and to the extent possible, quantify and document the amount of potential cost savings. 14. Develop and implement written internal policies and procedures that will address stipend payments, including requirements for supporting documentation of unsuccessful bidders’ actual costs, determination of compensated value, and limit payment to no more than the unsuccessful bidders’ actual costs or the stipend amount, whichever is less. 55 Chapter 5: Conclusions and Recommendations 15. Better document its office space requirements and regularly review its office lease agreements to identify any unoccupied usable area. To reduce current operational costs so that it only pays for space that is needed and to find potential future savings, if space is unoccupied, HART should consider renegotiating the lease, subleasing the space, or allowing other city agencies to use or rent the space until HART actually needs the space. 16. Develop written policies and update procedures for contract administration. Clearly distinguish the roles and responsibilities of project managers, contract managers, and contract administrators in contract administration policies and procedures. Promote increased awareness of procurement and contract administration file recordkeeping by providing additional training to staff, and develop more robust guidance, policies, and procedures that address the variety of contracts and associated invoices HART receives in order to help standardize the invoice payment process and prevent improper payments. 17. Develop plans for annual and ongoing operations and maintenance of the rail system once it is completed and operational. The plan should address subsidies needed to fund rail operations and maintenance costs, maximize fare box recoveries and ridership; minimize city subsidies; address operations and maintenance (O&M) policies, subsidy sources, and alternative revenues. 18. Fill the operations and maintenance position and other key vacancies. The recruitment should include a succession plan to fill key positions in the event of any unexpected departures. Management Response 56 HART is staffed with many dedicated individuals who are striving to successfully complete the rail project. We are therefore disappointed by the HART responses which mischaracterize and misrepresent the audit discussions and the history of the audit. HART had over 5 weeks to respond to the draft reports. Distribution of the confidential drafts was limited to only authorized HART and city officials. HART requested and received early distribution of the drafts and conceded it distributed the confidential drafts within its organization without our permission. HART’s attempts to discredit the audit work and attempts to intimidate the auditors were unprofessional. Chapter 5: Conclusions and Recommendations In its response to the management discussion draft, HART disagreed with the audit findings and recommendations and provided additional documents and data which we closely examined. Based on our analysis and verification of the additional information, we modified the report and updated the findings. However, our analysis and verification of the additional information reaffirmed our audit results. Based on our audit work and supporting work papers, we stand by our audit findings, conclusions, and recommendations. In the formal, management response, HART states the audit did not conform to generally accepted government auditing standards, the auditor’s independence was impaired, and timeline changes were not communicated. HART states the findings were not supported, not sufficiently documented, the auditors lacked technical expertise, and did not follow AICPA standards. HART disagreed with 11 draft recommendations and partially agreed with 10 draft recommendations. It is our hope that HART will reconsider and implement all the recommendations needed to improve its financial management, planning, project management, contract administration and other operations to ensure the rail project costs are minimized. We believe the recommendations will help HART to successfully complete the new rail project. Nominal changes and edits were made to this report to enhance the report format and to better communicate the audit results. The substance of the findings and recommendations remain substantively unchanged. A copy of the HART executive director and CEO’s response is provided in Appendix H. 57 58 Chapter 5: Conclusions and Recommendations This page intentionally left blank. Appendix A: Audit Objectives, Scope, and Methodology Appendix A Audit Objectives, Scope, and Methodology City Council Resolution 15-90 authorized the City Auditor to conduct a performance audit of HART to determine the adequacy of its processes to ensure that the rail project is constructed and completed economically, effectively, and efficiently. The resolution asked for an examination of seven inter-related areas. We organized the audit to address those areas through the following objectives: • Assess the reliability of HART’s financial information provided to government decision makers and the public about the project’s fiscal challenges. • Determine whether HART has a reasonable subsidy plan to fund future operational and maintenance costs. • Assess compliance of HART’s procurement and contract management practices for its contracts awards and associated expenditures. • Evaluate the project’s financial viability and the likelihood of other factors that could potentially increase the project’s revenue shortfall and cost overruns. To meet our objectives, we reviewed applicable city, state, and federal laws, and rules and regulations. These included city and the State of Hawai`i procurement policies and procedures; FTA requirements; and contract and project management general guidance and best practices. We reviewed HART’s policies and procedures; and conducted interviews with key management, staff, and consultants to obtain information about HART’s current fiscal condition and financial plans for the future. We did not review the most current financial plan because it was being updated at the time of our audit. HART referred us to the original 2012 Full Funding Grant Agreement (FFGA) financial plan and PowerPoint slides for project cost updates. We relied on information provided by HART. We examined records stored in the city’s Financial Management System (C2HERPS) and Document Management System (DocuShare); and HART’s Contract Management System (CMS). These records included budget documents from fiscal year (FY) 59 Appendix A: Audit Objectives, Scope, and Methodology 2011 through FY 2015, annual financial statements, revenues and expenditures reports1, and various correspondence for the period of July 1, 2007 through June 30, 2015. We assessed the reliability of HART’s financial information by tracing records in C2HERPS, CMS, and internal and external reports to source documents. Although we questioned the completeness and accuracy of some data, we relied on documents and reports provided by HART because they were the only information available at the time of our audit. These documents and reports included HART’s Full Funding Grant Agreement (FFGA) financial plan, the FFGA, current and past business plans, HART Monthly Progress Reports, Project Management Oversight Consultant (PMOC) Monthly Reports from October 2011 through December 2015, Monthly HART Facts, HART board meeting minutes, and other correspondence. We did not test HART’s financial statement information, which is audited by independent auditors. On a test basis, we examined 25 construction and professional services contract files (including contract solicitation, bid, and award documents) based on contracts awarded as of June 22, 2015. The majority of the contracts we reviewed were awarded by HART using competitive sealed bidding, competitive sealed proposal, professional services procurement, or sole source procurement methods. We did not examine the procurement process as part of our review. We also selectively reviewed procurement file documentation. We conducted interviews with key staff and consultants to obtain an understanding of HART’s contracts and its invoice payment process. We also interviewed project managers who directly managed the contracts we reviewed. We examined, on a test basis, 50 construction, professional services, and operational invoices paid during the period of July 1, 2007 through June 30, 2015. For each invoice, we evaluated whether these invoices were properly reviewed and approved, adequately documented to support construction and consulting activities, and consistent with the contract terms before payment authorization. We also evaluated whether the payment practices complied with city policies and procedures, and if industry best practices were in effect during the audit period. 1 60 The forecast reports were titled Forecast Costs by Contract with Details. Appendix A: Audit Objectives, Scope, and Methodology We compared HART’s project cost management practices to best practices in the Project Management Institute’s Guide to the Project Management Body of Knowledge (PMBOK Guide) with a focus on project cost management; the Government Finance Officers Association (GFOA) guidance on capital project monitoring and reporting; and the U.S. Government Accountability Office’s (GAO) Cost Estimating and Assessment Guide and Schedule Assessment Guide. We identified other localities that have design-build-operatemaintain rail systems. Based on the availability of information, we compared forecasted and actual capital costs; operation and maintenance costs; forecasted and actual ridership data; subsidy sources; and farebox recovery rates with each capital project. We visited HART’s leased office space and calculated the office vacancy rate as part of our review of non-project costs. In 2012, the Office of the City Auditor issued an audit report, Audit of the Honolulu Authority for Rapid Transportation (HART) Public Involvement Programs, which identified several areas for improvements. The report stated HART routinely paid consultant and sub-consultant invoices despite minimal documentation; the agency’s CMS lacked sufficient data; and that consultant performance and work products were not formally monitored and evaluated. During the audit, we assessed whether these concerns were resolved. We performed this audit from May 2015 to April 2016 in accordance with generally accepted government auditing standards (GAGAS). These standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We obtained evidence that provides a reasonable basis for our findings and conclusions based on our audit objectives. 61 62 Appendix A: Audit Objectives, Scope, and Methodology This page intentionally left blank. Appendix B: Glossary of Acronyms Appendix B Glossary of Acronyms ARRA C2HERPS CEO CFO CMS EPLS FFGA FTA FY GEC GET HART HR HRTP LR NTP O&M OCA OMP PM PMOC PMP PMSC RFC ROM SAM American Recovery and Reinvestment Act City and County of Honolulu Enterprise Resource Planning System Executive Director and Chief Executive Officer Chief Financial Officer Contract Management System Excluded Parties List System Full Funding Grant Agreement Federal Transit Administration Fiscal Year General Engineering Consultant General Excise Tax Honolulu Authority for Rapid Transportation Heavy Rail Honolulu Rail Transit Project Light Rail Notice to Proceed Operations and Maintenance Office of the City Auditor Operations and Maintenance Plan Project Manager Project Management Oversight Consultant Project Management Plan Project Management Support Consultant Request for Change Rough Order of Magnitude System for Award Management 63 64 Appendix B: Glossary of Acronyms This page intentionally left blank. Appendix C: Glossary of Terms and Definitions Appendix C Glossary of Terms and Definitions Change Orders are written orders or alterations within the scope of the contract that direct the contractor to make changes authorized by the contract with or without the consent of the contractor. Contract changes within the scope of the contract may relate to specifications, delivery point, rate of delivery, period of performance, price, quantity, or other provisions of the contract. Chief Procurement Officer is Honolulu Authority for Rapid Transportation’s (HART) Executive Director or designee. City and County of Honolulu Enterprise Resource Planning System (C2HERPS) is an Oracle based enterprise resource management and reporting system for the City and County of Honolulu. Contingency is an allowance in cost estimate and schedule for unknowns, typically based on the lack of detail in the construction documents, unknowns anticipated during construction, and based on the difficulty level of the work. Contract Administrator is the person identified in a Contract’s Special Provisions who is designated to manage the various facets of the contract to ensure satisfactory performance in accordance with the contractual commitments and that obligations to HART are fulfilled. Contract Amendment is one type of formal contract modification. It must be in writing. Contract Management System (CMS) is HART’s Oracle Primavera document management system and contract control software for the project. HART uses CMS to track and store all pertinent documents related to the project, including but not limited to, contracts, submittals, request for information, meeting minutes, transmittals, purchase orders, cost worksheets, change orders, and invoices. Contract Manager coordinates with the Project Manager regarding Contract Change Orders for compliance with HART policy and procedures and compliance with contract terms and conditions. The Contract Manager will review merit determination and Negotiation Strategy Memos and provide recommendations to the Project Manager. Contract Modifications are any written alteration within the scope of the contract to specifications, delivery point, rate of delivery, period of performance, price, quantity, or other provisions in the contract executed between the government and the contractor. This includes contract amendments, change orders, and task orders. Design-Build is a contract delivery methodology under which HART contracts with a single entity that has responsibility for the design and construction of the project. Estimate at Completion (EAC) is the forecast total cost of completing the work package scope by contract, as well as the total project scope. 65 Appendix C: Glossary of Terms and Definitions Excluded Parties List System (EPLS) was a web-based system that identified those parties excluded from receiving federal contracts, certain subcontracts, and certain types of federal financial and non-financial assistance and benefits. Fare Revenue is revenue earned from carrying passengers in regularly scheduled service. Passenger fares include the base fare, zone premiums, and peak period premiums. Farebox Recovery Rate of a passenger transportation system is the proportion of the amount of revenue generated through fares by its paying customers as a fraction of the cost of its total operating and maintenance expenses. Financial Plan documents the recent financial history of the transit agency, describes its current financial health, documents projected costs and revenues, and demonstrates the reasonableness of key assumptions underlying these projections. Firm Fixed Price Contract includes a price that remains fixed irrespective of the contractor’s cost experience in performing the contract. A firm fixed price contract may include an economic price adjustment provision, incentives, or both. Full Funding Grant Agreement (FFGA) is an agreement between the federal government FTA and HART as a semi-autonomous agency of the city and county of Honolulu that sets forth the scope of the project that will be constructed using federal and non-federal funds, establishes a financial ceiling with respect to FTA’s participation in the project, establishes a time for completion and specifies the mutual understanding, terms and conditions relating to the construction and management of the project. Heavy Rail is defined as a mode of transit service (also called metro, subway, rapid transit, or rapid rail) operating on an electric railway with the capacity for a heavy volume of traffic. It is characterized by high-speed and rapid acceleration passenger rail cars operating singly or in multicar trains on fixed rails; separate rights-of-way from which all other vehicular and foot traffic are excluded; sophisticated signaling, and high platform loading. Invoice is the contractor’s request for compensation for services provided based on the contract for the project. Light Rail is defined as a mode of transit service operating passenger rail cars singly (or in short, usually two-car or three-car, trains) on fixed rails in shared or right-of-way that is often separated from other traffic for part of the way. Light rail vehicles are typically driven electrically with power being drawn from an overhead electric line via a trolley or a pantograph; driven by an operator on board the vehicle; and may have either high platform loading or low level boarding. National Transit Database (NTD) is a federally mandated database reporting system, established by Congress to be the Nation’s primary source for information and statistics on the transit systems of the United States. Recipients or beneficiaries of grants from the FTA under the Urbanized Area Formula Program or Other than Urbanized Area (Rural) Formula Program are required by statute to submit data to the NTD. Over 660 transit providers in urbanized areas currently report to the NTD through the Internet-based reporting system. Each year, NTD performance data are used to 66 Appendix C: Glossary of Terms and Definitions apportion over $5 billion of FTA funds to transit agencies in urbanized areas. Annual NTD reports are submitted to Congress summarizing transit service and safety data. Notice to Proceed is the official notification to the contractor that they may proceed with the work based on the conformed documents issued to the contractor at the time of notification. Operating and Maintenance Cost is the recurring costs of providing public transportation service. They include: all employees’ wages and salaries; fringe benefits; operating supplies such as fuel, and oil; contractors’ charges for services; taxes; repair and maintenance services, parts, and supplies; equipment leases and rentals; marketing; lease or rental costs; and insurance. Operating expenses include administrative expenses. Operating costs exclude fixed costs such as depreciation on plant and equipment, costs of providing transportation services not available to the general public, and interest paid on loans on capital equipment. Operations and Maintenance Plan documents five years of historical data and presents 20 years of projected system operating revenues and O&M costs to demonstrate the capability of the agency to operate and maintain the proposed project while providing existing levels of transit service. Project Controls are acts of the project management staff assisting the project controls manager in all aspects of cost, schedule, contract administration, and configuration management. Project Manager is responsible for managing scope, schedule, and budget. The Project Manager has authority to initiate, negotiate, and process changes. The Project Manager determines merit, oversees and monitors contract changes, leads negotiations and prepares Contract Change Order documents. The Project Manager may delegate responsibility. Project Management Oversight Consultant (PMOC). The Federal Transportation Agency (FTA) hires a consultant to provide oversight of the HART rail project. The consultant provides continuous review and evaluation of the rail project to ensure compliance with federal requirements and to monitor project progress in areas such as time, budget, plans, and specifications. Ridership is the number of passengers using a form of public transportation. Rough Order of Magnitude (ROM) is an estimate provided by the contractor or general engineering consultant (GEC) for a request for change (RFC) prior to the RFC having authorization from HART for final pricing. The ROM will include schedule changes based in weeks and costs based on $10,000 increments. Subsidy is an allocated amount of financial assistance from the government. System for Award Management (SAM) is the official U.S. government system that combines federal procurement systems and the Catalog of Federal Domestic Assistance into an e-procurement system. Transit Agency (also called transit system) is an entity (public or private) responsible for administering and managing transit activities and services. Transit agencies can directly operate transit service or contract out for all or part of the total transit service provided. 67 Appendix C: Glossary of Terms and Definitions Utility Agreement is a contract with a utility company which defines the scope of a relocation, including reimbursement, liability, right-of-entry, insurance, and schedule to complete the work. Westside Stations Group consists of the first nine stations along the rail route. 68 Appendix D: Contingency Balance Comparison* (by Month) Appendix D Contingency Balance Comparison (by Month), (Dollars in Millions) No. PMOC Monthly Reports Contingency Allocated 1 Unallocated 2 Month HART Contingency Drawdown Spreadsheet Balance Ending Balance Oct 2011 N/A N/A N/A 2 Nov 2011 N/A N/A N/A N/A -- 3 Dec 2011 N/A N/A N/A N/A -- 4 Jan 2012 N/A N/A N/A N/A -- 5 Feb 2012 N/A N/A N/A N/A -- 6 Mar 2012 N/A N/A N/A N/A -- 7 Apr 2012 N/A N/A N/A N/A -- 8 May 2012 N/A N/A N/A N/A -- 9 Jun 2012 N/A N/A N/A N/A -- 10 Jul 2012 N/A N/A N/A $649.4 -- 11 Aug 2012 N/A N/A N/A $654.5 12 Sep 2012 N/A N/A N/A $654.5 -- 13 Oct 2012 N/A N/A N/A $654.4 -- 14 Nov 2012 N/A N/A N/A $654.7 -- 15 Dec 2012 N/A N/A N/A $654.4 -- 16 Jan 2013 N/A N/A $478.0 $654.4 ($176.40) 17 Feb 2013 N/A N/A $476.0 $652.8 ($176.80) 18 Mar 2013 N/A N/A $467.8 $652.3 ($184.50) 19 Apr 2013 N/A N/A $441.5 $654.3 ($212.80) 20 May 2013 N/A N/A $447.4 $654.3 ($206.90) 21 Jun 2013 N/A N/A $400-$430 $654.3 ($224.3-$254.3) 22 Jul 2013 N/A N/A $413-$439 $649.1 ($210.1-$236.1) 23 Aug 2013 N/A N/A $415-$443 $643.2 ($200.2-$228.2) 24 Sep 2013 N/A N/A $433.0 $637.1 ($204.10) 25 Oct 2013 N/A N/A $440.0 $615.0 ($175.00) 26 Nov 2013 N/A N/A $444.0 $610.0 ($166.00) 27 Dec 2013 N/A N/A $460.0 $609.8 ($149.80) 28 Jan 2014 N/A N/A $444.0 $608.3 ($164.30) 29 Feb 2014 $367.0 $61.0 $428.0 $608.2 ($180.20) 30 Mar 2014 $362.0 $60.8 $422.8 $608.2 ($185.40) 31 Apr 2014 N/A N/A N/A $590.7 -- 32 May 2014 $352.6 $63.8 $416.4 $568.4 ($152.00) 33 Jun 2014 $331.1 $66.3 $397.4 $563.4 ($166.00) 34 Jul 2014 $323.8 $66.3 $390.1 $563.7 ($173.60) 35 Aug 2014 N/A N/A N/A $559.4 -- 36 Sep 2014 $324.6 $66.3 $391.0 $550.1 ($159.10) 37 Oct 2014 $281.0 $67.0 $348.0 $531.8 ($183.80) 38 Nov 2014 $274.1 $56.1 $330.2 $542.5 ($212.30) 39 Dec 2014 $267.5 $66.5 $334.0 $523.8 ($189.80) 40 Jan 2015 N/A N/A N/A $519.8 -- 41 Feb 2015 $291.8 $11.5 $303.3 $515.6 ($212.30) 42 Mar 2015 $291.8 $11.5 $303.3 $510.7 ($207.40) 43 Apr 2015 N/A N/A N/A $510.4 -- 44 May 2015 $291.8 $11.5 $303.3 $509.8 ($206.50) 45 Jun 2015 $291.8 $11.5 $303.3 $501.7 ($198.40) 46 Jul 2015 $291.8 $11.5 $303.3 $500.7 ($197.40) 47 Aug 2015 N/A N/A N/A $492.9 -- 48 Sep 2015 $291.8 $11.5 $303.0 $492.8 ($189.80) 49 Oct 2015 $291.5 $11.5 $303.0 $492.8 ($189.80) 50 Nov 2015 $291.5 $11.5 $303.0 $489.0 ($186.00) 51 Dec 2015 $291.5 $11.5 $303.0 $488.2 ($185.20) ** ** ** ** ** N/A Difference 1 -- (**) Some quarterly reports do not contain contingency balance information. Source: Office of the City Auditor (OCA) comparison of PMOC Monthly and Quarterly Reports from October 2011 through December 2015 and HART Contingency Drawdown Spreadsheet Allocated contingency is allocated to each work package to address potential uncertain changes within each respective work package. 2 Unallocated contingency funds unknown changes to the project and not currently allocated to a particular work package. 1 69 70 Appendix D: Contingency Balance Comparison* (by Month) This page intentionally left blank. Appendix E: General Excise Tax (GET) Receipts Comparisons* (by Quarter) Appendix E General Excise Tax (GET) Receipts Comparisons (by Quarter), (Dollars in Millions) Year 2011 PMOC Monthly HART Internal Reports Spreadsheet GET Surcharge Actual GET Receipts Received Surcharge To Date Since Quarter 2007 Receipts Difference 4 $746 $810 ($64) 2012 1 $796 $859 ($63) 2012 2 $859 $906 ($47) 2012 3 $907 $932 ($25) 2012 4 $554 $974 ($420) 2013 1 $554 $1,029 ($475) 2013 2 $651 $1,080 ($429) 2013 3 $651 $1,143 ($492) 2013 4 $1,140 $1,187 ($47) 2014 1 $1,180 $1,248 ($68) 2014 2 $1,180 $1,298 ($118) 2014 3 $1,480 $1,346 $134 2014 4 $1,480 $1,404 $76 2015 1 $1,404 $1,469 ($65) 2015 2 $1,469 $1,522 ($53) Source: OCA comparison of PMOC Monthly and Quarterly Reports from October 2011 through November 2015 and HART GET Collection Schedule as of November 2, 2015. 71 Appendix E: General Excise Tax (GET) Receipts Comparisons* (by Quarter) This page intentionally left blank. 72 Appendix F: Procurement Documentation File Appendix F Procurement Documentation File Where appropriate, the procurement documentation file should contain: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Purchase request, acquisition planning information, and other pre-solicitation documents Evidence of availability of funds Rationale for the method of procurement (negotiations, formal advertising) List of sources solicited Independent cost estimate Statement of work/scope of services Copies of published notices of proposed contract action Copy of the solicitation, all addenda, and all amendments Liquidated damages determination An abstract of each offer or quote Contractor's contingent fee representation and other certifications and representations Source selection documentation Contracting Officer's determination of contractor responsiveness and responsibility Cost or pricing data Determination that price is fair and reasonable including an analysis of the cost and price data, required internal approvals for award Notice of award Notice to unsuccessful bidders or offerors and record of any debriefing Record of any protest Bid, Performance, Payment, or other bond documents, and notices to sureties Required insurance documents, if any Notice to proceed Source: FTA Best Practices Procurement Manual, November 2001 73 74 Appendix F: Procurement Documentation File This page intentionally left blank. Appendix City Council Resolution 15-90 CITY AND COUNTY OF HONOLULU HONOLULU, HAWAII 1 5 RESOLUTION REQUESTING THE AUDITOR TO CONDUCT A PERFORMANCE AUDIT OF THE HONDLU LU RAIL TRANSIT PROJECT. WHEREAS, at a cost of $5.2 billion and rising, the Honolulu Rail Transit project ("project") represents the largest and most expensive capital project in the history of the 1Clity and County of Honolulu; and WHEREAS, on July 1, 2011, the Honolulu Authority for Rapid Transportation was established through an amendment to the Revised Charter of the City a and County of Honolulu and is authorized to develop, operate, maintain and expand the project; and WHEREAS, the plan to pay for this project has always centered on federal funding of $1.55 billion that is being awarded to HART in phases overtime, as well as a half-percent general excise tax surcharge for rail that has been collected since 200?;and WHEREAS, with construction underway, budgetary, ?nancial, funding and cost concerns threaten the continued viability of the rail transit project, re?ected in a growing shortfall of almost $1 billion due to reponed factors such as project delay costs, contract Ioid escalations, and low surcharge revenues, as well as the need to cover future operating and maintenance costs, and extend the rail line to areas such as Mama and West Kapoiei; and WHEREAS, these budgetary, ?nancial, funding and cost concerns have resulted in HART appealing to both the City Council and the State Legislature for ?nancial assistance, including asking the City Council to adopt Resolution which would authorize the issuance of City general obligation bonds and notes up to $1.9 billion. and asking the State Legislature to extend the GET surcharge in perpetuity; and WHEREAS, both City councilmembers and State legislators have expressed continuing frustration over the lack of detailed ?nancial information, de?nitive construction costs, and solid financial plans with which to base decision?making for a project with the size, projected growth and uncertain ?nal funding needs that the rail project represents; and WHEREAS, the Council finds that, given the scope of the project and the unknown magnitude of the funding needed in the future, a performance audit is warranted in order to provide accountability and public transparency, particularly at a time when public and government discussion, deliberation, and decision-making are 0C32015-0265f3f31f20'15 8:34 AM 1 75 76 Appendix G: City Council Resolution 15-90 CITY COUNCIL .- of} cnvauo OF HONOLULU ND 1 . HONOLULU, HAWAH RESOLUTION focused on the soundness and continued viability,r of the project and its impact upon the residents of Honolulu. now, therefore, BE lT RESOLVED by the Council of the City and County of Honolulu that the City Auditor is requested to investigate and conduct a performance audit of the Rail Project to determine the adequacy of its processes to ensure the Rail Project is constructed and completed economically, effectively and ef?ciently; and BE IT FURTHER RESOLVED that with an audit objective of providing government decision-makers with suf?cient information to form the basis for legislative decisions that provide appropriate and justified funding, while also remaining careful stewards of taxpayer dollars, the Council requests the City Auditorto examine. at a minimum, the following: HART's financial plan, including any and all contingency ?nancial plans and updates ?led subsequent to the establishment of the initial financial plan; (2) Contract awards and associated expenditures {including subcontractor costs), as well as details regarding what the expenditures are being used for; The amount of cost increases, over and above projections in financial plans, and details on cost containment measures and the effectiveness of these measures to mitigate increasing costs and growing shortfalls; A detailed accounting of all revenues and expenditures for the rail project from 200? through the end of 2014; The point atwhich HART should have known or became aware of the potential shortfall; The likelihood and potential for additional cost overruns exceeding the current shortfall being reported by and The potential continuing costs after construction is completed and the Rail Project is operational; and 008201 5-026Eif?f31 ?201 5 3:34 AM 2 Appendix G: City Council Resolution 15-90 ?it. CITY COUNCIL r? -. CITY AND COUNTY OF HONOLULU Rama" HONOLULU. HAWAII No 15-90 RESOLUTION BE IT FURTHER RESOLVED that the Cit;f Auditor is requested to meetwith, and coordinate audit efforts with, the State Auditor. should the 2015 State Legislature pass a measure or measures requesting a similar State audit to ensure that both City and State audits are not duplicative but act synergistically to maximize resources, findings and recommendations; and BE IT FINALLY RESOLVED that copies of this Resolution be sent to the Mayor: -.- .- OE: Managing Director, the Executive Director of the Honolulu Authority for Rapid T?ehsportation. and the City Auditor. 351?2?? IN UCED I.) r3". I I I. ?1 7' (fl DATE OF INTRODUCTION: _lEtAR319n15 Honolulu, Hawaii Councilmembers 008201 502688819015 8:34 AM 3 77 78 Appendix G: City Council Resolution 15-90 CITY COUNCIL AND OF HONOLULU HAWAII CERTIFICATE RESOLUTION 15-90 Introduced: Committee: TRANSPORTATION By: ERN EST MARTIN RESOLUTION REQUESTING THE CITY AUDITOR TO CONDUCT A PERFORMANCE AUDIT OF THE HONOLULU RAIL TRANSIT PROJECT. Title: Voting Legend: TRANSPORTATION (JR-159 RESOLUTION REPORTED OUT OF COMMITTEE FOR ADOPTION. COUNCIL (JR-159 AND RESOLUTION 1530 WERE ADOPTED. Tr" AYES: ANDERSON, ELEFANTE, FUKUNAGA, KOBAYASHI, MARTIN, MENOR, OZAWA. 2 ABSENT: MANAHAN, PINE. I hereby certify that the above Is a true record of action by the Geo noil of the City no LEN KA ASHI, ACTING CITY CLERK EST Y. MARTIN, CHAIR {We PRESIDING OFFICER Appendix Management Response 80 Appendix H: Management Response This page intentionally left blank. Appendix H: Management Response IN REPLY REFER TO: A I HONOLULU AUTHORITY for RAPID TRANSPORTATEON Daniel A. Grabausl-ias EXECUTIVE DIRECTOR AND CEO . BOARD OF DIRECTORS Apr? 14' Damien T.K. Kim VICE CHAIR . . . George I. Atta Mr. Edwrn S. W. Young, Crty Auditor "?3333 git-1:52:31 Office Of the Auditor Terribliujii City and County of Honolulu wr?gr?ear??zigangri; 1001 Kamokila Boulevard, Surte 216 ?Efrem.?- Lee Ivan M. Lur-Kwan Kapolei, Hawaii 96707 Dear Mr. Young: Subject: Audit of the Honolulu Authority for Rapid Transportation (HART) Report Dated April 2016 Enclosed please ?nd response to the audit report. As an organization, HART welcomes critical, vigorous, and healthy oversight and, therefore, takes the work of the Of?ce of the City Auditor (OCA) seriously, as it does with all federal, state and local oversight, reviews, and audits. Accordingly, it is discouraging when a report is written in a fashion to intentionally mislead, is issued in an improper manner, and conceded to politically motivated pressure. As a result of the improprieties and irregularities noted during the audit and with this report, HART cannot say with any degree of certainty that this audit was performed in accordance with auditing standards or that the associated report is free from bias. This correspondence provides comprehensive response to this audit report, questions the auditing standards applied, responds to the report?s recommendations, and addresses the report?s content point by point. See Attachment 1. Background The OCA conducted this audit from May 2015 to April 2016. During this period, HART spent signi?cant time and resources to respond to a large number of OCA requests, provided extensive information and documents, arranged interviews with HART personnel, and answered all questions regarding the project. On March 3, 2016, HART received a draft audit report. Upon review, HART found that the report contained factual errors, misleading statements, and unsupported conclusions. On March 28, 2016, HART provided the OCA a comprehensive written response to the draft audit report. On March 31, 2016, the OCA met with HART staff at the exit conference and notified HART that the ?nal report would be released on April 15th with or without management response. Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 2 April 14, 2016 On April 8, 2016, HART received the final audit draft. The OCA incorporated some of comments and clari?cations into the ?nal version. Unfortunately, errors still remain in the final audit report. Statements made in the audit report are not supported by relevant or directly applicable facts, resulting in misleading assertions and incorrect conclusions. Purpose and Applicability of Generally Accepted Government Auditing Standards (GAGAS) GAGAS provide a framework for conducting high-quality audits with competence, integrity, objectivity, and independence. This framework outlines requirements and guidance related to ethics, independence, auditors? professional judgment and competence, quality control, performance of the audit, and reporting. Because auditing is essential to government accountability to the public, the public expects auditors who conduct their work in accordance with GAGAS to follow ethical principles. Ethical principles apply in preserving auditor independence, taking on only work that the auditor is competent to perform, performing high-quality work, and following the applicable standards cited in the auditors? report. HART cannot reasonably conclude this audit was performed in accordance with these standards as outlined in the de?ciencies below. 1. Auditor?s Independence Was Impaired HART disagrees with many of the audit report?s ?ndings and recommendations as the auditor?s independence was impaired. Integrity and objectivity are maintained when auditors perform their work and make decisions that are consistent with the public interest and those relying on the auditors? report. On June 22, 2015, the auditor originally informed HART, the Managing Director, and the Director of the Department of Transportation Services that the audit report deadline was July 1, 2016. However, after City Council passed Bill 23 the General Excise Tax (GET) Surcharge Extension, the auditor pushed to complete ?eldwork as quickly as possible. When HART pointed out errors and misrepresentations of facts in the original discussion draft, the auditor responded by cutting 2 1/2 months off the reporting deadline. Due to the new deadline, the OCA had only one week to review responses and additional documentation. There was not suf?cient time to ensure that due professional care was taken to review the documentation and correct the number of errors that HART had highlighted. Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 3 April 14, 2016 Additionally, the ?nal draft report was released prematurely to the City Council and subsequently the news media. This ?nal draft report was incomplete as it did not include management response. Under Government Auditing Standards, the auditor is required to allow HART a reasonable amount of time to provide comments to the ?nal draft before it is released to those charged with governance such as City Council. The premature release of the draft audit report does not support an audit environment free from undue in?uence and bias. In a letter from the City Council Chair, dated April 7, 2016, addressed to the Mayor, the Chair refers to the results of the audit report. HART analyzed the document properties of the .pdf uploaded on the Star-Advertiser website and found that the letter was scanned at 1:59 pm, which was two hours before HART received the ?nal draft report. This calls into question the integrity and independence of the auditor and others who released con?dential material. 2. ?gni?cant Change in Reporting Timeline Was Not Communicated GAGAS states that the Auditor should communicate an overview of the objectives, scope, methodology, and timing of the performance audit and planned reporting to management of the audited entity. At the audit entrance conference held on June 22, 2015, the Auditor communicated to HART, the Managing Director, and the Director of the Department of Transportation Services, the timing of the performance audit. The of?cial start date of the audit was July 1, 2015, and the reporting deadline for the final audit report was July 1, 2016. Prior to the exit conference held on March 31, 2016, the OCA requested a private meeting with CEO. During this meeting, the OCA informed CEO that he was under a great deal of pressure by members of the City Council to release the final audit report in the following two weeks. During the exit conference, the OCA informed HART that the report would be issued with or without management response in mid-April. The deadline was later confirmed to be April 15, 2016. This represented a significant change in the timing of the audit report and was not communicated verbally or in writing to HART prior to the meeting on March 31, 2016. Additionally, the'fact that the OCA was under extreme pressure to move up the reporting deadline is of serious concern because in doing so did not allow the auditor suf?cient time to complete the audit with due professional care. The auditor did not maintain his independence and, thus, failed to follow the professional standards set forth in GAGAS. Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 4 April 14, 2016 3. Mort Missing Suf?cient Appropriate Evidence to Support Findings According to GAGAS Reporting Standards, audit reports should clearly communicate the results of audits to those charged with governance and the audit entity. Standards prescribe that the audit report should contain suf?cient and appropriate evidence to support the ?ndings and conclusions as they relate to the audit objectives. Moreover, ?ndings should be clearly developed in order to assist management and oversight of?cials in understanding the need and so that they take corrective action. The audit report was written in a manner that included statements and conclusions without any supporting details or references. It appears that the conclusions were written and then the evidence gathered to support it. Evidence gathered to the contrary was labeled insufficient, none, or simply ignored. As such, HART remains unclear on what basis the report?s conclusions were drawn. During the exit conference, HART asked the OCA to provide the basis for some of the statements and conclusions contained in the draft audit report. The OCA failed to provide further information and said HART was not allowed to ?audit the auditor?. Therefore, HART is not in a position to understand what the conclusions were based on and was left to respond, in general, to the report?s errant assertions. 4. Findings Were Not Suf?ciently Documentegn the Report Auditors should plan and perform audit procedures to develop the elements of a ?nding necessary to address the audit objectives. Per GAGAS, the elements include Criteria, Condition, Cause, Effect, and Recommendation. The standards further state that audit evidence that supports the ?ndings, conclusions, and recommendations should be documented before the report is issued. This information was not provided in the report as required nor to HART. As a result, HART believes the findings are not sufficiently documented. 5. Audit Team Lacked Technical Expertise initial response to the draft audit report pointed out that the audit team lacked the technical expertise in engineering, project management, procurement, and Information Technology (IT) systems to perform its evaluation. In some instances, the auditors ignored documentation because they didn?t understand it. Without the required technical knowledge, the auditors? judgment was insuf?cient. The auditors were looking for absolute assurance because they lacked the knowledge base to make a judgment on what was reasonable Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 5 April 14, 2016 documentation. As a result, many of the findings in the report re?ect the auditor?s lack of knowledge in these areas. Furthermore, most of the items that the auditors took issue with had been previously audited or reviewed by technical experts without the same findings. Government Auditing Standards require auditors to consider whether the reason for this difference in audit results is due to a lack of technical knowledge. The auditors failed to request to review other auditors? work that overlapped with their own as encouraged in the auditing standards Triennial, Procurement Review, Single Audit, Improper Payment Elimination Recovery Act (IPERA) Audit, Annual Independent Financial Audit, Financial Management Oversight Review). 6. Audit Report Did Not Follow American Institute of Certi?ed Public Accountants (AICPA) Attestation Standards The City Council Resolution requested the auditor to perform an examination in addition to a performance audit. GAGAS standards require examinations to follow AICPA attestation standards. AICPA attestation standards require the auditor to render an opinion on whether the assertion in the items examined is presented, in all material respects, based on the criteria. The auditor?s report does not express an opinion on any items examined. Furthermore, since the auditor never obtained written assertions from HART, the report should be restricted in accordance with the attestation standards. Responses to Audit Report Recommendations 1. Increase efforts to regularly update its financial plan. The cost changes and adjustments are necessary to re?ect the current ?nancial condition of the project. Updates should be supported by detailed, source documentation. Response: HART disagrees with this recommendation in that the Auditor?s report fails to distinguish between a ?formal? update that is the basis for the full funding grant agreement and regular ongoing updates of the financial plan that compare the approved plan to actual and updated information. The Auditor's report assumes that if the formal plan is not updated, then there has been no update to the financial plan. This assumption is incorrect and is not supported by the evidence and public record. The Federal Transit Administration (FTA) has requested a ?formal? updated financial plan. A formal plan will be the basis for any revision to the full funding grant agreement. The updated ?formal? financial plan update could not be completed until: (1) the full passage of the GET surcharge extension; and (2) a full risk refresh by the FTA is completed. Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 6 April 14, 2016 The GET surcharge extension was enacted on February 1, 2016. HART and the FTA began the risk refresh on March 30, 2016. The risk refresh examines the scheduling and potential cost of the project based on the most current cost and schedule information. The ?formal? financial plan update is expected to be completed in approximately four months following this risk refresh process. Although the formal updated financial plan will not be available until the end of the year, the current financial plan is and has been continuously updated and reviewed by HART since its initial completion in June 2012. These updates have been shared with the HART Board of Directors, City Administration, City Council, State Legislature, and the State Department of Budget Finance several times since 2014 after resuming construction and overcoming legal challenges in state and federal court. The following is a summary of updated financial plan reports: June 2014: Updated financial plan relative to the Debt Management Plan update December 2014: Project Cost Update January 2015: City Council Approval of Debt Financing Resolution with updated financial plan attached February 2015: Updated Financial Plan in Response to several detailed questions from the State Budget Finance Director August 2015: FTA review of ?draft? financial plan update October 2015: Project Cost Update November 2015: Letter to City Council reflecting October 2015 Update in Financial Plan January April 2016: financial plan cash flow updates to City Treasurer February 2016: HART Permitted Interaction Group established to review assumptions to formal updated to the financial plan The Auditor?s report also recommends that updates should be supported by detailed source documentation. The updates to the financial plan are reviewed by transportation financial experts who are the most experienced and technically quali?ed to review financial plans. In August 2015, the FTA reviewed a ?draft? financial plan with the extended GET figures and the updated project costs. Based on the expert review, the FTA took an extraordinary step of accepting the ?draft? as fulfilling of a triennial review action, even though the GET surcharge extension had not been enacted. In addition to FTA experts, the State Department of Budget Finance has also reviewed an update to the ?nancial plan. reviewed the plans revenue assumptions, project cost, and operating cost assumptions in February 2015, and validated and confirmed financial revenue and cost assumptions to several legislative committees during the 2015 Legislative Session that resulted in granting a five-year extension of the GET Surcharge to complete the project in May 2015. Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 7 April 14, 2016 financial updates are sound, and are well documented in that they have met the review and scrutiny of both federal and state financial reviews. 2. Update its Operations and Maintenance Plan (0MP) to address funding, management, and other transit needs. Response: HART disagrees with the auditor?s assertion that the 0MP is outdated and unreliable as a decisjon?making tool. The 0MP has been updated and has been submitted to the Project Management Oversight Contractor (PMOC) in September of 2015, prior to the date of the audit report. During the audit, the auditor interviewed the Core System?s Design-Build-Operate- Maintain (DBOM) Contract Project Manager. However, the auditors did not discuss operating costs or the 0MP plan with him. The 0MP was already updated, reviewed internally and is currently under review with the HART is awaiting their response. The plan was updated to incorporate the operational shift from 2-car consists to 4-car consists. The previous version was still in an acceptable format for use as a decision- making tool. While plans will be updated following significant changes in key system elements the plan serves as a tool for defining the system operations, system elements, plan for operations and maintenance of the system and staf?ng. Exhibit 3.4 Key Management Plans - this table should be updated to reflect that the OMP plan has been updated and submitted to the PMOC. 3. Consistently and accurately report on project cost information, identify and explain variances if internal and external reports are intended to be different so that policy makers and the public receive consistent and reliable project cost information. Response: HART disagrees with the assertion in this recommendation. HART does consistently and accurately report on project cost information. HART regularly updates and reconciles critical financial data consistent with procedures. Examples include and are not limited to: project cost expended, pstimate- at-completion projections, project contingency drawdown, overall project progress, contractor notice-to-proceed values, executed change orders, contracts awarded, and others. Also, HART provides extensive, consistent and reliable project cost information through many formats available to all stakeholders, policy makers, and decision makers. Project cost data is directly provided to the HART Board of Directors, FTA, PMOC. and is published in the local press on a basis. Further, the Progress Report, Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 8 April 14, 2016 Balanced Scorecard, and other presentations made at public meetings are available on the HART website. 4. Develop methods to ensure data used in HART, PMOC, and other reports are consistent, accurate, reliable, and can be reconciled among all the reports using the data. Response: HART disagrees with the assertion that project data represented in our reports are inconsistent or unreliable. HART consistently updates and reconciles critical financial information in line with procedures. The auditor?s finding seems to be based on faulty analysis as detailed in the HART responses provided in the attachment. For example, the auditor is misrepresenting the consistency of the reports reflected in Exhibit 2.5. As HART staff explained to the auditors it is not reasonable to compare reports that exclude FTA ineligible cost (eg. Forecast Cost Report, HART Facts report) to reports that include FTA ineligible cost List of Awarded Contracts Summary) as of June 22, 2015 because the data is filtered differently. Cost reports provided to FTA excludes ineligible costs, therefore many cost reports by default, filters those costs out. HART explained on multiple occasions to the auditors that because Contract Management System (CMS) is a live database comparing two reports run on different dates will not tie out/be equal since it?s highly likely the information will have changed during that timeframe. Despite being informed otherwise, the auditors are comparing four reports that have different data fully aware the reports will not match. Another example of the auditor?s misunderstanding of data is provided in Exhibit 2.7 and the subsequent bulleted items highlighting the variances in delay costs. This entire section, including sub-bullets are irrelevant because they are mischaracterizing a working spreadsheet they were provided as a management tracking tool and identifying variances from CMS database. The auditors requested support for the cost impact of the delays highlighted in the December 18, 2014 Risk presentation. HART staff provided said spreadsheet, a working document tracking delay costs as snapshot at a particular point in time. However the auditor insisted on viewing this as a management tool. All projected changes and claim values are tracked via CMS which is management tracking tools for cost data and updated 5. Develop a process for tracking and monitoring all costs, including the status of delay claim costs. Response: HART disagrees with this recommendation as it already has a process for tracking and monitoring all costs, including the status of delay claim costs. All projected changes, claim values, and costs are tracked in CMS which is management tracking tool for cost data and is updated Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 9 April 14, 2016 Every delay claim is fully supported and documented through the entire change order process. The auditors wanted to see a separate report for tracking and monitoring delay costs. HART understands the information contained in its reports and is able to extract information as necessary. 6. Support its cost estimates with consistent, reliable and suf?cient information. To do so, HART should thoroughly document details, including any forecasting methodology and assumptions made to support of its cost estimates. Response; Every estimate that HART has produced in the recent past and will produce for the remainder of the project will be accompanied with a Basis of Estimate report. For larger estimates this can be many pages documenting the assumptions, inclusions, exclusions and methodology. For smaller estimates the basis of estimate report can take the form of a memorandum. The purpose of this report is to state the estimating methodology used for the development of costs, provide the accuracy range that anticipated bids could be received within, reference the scope included, and scope excluded. Samples containing the above methodology have been provided. Our recent track record with the stations packages has been accurate usually within low single percentage points of the final contractors? bids. Engineers' Estimates. Lowest Bid Second Lowest Bid FHSG 5 76,211,602 5 77,726,000 5 84,254,238 95% WOSG 74,077,000 56,088,470 32.1% 66,543,692 11.3% KHSG 108,106,000 5 112,719,789 115,805,845 Note: Chart's negative number represents an estimate lower than contractors' bids. Positive number re?ects an estimate higher than contractors? bids. 7. Replace the contract management system (CMS) with a system that is more user friendly and more appropriate to managing the HART construction project. If the CMS system is retained, HART should de?ne which CMS data elements, data ?elds, and functions should be used and which parts should be deactivated or eliminated. Response: HART strongly disagrees with the recommendation. The HRTP project is a $6.5 Billion project, with more than $4.5 Billion scheduled for expenditures in the next 5 years, equivalent to almost a billion dollars expenditures per year in the coming five years. There are only a handful of systems that can handle such enormous amounts of construction-related data, and they all follow similar architecture and organization, namely Oracle CMS, Meridian?s Prolog, Bentley EADOC and Autodesk Constructware. All these software packages are appropriate for managing the HART construction project, and all require extensive training and commitment. HART implemented a training program and continues implementing it for its selected software. Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 10 April 14, 2016 in the past several years, HART has invested thousands of hours in training and mentoring thousands of users. Current HART CMS Users 396 with Login Overall HART CMS users >850 from beginning of program (includes personnel turnover) CMS Classes for all HART, 454 classes Contractor and Consultant Users Number of students 2401 people attending User friendliness cannot be considered a relevant criteria as all these software programs require advanced staff proficiency in construction concepts and management processes which is several steps above basic computer applications literacy. A multi? billion dollar construction program cannot be managed with basic spreadsheets, user- created databases and/or similar basic documents and it is a fundamental error when one fails to estimate the complexity of a system that deals with tens of terabytes of data and attempts to use inadequate tools to manage it. None of these systems mentioned allows the proper processes to be followed, nor are they geared to handle size and complexity of the records. They do not provide date-stamping, records of exchange between entities. and archival records maintenance that are required for auditing purposes. After thorough evaluation, CMS was chosen as the centralized project management and document control system aimed at creating and tracking all documents related to the project. It provides one centralized database, paperless routing using workflows, and access to real-time data 24/7 via the Internet. Its benefits include: 1. Bringing hundreds of users together in a real-time, online construction project management environment where data is recorded and tracked. 2. Automating construction processes to ensure consistency in data preparation, collection, processing and archiving. 3. Reducing project cost, schedule risk and exposure by enabling teams to collaborate on critical information made readily available on a 24]? basis. Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 11 April 14, 2016 Facilitating user adoption within HART through a construction-specific solution and role-based user interfaces aimed at recording all relevant and required data at each step of construction processes. Bringing extended project teams together across geographical locations, in online and offline environments, and across desktop, browser and mobile technology and devices. Managing all construction data, from the field to the office. Tracking key performance indicators to monitor all levels of project performance using metrics and dashboards Viewing and managing selected projects or groups of projects; create multi- project reporting, and access charts showing project budget. costs and contract status Manage budgets, commitments, costs, savings and overruns with detailed cost control functionality 10. Track change orders and see their impact on budgets and costs 11.Effectively manage daily work journals, crews, inspections, punch lists and other quality control tasks in a centralized tracking system The following are the different modules established within CMS to track data: Design And Construction Documents 1. RF Is 2. Submittals 8. Memos 15. Specifications 3. Change Orders 9. Daily Logs 16.Drawings 4. Extra Work Orders 10. Meeting Scheduler 17. Schedules 5. Design Clarification 11.Meeting Agendas 18.Job Site Photos Memos 12.Meeting Minutes 19. Environmental Documents 6. Quality Inspections 13.Action Items 20.Custom Documents 7. Punch Lists Tags 21.Correpondence Financial Module Features 1. Funding Sources 2. Division, Program and Project Financial Reports 4. Budgets 7. Pay Requests 3. Management 5. Contract Tracking 8. Contract Change Proposals Processes 6. Schedule of Values 9. Change Orders Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 12 April 14, 2016 All of the above are set with active processes meant to manage project delivery. There are no modules that can be eliminated to ?simplify or make system use easier?. All data input is through role based entry points which are specific to each user. HART expects the volume of documents to quadruple by the end of the project. Oracle CMS is the leading solution provider in the industry and best suited to handle such volume of document. Regarding the matter of CMS data elements, data fields, and functions that should be used and which should be deactivated or eliminated, HART notes all viable software choices presented common business processes and workflows to help user implementation for the following goals: 1. Project Delivery 2. Cost Controls 3. Standard Reports As such, and to enable the generation of management reports and forecasts, there are no data fields, or modules or data elements that can be arbitrarily eliminated. Nor does elimination make the basic use of the CMS any easier. Finally, any discrepancies and inaccuracies within the database as mentioned in the recent audit report are due either to 1. timing of data entry 2. reports that are tailored to specific needs of departments which cause various filter implementations (such as for example maintaining or removing ineligible costs) 3. timing of report (since the database is always live, a report run at different dates will most likely show different results even if filtered to the same data date (as credits and debits are added or processed into the record) 4. stage at which data is reported (some output include forecasts which are of the rough order of magnitude type, whereas other exclude them due to the fact that have not been validated) backlogged or missing scans and/or attachments user errors in data recording or data processing such as missing steps or entering information prematurely in a final stage, which can be resolved by additional training. 7. entry errors such as inaccurate figures entry, which are rare but can happen 9m Items 6 and 7 are specific issues that HART CMS trainers are on the lookout for and address regularly to prevent their reoccurrence. HART believes that its efforts have led Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 13 April 14, 2016 to the establishment of an accurate, complete and functioning database properly serving the needs of the program. 8. Use the city's CZHERPS enterprise resource planning system to develop, monitor, track, and report budget, financial, and accounting data. The CMS system should not be used for these purposes. Response: HART uses the city?s C2HERPS system as its official financial system of record. All items of revenue and expenditures are recorded in CZHERPS, and that information is used to prepare annual audited financial statements. The information recorded in CZHERPS is also used to prepare annual budget and cash flow projections. as well as various reports provided to the Mayor and City Council, progress report, FTA grant reimbursement requests. and FTA quarterly reports. Contract Management System (CMS) supplements the city*s CZHERPS system and includes more detailed contract information and documents that cannot be recorded in the City?s financial system, CZHERPS. CMS is used to internally review and approve construction contract invoices, and also includes documentation of the process and the approvals of all other documents that are required for the ef?cient and effective completion of the project. Also refer to response to recommendation #7 above regarding the use of CMS. The City?s CZHERPS system only allows access to city employees and access to reports are further restricted by the employee?s title and classification. This would limit the effectiveness of the C2HERPS as the automated system to manage the rail project as many of the project managers and executive management team are not city employees and would not be allowed to have access. 9. Develop a forecasting model to best predict escalation costs and support it with documentation. Response: As the OCA audit report indicates on page 13, "Cost estimating, by nature, is imprecise.? HART has established procedures and has methodologies in place to prepare cost estimates based upon historical data as well as using the latest bid results, other agency procurement results, market studies, and independent economic reports Rider Levett Bucknall Quarterly Reports). All of HART's Independent Cost Estimates (ICE) prepared for all upcoming contracts are based upon sound, detailed estimating techniques, including a description of the estimated scope of work, a basis of the estimate, estimating assumptions, and an indication of the range of possible results. These ?estimating essentials? are also bullet- listed in the OCA audit report on page 14. HART agrees with this guidance and provided several completed ICEs to the auditor that comply with these requirements. Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 14 April 14, 2016 Predicting escalation is even more imprecise and HART relies upon independent economic data sources to use within its estimates (depending on the type of estimate and the type of project being developed) as HART explained several times to the OCA auditors. In addition, HART explained to the auditors that there is no specific factor used that can be attributed directly to ?escalation? alone. Rather, as HART prepares its estimates (ICEs) and summarizes several cost estimates into its budget forecast updates, there are items that can be readily identified to a specific cause, but for those costs that cannot (market conditions, contractor risk assessments, building type risks, schedule risks, third party risks), HART assigns to ?escalation.? In comparing budget forecasts (made up of several cost estimates) prepared at different times. it is not unusual that these two budget forecasts be different because more is known about the project at the later timeframe. Therefore, the escalation category is expected to change over time as well, depending on the events occurring between these two time periods, and is driven by differences in the totals of the cost estimates, not by a specific escalation factor. For the specific escalation issue highlighted in the OCA audit report (Cost Escalation line item increasing from $45 million at the December 2014 HART Board Meeting to $240 million at the October 2015 HART Board Meeting), HART explained that the $195 million increase was not calculated by a specific rate (as detailed above), but that the Cost Escalation was a subset of several factors within the project cost estimates that made up the total budget forecasts. HART explained that the known events that occurred between the two budget forecasts included impacts due to past and potential lawsuits and delays, utility relocations, and overhead costs. The remaining difference between the detailed cost estimates prepared during each time period (December 2014 and October 2015) that made up the total budget forecasts were therefore attributed to escalation. HART agrees that this response may give an impression that all HART escalation forecasting is done in a way that groups several unknown or unforeseen conditions and places them in one cost category without considering (or detailing) a specific escalation rate. We understand the OCA audit report?s point for that specific case and will take prudent measures to ensure that these costs are presented in a more transparent way. As stated previously, the HART methodology involves using multiple independent sources to apply market factors and escalation rates to unique situations and construction scope along the alignment. 10. Make it a priority to analyze significant changes to the project, determine how it will affect the project?s overall costs and schedule, and regularly update key management plans to re?ect those changes to ensure that stakeholders and the public are informed of signi?cant changes in a timely manner. Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 15 April 14, 2016 response is broken down into the following parts: Part 1 Make it a priority to analyze significant changes to the project, determine how it will affect the project?s overall costs and schedule Part 2 regularly update key management plans to reflect those changes Part 3 ensure that stakeholders and the public are informed of significant changes in a timely manner Part 1 HART does make it a priority to analyze all changes to the project in a timely manner. The project controls department receives and reviews cost and schedule reports from all design and construction contracts. Many of the plans identified in the audit report were identified to be updated in August 2014. This happened to be right before the 9 station bid package bid proposals were received and greatly exceeded the original estimates. At this time HART, working with its partners, had to analyze and restructure the program execution method in order to contain cost. Re?structuring the Contract Packaging Plan (one 9?staiton package into three 3-station packages) resulted in numerous changes including re-organization of staffing and consultant services to manage the separate contracts, re-allocation of funding to cover higher costs, update of the master schedule and the risk and contingency management plan. It was also during the 2015 year that the FTA performed the Triennial Review of HART and DTS. Also, during this time, City Council took up a resolution to remove Section 5307 monies from the HART project funding. Part 2 Staff has been working to update the program plans and keep up with the changes. Draft revisions have been created and shared with the PMOC and FTA, but are awaiting ?nal sign-off for a variety of measures. For example, the Financial Plan update was on-hold until it became apparent the GET would be extended. The update of the financial plan impacted the contract packaging plan as HART was now facing a budget de?cit that would require major scope reductions had the GET not been extended. Staff created numerous draft schedules to match the contract packaging plan options along with estimated costs. The project management plan has been updated in draft form, but pending the update to the CPP and Financial Plan, the draft is waiting for final approval. HART will be entering the Risk Refresh Workshop with the FTA at the end of March and believes that comments and reports resultant of that workshop will help finalize the plans. Part 3 HART agrees that stakeholders and the public should be informed of signi?cant changes and works hard through various channels to do so. The following is a list of the various ways in which HART strives to keep our project partners up to date: Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 16 April 14, 2016 Written correspondence and presentations to the State Legislature and City Council HART Board Meetings Reports HART Facts Ad Public Access Television Program HART Business Outreach Programs Town Hall meeting series Weekly E-blasts and Traf?c Advisories The HART Website and Social Media The HART Board had tasked the Executive Director and CEO to make transparency a primary concern for the agency. The agency received and adopted the recommendation from a City Council member to televise HART Board meetings. 11.Ensure project managers prioritize budget management, compare actual costs to cost estimates, analyze any differences and make adjustments as necessary to prevent or minimize cost overruns. Response: HART agrees that effective project management should focus on effective budget management and HART Project Managers do monitor and manage contract costs (budgets) through the monitoring of actual costs to contract amounts, identification of issues that may have a cost impact, and managing contract contingency and the change process. HART concurs there is room to improve and initiated a global approach to project management, starting with contract administration training for the scheduled for June 2015. Concurrently, HART is realigning on-site construction oversight and contract administration between consultant Resident Engineers who have the in-depth knowledge and skills in working on large, complex transit systems projects, with the City project managers who have the local, City, HART knowledge and background, to better balance responsibilities and control. 12. Not make concessions on retainage to contractors, as it diminishes ability to ensure proper performance and could be misconstrued as favoritism or biased. Response: All required retainage amounts are determined by the terms of the contract. HART does not make or grant any concessions that may be misconstrued as favoritism or bias. 13. Document its cost-saving strategies and to the extent possible, quantify and document the amount of potential cost savings. Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 17 April 14, 2016 Response: HART provided several documents to the auditor containing hundreds of pages of information to support the decision to repackage the Westside Stations Group (WSSG) into three separate 3-station packages: Farrington Highway Station Group West Oahu Station Group and Kamehameha Highway Station Group (KHSG). These documents included: 0 Bid Assessment and Path Fon/vard Recommendation, dated 8/20/2014 0 Request to Cancel WSSG Solicitation, dated 9/9/2014 Value Engineering, Scope Reduction, and Scope Transfer Ideas for each individual west side station, with varying dates from 9/4/2014 through 9/26/2014 0 FHSG Scope Revision document used to direct the designer of record to implement various design changes, dated 9/4/2014 0 Contemporaneous Meeting Notes summarizing meetings with various WSSG contractors, dated 9/11/2014 through 9115/2014 0 White Paper on Delivery and Procurement Strategy for Upcoming Contracts, dated 11/7/2014 HSG Independent Cost Estimate (ICE) - dated 12/10/2014 H2R2 Ramp ICE - dated 1/13/2015 WOSG ICE - dated 4/2/2015 KHSG ICE - dated 8/17/2015 HART did perform quantitative analysis to justify a major decision to repackage a bid for the 9 stations in the Westside Stations Group. There is no other recent event on the program that required more internal scrutiny, analysis. and debate than the decision to repackage the nine stations into three 3-station packages. HART provided the documentation listed above to illustrate the steps taken to solve the ongoing budget concerns, with Rough Order of Magnitude (ROM) estimates included that were based on our consultants? value engineering, estimating, and risk management expertise. HART pointed out to the auditors that one of the most significant steps towards potential lower construction bids was to understand the contractor?s perspectives, information that could not have been solicited during an active procurement, thus HART cancelled the original bid process on 9/9/2014 to allow HART to meet with and gather feedback from contractors. HART provided the auditors with resultant lists of information gathered from the conversations held with general contractors (both those that did and those that did not bid the WSSG package) and subcontractors. HART also took prudent non- quantitative steps to simplify drawings and specifications, modify contract terms and conditions, and ease schedule restraints to make the three packages more attractive to potential bidders. It is apparent that the auditors did not take the entirety of this complex information into consideration to reach their conclusions. Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 18 April 14, 2016 In the end, the actual bids received for these station packages and the H2R2 Ramp were $38 million below the previously submitted $294 million low bid for the WSSG contract. HART believes that this outcome was predicted, using analysis and documented analysis, as well as past experience, and current or historical data.? The lCEs prepared prior to bidding each of the three station packages and the Ramp H2R2 project totaled $263 million, or $31 million below the original $294 million low bid amount and very close to the actual $38 million savings realized. HART also objects to the OCA auditor?s assertion that the August 2014 Report indicated that the overall cost of the $5.2 billion project would not change. The auditor?s assertion is tied to the independent cost estimates that were prepared over the following year (lCEs: 12/10/14; 1/13/15; 4/2/15'. and 8/17/15 as listed previously), information that was not known at the time of the August 2014 Report. It is disingenuous to state that HART knew something based on information that was developed after the fact. 14. Devel0p and implement written internal policies and procedures that will address stipend payments, including requirements for supporting documentation of unsuccessful bidders? actual costs, determination of compensated value, and limit payment to no more than the unsuccessful bidders? actual costs or the stipend amount, whichever is less. Response: Stipends are only provided for certain large and complex design-build (DB) contracts as authorized under HRS Sec. The stipend amount set by HART (at the time, Rapid Transit Division of City?s Department of Transportation Services is based on industry estimates for the cost to prepare proposals. The industry estimate for preparing proposals is between 5% and 1% of the construction costs. For a contract, the cost for preparing proposals is estimated between has never paid more than $500,000 stipend to an unsuccessful offeror, which is well below the actual costs for preparing proposals. A uniform stipend amount is also consistent with Federal Transit Administration?s (FTA) Best Procurement Practices on the subject matter. Furthermore, solicitation documents expressly stated the terms under which an unsuccessful offer would be entitled to stipends and the not-to-exceed amount to be paid to each unsuccessful offeror; that is, a minimum 41% overall qualitative score was required to receive a stipend. HART agrees to memorialize its policies and procedures into a ?policies and procedures" manual. 15. Better document its office space requirements and regularly review its office lease agreements to identify any unoccupied usable area. To reduce current operational costs so that it only pays for space that is needed and to ?nd potential future savings, if space is unoccupied, HART should consider renegotiating the lease, subleasing the space, or allowing other city agencies to use or rent the space until HART actually needs the space. Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 19 April 14, 2016 Response: HART currently leases 63,927 sq. ft. (55,110 useable sq. ft.) of office space at Ali?i Place. HART conducted a physical inventory on 3/15/2016 and determined its occupancy rate to be 88% comprised of the following breakdown: Suite 150: 31.5 of 34 available workstations in use 93% Occupancy Rate 11th floor: 63 of 68 available workstations in use 93% Occupancy Rate 17th floor: 65 of 68 available workstations in use 96% Occupancy Rate 23rd floor: 48 of 66 available workstations in use 73% Occupancy Rate Overall: 207.5 of 236 available workstations 88% Total Occupancy Rate Note: Workstations include locations designed for someone to sit and work on a full time basis. This includes offices. cubicles. desk/work areas, etc. The vacant space is necessary and reasonable given that, as of February 23. 2016, HART had (11) vacant positions and is contractually required to provide office space for multiple contractor/consultant firms; HART currently houses employees from the following firms: HDR (lnfraConsult), CH2M Hill. Paragon Partners Ltd., Lea+Elliot, PGH Wong, RM Towill, SSFM, Stantec, Lawson Associates, and the Solis Group. HART also hires local college students as Engineering and Student Interns on a part time basis in accordance with the City County of Honolulu Department of Human Resources policies when office space is available and we have substantive work appropriate for Interns. Additionally, it should be pointed out that the practice of providing office space to consultants was a recommendation in a prior City Auditor audit reference City Auditor report 13-03, recommendation #8 Lease all office space to reduce consultant overhead rate charges. The numbers reported in the audit appear to have focused solely on the 23rd floor, which is the least occupied floor. The recommendation that HART renegotiates its lease ?so that it only pays for space that is needed? is an oversimplified view of how long term leasing works and is not practicable for an organization such as HART. HART is currently responsible for overseeing the construction of the Honolulu Rail Transit Project. As an organization, HART functions differently from a regular City Department and the personnel needs in terms of both quantity and areas of expertise needed vary significantly over time as opposed to an established City Department. Specifically the personnel/workstation needs evolve and change throughout the project lifecycle depending on which phase of the project we are in. A project?s staffing level is dynamic and is directly related to the type and amount of work going on at any given time. In order to accommodate the project?s personnel needs, HART has taken great effort to structure and negotiate the current leases to meet current and future office space requirements. approach considered the number of personnel required as the project progresses towards completion and optimizes the office space requirement and the reSulting cost to the project. Starting in 2018, the of?ce leases begin a structured reduction of office space over the remainder. All while being Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 20 April 14, 2016 cognizant that long-term leasing of office space yields the best rates for the lessee and ultimately benefitting the taxpayer. Therefore the recommendation that HART only lease office space it needs implies that HART change the amount of leased space depending on how many people are onboard at any given time. This is an impractical view of office leasing practices and would result in drastically increased costs as well as inefficiencies that would affect the overall project in a negative way. 16. Develop written policies and update procedures for contract administration. Response: HART agrees and has initiated updating all contract administration procedures. An updated Contract Change Procedure 1) was issued in September 2015. Revisions to the remaining procedures are developed and under review. The target date for updating contract procedures is August 2016. 17. Clearly distinguish the roles and responsibilities of project managers, contract managers, and contract administrators in contract administration policies and procedures. Response: HART agrees and has already initiated efforts to update procedures and clarify roles responsibilities. The updated procedures will reflect current roles responsibilities for project managers and contract managers. The target date for updating contract procedures is August 2016. 18. Promote increased awareness of procurement and contract administration file recordkeeping by providing additional training to staff. Response: HART believes in continued training of staff and intends to continue with this practice. 19. Develop more robust guidance, policies, and procedures that address the variety of contracts and associated invoices HART receives in order to help standardize the invoice payment process and prevent improper payments. Response: HART agrees and has initiated updating its Contractor Progress Payment procedure (5.CA-03) and is preparing a draft contractor invoice procedure that will standardize the invoice payment process. The target date for updating contract procedures is August 2016. 20. Develop plans for annual and ongoing operations and maintenance of the rail system once it is completed and operational. The plan should address subsidies needed to fund rail operations and maintenance costs, maximize fare box recoveries Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 21 April 14, 2016 and ridership; minimize city subsidies; address operations and maintenance policies, subsidy sources, and alternative revenues. Response: HART is actively planning and continuing to develop cost estimates per updated OMP now being reviewed by FTA. Rail fare policy is being discussed at HART, and note that farebox recovery planning efforts are unden/vay and these are concurrent with efforts to create an integrated multi- modal fare system that will work on bus and rail. HART believes it would have been premature to begin farebox recovery ratio planning efforts until the fare system was determined and the agreements for its funding and operation in place. The HART Fare PIG decided to explore farebox recovery options using the most up to date ridership estimates (as pervious ridership estimates were developed using land use, population and employment data that were available prior to 2010) working in tandem with the update of the regional transportation planning model efforts currently being led by Oahu Metropolitan Planning Organization (OMPO). In this way, the ridership estimates used by HART for revenue planning purposes will match those used by the City and the OMPO for transportation planning and funding purposes. Those ridership estimates will be available in the spring of 2016. At that time, microeconomic modeling that considers the impact of fare prices by market segment in comparison to historical elasticity of demand, transit competition such as parking pricing, taxi pricing, bike and care share pricing will be undertaken. Options for actual fare revenue recovery potential will be developed for the HART Board?s consideration with recognition of the FFGA constraints. Options will include potential fare products and pricing. Once the HART Board recommended farebox recovery rate is determined, revenue estimates will be incorporated in the latest version of the operations and maintenance financial plan. A source of funding whatever the rail subsidy is determined to be has not been identified as yet, and will be done collaboratively between the HART Board, Mayor and Council before any decision regarding fare policy/subsidy is made, and there is ample time to do so. The following table is a partial listing of various Board (and Staff activities) related to rail operations, fare policies, and subsidy reduction measures. This illustrates commitment and actions regarding rail costs and fare policy. Date I Description Group 7/1/2011 HART Board sworn in Board 7/7/2011 Financial Plan Discussed Finance 9/8/2011 DBOM Core Systems Contractor Capacity Finance/POC 9/29/2011 Safety and Security Presentation POC 10/6/2011 Update on DBOM Core Systems Contract Board 10/13/2011 DBOM Core Systems Contract Discussion Board Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 22 April 14, 2016 Date Description Group 11/3/2011 Presentation on Scope of DBOM Core Systems Board 11/ 17/2011 Update on DBOM Core Systems Contract Board 12/1/2011 Update on DBOM Core Systems Contract Board 3/16/2012 Bus and Handi-Van Operating Costs Finance 6/28/2012 FFGA Financial Plan Update Board 8/9/2012 Discussion on Fare Gates Finance 8/9/2012 Discussion on Rail/Bus Synergies Finance 8/9/2012 Discussion on Train Seating Finance 8/9/2012 Discussion on Financial Plan Operating Budget Finance 8/9/2012 FFGA Financial Plan Update Finance 8/30/2012 Fare Gates Board 8/30/2012 Train Seating Board 10/4/2012 FFGA FCA Update Operating Budget Discussion Finance 11/29/2012 FFGA Financial Plan Discussion Board 12/6/2012 FFGA Financial Plan Discussion Finance/POC 12/6/2012 Review of HART Operating Statement Finance/POC 12/6/2012 Discussion of DBOM Core Systems Finance/POC 2/7/2013 Update on DBOM Core Systems Contractor Capacity Finance/POC 10/17/2013 Discussion on Maintenance Yard Automation Finance/POC 12/19/2013 Discussion of Operating Budget Admin Costs HR 12/19/2013 DTS presentation on Fare Collection Study Board 12/19/2013 Formation of Fare Policy Permitted Interaction Group Board 12/ 19/2013 Four-Car train Presentation Board 1/ 16/2014 Update on Working Group Board 1/16/2014 Fare Study Workshop Presentation Board Automated Maintenance Yard Configuration change 2/13/2014 order Finance/POC 2/ 13/2014 Fare Study Workshop Presentation Board 4/24/2014 Presentation on Safety and Security Board 8/ 14/2014 Working Group Update Board 10/9/2014 Report on Fare Policy Permitted Interaction Group Board 10/23/2014 Report on Fare Policy Permitted Interaction Group Board 11/13/2014 Core Systems Fare Vending Machines Finance/POC Interim Report of the Fare Policy Permitted Interaction 3/12/2015 Group Board 5/21/2015 Update on Working Group Board Presentation on Sustainability and Photovoltaic 11/24/2015 Programs Board 11/24/2015 Update on Integrated Fare Systems Board Appendix H: Management Response Mr. Edwin S. W. Young, City Auditor Page 23 April 14, 2016 Date Description Group 2/18/2016 Formation of Financial Plan Permitted Interaction Group Board 21 . Fill the operations and maintenance position and other key vacancies. The recruitment should include a succession plan to fill key positions in the event of any unexpected departures. Response: HART conducted a very diligent and extensive search for the candidate with the required high level experience and background to fill this position. This thorough search has paid off and HART has hired an individual with outstanding credentials, who started on March 28. 2016, as the HART Director of Operations and Maintenance. Additionally, HART will be modifying the organizational structure to include a Deputy Director position under the Director of Operations and Maintenance. The expansion of this area and the reduction of other areas have already been identified in the staffing and succession planning since the beginning of the project. Daniel A. Grabauskas Executive Director and CEO Attachment: HART's Response to Audit Report Content Chapters 2, 3 4 Appendix H: Management Response HART’s Response to Audit Report Content – Chapters 2, 3, & 4 Note: Areas of the draft audit report highlighted in yellow are followed by HART responses in red text. Chapter 2: HART Needs to Improve Financial Management and Planning Summary Despite having a goal of completing the project on time and on budget, Honolulu Authority for Rapid Transportation’s (HART) project costs have increased $1.3 billion (25 percent) from the original estimate of $5.2 billion to an estimated $6.5 billion. HART’s processes can be improved to construct and complete the project more economically, effectively, and efficiently. Specifically, we found that HART’s financial and operating plans are not reliable or current; and HART’s financial plan has not been updated to reflect the rail project’s most current financial condition in spite of the significant cost increases. RESPONSE: HART is in the process of revising its Financial Plan in light of the extension of GET Surcharge through 2027. Prior to this, HART has continuously monitored, estimated and projected changes/updates to the plan. HART also regularly updates and reconciles its financial records and information. This includes but is not limited to: • • • • • • • • • • • • • GET revenue received and projected New Starts funding Drawdown Project cash balance Project cost and cash flow projections Project cost expended Estimate-at-completion projections Project contingency drawdown Project risk register Overall project progress Contractor Notice-to-proceed values Executed change orders Contracts awarded And others HART needs to strengthen its controls over financial information reporting to ensure data is complete and readily available from its Contract Management System (CMS); delay claims are adequately tracked, monitored, and reported; and pending utility agreements, contingency allowance figures, and general excise tax (GET) county surcharge forecasts are accurately reported. Absent the improvements, we anticipate additional shortfalls and cost overruns will occur. Response: The reality is external factors beyond anything under HART’s control are what caused HART’s revenue shortfalls and cost overruns. All the financial information Attachment 1: HART’s Response to Audit Report Content Page 1 of 62 Appendix H: Management Response reporting in the world would not have prevented external factors that caused the project’s funding deficit. HART does not agree with many of the assertions that the auditor made throughout this chapter to support the conclusions above. The OCA provides no data in its findings showing that the recommended improvements would prevent additional shortfalls and cost overruns. Background HART’s mission is to plan, design, construct, operate, and maintain Honolulu’s highcapacity, fixed guideway rapid transit system. To fulfill its mission, HART is responsible for completing the project on time and within budget and ensuring the design and actual construction of the project will facilitate the delivery of a safe, high quality, and costefficient service in the future. HART is also responsible for maintaining public trust through prudent and transparent use of financial, human, and environmental resources. Other HART responsibilities are to support the creation of mixed use, pedestrianfriendly, compact development along the rail line; pursuing partnerships with the private sector to create economic opportunities and generate income and cost savings for the rail transit system; and fostering an organization that is open, accountable, inclusive, and delivers better than promised results. As of November 2015, HART received $472.5 million of the $1.5 billion federal grant for the rail project. Per the 2012 Full Funding Grant Agreement (FFGA), HART and the city had to comply with the grant terms to receive the remaining balance of $1 billion. The exhibit below details the allocations by federal fiscal year. Attachment 1: HART’s Response to Audit Report Content Page 2 of 62 Appendix H: Management Response Exhibit 2.1 New Starts Grant Allocation (by federal fiscal year)1 Source: HART Monthly Progress Report, December 2015 based on data as of November 27, 2015 PMOC expressed issues and concerns in 2012 The FTA hired an independent consultant to monitor the construction project, identify problems, and to report deficiencies or concerns. The HART project management oversight consultant (PMOC) monthly reports discussed issues and concerns over the viability of HART’s operations. In the monthly reports, the PMOC questioned the following:  The adequacy of HART’s ability to “forecast costs for the existing design-build contracts.” It emphasized that it is critical that this issue be quickly corrected to demonstrate that the grantee has the Technical Capacity and Capability going forward.”2 1 The federal fiscal year runs from October 1 to September 30. The table covers federal funding only. Total rail project funding includes federal, state, and local funding sources. Federal funding includes $4 million out of $214 million of FTA Section 5307 Formula and American Recovery and Reinvestment Act (ARRA) funds. State funding includes the state half percent (0.5 percent) General Excise Tax (GET) county surcharge to fund the rail project. The GET county surcharge originally was to expire December 31, 2022, but was extended through December 31, 2027 to cover the additional project cost increases and revenue shortfall. HART estimates the five-year extension will generate revenue in the range of $1.2 billion to $1.8 billion. GET collections in FY 2015 totaled $1.522 billion. City funding includes general obligation bonds (debt financing) to fund construction of the rail project. In November 2015, the city council approved legislation that allowed the city to issue up to $350 million in general obligation commercial paper to fund rail project related improvements and equipment. HART also anticipates using the funds to cover its short-term cash flow needs. In 2015, city council resolution 15-18 eliminated the use of $210 million of Federal Section 5307 grant monies in the project’s financial plan to ensure the funds were only used for city transportation services (i.e. TheBus and Handi-Van services). 2 In the October 2012 monthly report, the PMOC noted that HART and the PMOC have held monthly breakout sessions to review the status of the forecast costs, schedule management, risk management, and cost containment measures. The report noted that these breakout sessions Attachment 1: HART’s Response to Audit Report Content Page 3 of 62 Appendix H: Management Response  The lack of “technical capacity and capability specific to project controls.3”  The sufficiency of contingency reserves; and  The need to develop, update, and implement secondary risk mitigation measures. Response: Throughout this report the OCA pulls little snippets out of thousands of pages of PMOC monthly reports to support claims about the PMOC’s opinion at the time. However, these snippets did not include the whole story conveyed in the PMOC’s reports. The OCA did not interview the PMOC and failed to gain an understanding of the purpose of the different types of PMOC reports. The PMOC’s role is to provide project oversight on behalf of the FTA. Thus expressing their concerns, on any aspect of the project, is part of their job. In 2012, the City was on the path to obtaining a FFGA. The PMOC issues OP reports prior to entering into a major phase of the project, which supersedes any information in the monthly reports. In OP 52, the PMOC expressed its professional opinion that HART demonstrated technical capacity to enter into a FFGA and specifically recommended including $644 million in total contingency. The FFGA was signed in December of 2012 and would not have resulted if not for their affirmation on the project’s readiness. OCA added footnote’s #2 and #3 in response to HART’s comments in the discussion draft rather than presenting all of the relevant facts in the body of the report. In December 2014, the HART’s chief executive officer (CEO) stated that the agency was facing a $600 million cost overrun and a $310 million revenue shortfall. The public statement notified the city council that project costs had increased and revenues were less than projected. Actions to resolve the shortfalls: In January 2015, the Hawai`i State Legislature introduced bills to extend the rail project GET county surcharge from December 2022 to December 2027. HART and city officials lobbied in support of the GET extension and the legislature passed the bill in May 2015. After the governor signed the bill in July 2015, the HART CEO reported to the HART Board of Directors on October 15, 2015 that the cost overruns had increased an additional $714 million, for a total of $1.3 billion. The exhibit below details the changes in estimated project costs and revenues. have resulted in increased confidence by the PMOC of the grantee’s ability to manage the project budget and schedule. 3 Project Controls are acts of project management staff in all aspects of cost, schedule, contract administration, and configuration management. In the February 2013 PMOC monthly report, HART acknowledged the situation and hired a new Project Controls Manager. Attachment 1: HART’s Response to Audit Report Content Page 4 of 62 Appendix H: Management Response Exhibit 2.2 Estimated Project Cost Increases and Revenue Shortfalls (dollars in thousands) Source: Office of the City Auditor (OCA) analysis based on HART Project Risks Update, December 18, 2014 and Project Cost Update, October 15, 2015. Impact of shortfalls: HART subsequently came under increasing scrutiny by policy makers, the local media, and the public. Throughout the project, local news reports drew public attention on the credibility of HART’s project cost information, and policy makers expressed concerns over the lack of detailed financial information provided by HART for decision-making purposes. The city council also expressed concerns related to the HART data, financial management and planning, decision making, contract administration, and postconstruction costs. As a result, the city council delayed approving the GET surcharge extension. After the Federal Transit Administration (FTA) issued a warning related to the lack of project funding, members of the city council approved the county surcharge in January 2016. In the Federal Transportation Agency letter to the Mayor, FTA stated the next $250 million federal installment will not be released until the city and HART provided a revised cost estimate and schedule, an updated financial plan, and a commitment of local funds to cover the increased cost estimates. The exhibit below details the timeline. Attachment 1: HART’s Response to Audit Report Content Page 5 of 62 Appendix H: Management Response Exhibit 2.3 Funding Milestones Source: OCA analysis based on various sources HART’s processes can be improved to more economically, effectively, and efficiently report project costs Regular reporting provides management with information necessary to make sound decisions and to be transparent and accountable to key stakeholders and the public. Our review found that HART can improve its financial management and planning by retaining and providing reliable project cost information to policy makers and decision makers. Response: HART provides extensive project cost information through many formats available to all stakeholders, policy makers, and decision makers. Project cost data is directly provided to the HART Board of Directors, FTA, PMOC, and local news outlets Attachment 1: HART’s Response to Audit Report Content Page 6 of 62 Appendix H: Management Response on a monthly basis. Further, the Monthly Progress Report, Balance Scorecard, and any other presentations made at public meetings are available on the HART website. Despite significant changes, HART has not regularly updated and reported accurate and reliable project cost information. Response: The consistency and reliability of HART’s financial data is validated by an independent financial audit conducted on HART’s financial statements annually which have resulted in an “unqualified opinion”, for the last 4 years. This means that the independent auditors’ have judged HART’s financial records and statements are fairly and appropriately presented. As a result, HART reports contain inconsistent project cost data which limit the overall usefulness of its financial planning, project cost, and funding information. More specifically, HART needs to:       Regularly update financial and operating information and plans; Provide reliable and consistent project cost information; Effectively track, monitor, and report on delay claims and related costs; Document and support utility cost increases and estimated cost overruns for project enhancements; Use specific and consistent factors in calculating and estimating escalation costs; and Properly report on GET county surcharge forecasts. Response: HART is in the process of revising its Financial Plan in light of the extension of GET Surcharge through 2027. Prior to this, HART has continuously monitored, estimated and projected changes/updates to the plan. HART also regularly updates and reconciles its financial records and information. This includes but is not limited to: • • • • • • • • • • • • • GET revenue received and projected New Starts funding Drawdown Project cash balance Project cost and cash flow projections Project cost expended Estimate-at-completion projections Project contingency drawdown Project risk register Overall project progress Contractor Notice-to-proceed values Executed change orders Contracts awarded And others Attachment 1: HART’s Response to Audit Report Content Page 7 of 62 Appendix H: Management Response Financial and operating plans are not regularly updated. An FTA grantee must demonstrate financial management and capacity to match and manage FTA grant funds and to cover cost increases and operating deficits. To ensure compliance with the FTA requirements, HART should follow best practices that ensure its financial and operating plans are regularly updated and are accurately reflected in its rail project financial reports. In the FTA 2015 triennial review4, FTA reported deficiencies in the project’s financial management and capacity. Specifically, HART’s financial plan did not demonstrate sufficient financial capacity to complete the project as currently planned. HART did not update its financial plans in light of the recent cost projections and current shortfall of GET surcharge receipts. Response: The Triennial Review was conducted during the week of February 2 – 5, 2015 and the final report was issued on April 9, 2015. The final report noted a deficiency for insufficient financial capacity based on the recent cost projections and current shortfall of GET surcharge receipts. The report also states that “HART will be revising its financial plan based on the current cost projections, updated revenue forecast, and local funding increase decisions by the State Legislature and City Council to be made this spring.” At the time of the triennial review, the State Legislature had not passed House Bill 134 extending the GET surcharge for five years. House Bill 134 was passed in May 2015 and signed into law as Act 240 on July 14, 2015. The Honolulu City Council did not pass City Council Bill 23 to adopt the GET surcharge extension until January 27, 2016. The triennial review team has been advised of the enactment of both Act 240 and Bill 23 and the noted deficiency has been cleared. More specifically, updating the financial plan was not discussed until April 2014. HART delayed communicating the potential cost increases to the city council until March 2015 after HART’s CEO announced a $910 million project deficit to the board in December 2014. The $910 million project deficit consisted of $600 million in increased costs, a 4 The United States Code, Chapter 53 of Title 49, requires the FTA to perform reviews and evaluations of Urbanized Area Formula Grant activities at least every three years. The site visit to the city occurred February 2 through 5, 2015. The final report was issued on April 9, 2015. As a corrective action to the finding, the FTA requested an updated financial plan by July 13, 2015. The plan should identify all funding sources for funding the HART project through completion within the FFGA scope and budget. HART submitted a draft financial plan to the FTA on August 14, 2015. The FTA closed the outstanding finding under the triennial review cycle on October 20, 2015. According to HART, the financial plan is being revised in light of the GET surcharge extension as of April 2016. Attachment 1: HART’s Response to Audit Report Content Page 8 of 62 Appendix H: Management Response $210 million reallocation of federal FTA Section 53075 funding to the Bus operations, and a $100 million GET county surcharge revenue shortfall. While there were indicators that led to the project deficit, project managers and staff in key positions stated they were unaware of HART’s fiscal condition until the December 2014 public announcement. As a result, corrective actions were not taken to ensure the FTA financial management and capacity concerns were satisfied. At the October 2015 board meeting, the HART CEO subsequently reported the project cost overrun had increased to $1.3 billion, and asked the city council and other policy makers to extend the GET county surcharge. The cost overruns are detailed in the exhibit below. Exhibit 2.4 Project Cost Estimates (dollars in thousands) (As of June 2012, December 2014, and October 2015*) Source: OCA analysis based on the FFGA financial plan, June 2012; HART Project Risks Update, December 18, 2014 and Project Cost Update, October 15, 2015. (*) This table excludes revenue shortfall of $210 million in New Starts Fund reduction and shortfall of $100 million in general excise tax (GET) county surcharge receipts. (**) Numbers may not add up due to rounding. Our analysis indicates the latest cost overrun figures are not reliable and will likely increase because the HART financial data and plans have not been updated to reflect the changes in the project costs. 5 Federal Section 5307 (49 U.S.C. § 5307) is a formula grant program for urbanized areas that provides capital, operating, and planning assistance for mass transportation. This program was initiated by the Surface Transportation Act of 1982 and became FTA's primary transit assistance program. The federal funds are apportioned to urbanized areas utilizing a formula based on population, population density, and other factors associated with transit service and ridership. Section 5307 is funded from both federal general revenues and trust funds, and is available for transit improvements for urbanized areas. Attachment 1: HART’s Response to Audit Report Content Page 9 of 62 Appendix H: Management Response Reliable and consistent project cost information is lacking Reporting inconsistent information can cause confusion for policy makers and the public. During our review of HART reports, we found reports that contained missing, outdated, and incomplete information. According to HART, not all contract information is populated in the HART contract management system (CMS). As a result, HART management and staff relied on CMS reports6 that were missing project cost information. For example, we found: Response: The auditor never obtained written assertions from HART on the subject matter included in their review. The auditor did not design the review with the proper level of assurance to come to this conclusion. Rather, the auditor audited for absolute assurance and the scope went from reviewing financial reports presented to government decision makers and the public, to covering internal ad hoc management reports. To meet the reporting requirements and needs of different HART departments and roles on the project several different CMS cost reports have been developed with different filters applied to the costing data. For example, many of the project controls reports are intended to report costs to the FTA, these reports do not include costs that are not eligible under the terms of the FFGA. The finance department requires reports of all contracts and agreements within the fiscal year, and Project Managers reports are limited to show only costs for contracts they have access to in the database. The reports are intended to be different and only the titles of the reports described their function. Most contract costs are directly entered by HART into CMS based on awarded contracts and agreements. However, the auditor selected several contracts that were not procured or administered by HART but paid with HART funds under MOU’s with other city departments. These contract costs and other Indirect Costs which are captured by the City accounting system and HART performs a reconciliation and true-up in CMS on a monthly basis to provide a complete as possible report of all HART costs.  Contract numbers in different reports contained different amounts. For example, in Exhibit 2.5, the contract balances in four different reports ranged from $2.6 million to $3.5 million. Response: HART’s Contract Management System (CMS) is a live database. Therefore the same report run at different times may yield different results if the underlying data had any changes to it. HART further advised the auditor not to compare two reports ran on different dates because it’s highly likely the data will have changed. HART information system staff reported that the CMS is a virtual, real time system, and reports extracted from the CMS will never be the same because the database contains real time changes. HART does not have any policies or practices to ensure HART data are consistent in all reports or that data are reconciled. 6 Attachment 1: HART’s Response to Audit Report Content Page 10 of 62 Appendix H: Management Response Additionally, HART explained to the auditor that it is not reasonable to compare reports that have different data criteria. In other words, the Forecast Cost Report and the HART Facts report exclude ineligible costs. The List of Awarded Contracts Summary as of June 22, 2015 includes ineligible costs. Cost reports provided to FTA excludes ineligible costs, therefore many cost reports by default filters those costs out. Despite being informed otherwise, the auditors are comparing four reports that have different data, fully aware the reports will not match. The auditor did not understand and is misrepresenting the consistency of the reports reflected in Exhibit 2.5. The table below summarizes the differences in the four reports being referenced: Report Run Date: FTA Ineligible Cost Included: List of Awarded Contracts Summary as of June 22,2015 8/13/2015 Yes List of Awarded Contracts Summary as of June 22,2015 11/5/2015 Yes Forecast Report to June 2015 7/2/2015 No HART Facts as of June 30, 2015 7/10/2015 No CMS Report Referenced: The auditors should acknowledge the reports contain distinctly separate data due to the reasons stated above or remove the statements and Exhibit 2.5 from the report.  CMS data was inaccurate. For example, HART executed a $100,000 professional services contract, but the expenditure report we reviewed indicated HART paid over $146,000 under the contract. HART staff later confirmed that there was a CMS error which excluded two contract amendments that totaled $250,000 from the CMS report. Response: The two (2) amendments were in CMS. They were only attached as files and HART inadvertently missed updating the costing fields which revises the contract sum. HART has since made the corrections in CMS. HART will also make the following improvement: All contracts, PO’s, agreements, change orders, amendments or any other cost documents will be given to HART Document Control to be filed and reconciled with CMS at the end of the departments workflow process. If the document has not been entered into CMS as required for reporting, the document will be returned to the department to enter it into CMS and distribute so everyone knows the action is completed and in CMS. Additional CMS training will be completed for Contract Managers and reports will be created and reviewed monthly. Attachment 1: HART’s Response to Audit Report Content Page 11 of 62 Appendix H: Management Response  Invoice data was incomplete. For instance, we found a missing invoice for $11,344 was not properly uploaded into CMS. Response: The report states that the invoice was missing from CMS and that the invoice data was incomplete. However, HART informed OCA this was not correct and OCA didn’t fix the error. The original invoice data was “posted” into CMS on 6/30/2015 and the contract administrator initiated a short pay on 8/28/2015. The amounts on the short pay form came directly from the amounts posted in CMS. As such, it would be impossible to process the short pay if the invoice data was not in CMS. The final signed (HART reviewed) invoice was not scanned into CMS. This was due to the urgency HART put on pulling the hard copies of the invoices for the auditors review in September 2015, just days after the invoice was paid. The .pdf of the original invoice was attached and HART’s review comments were electronically entered in the review tab of the invoice in CMS. Although the pdf of the final HART reviewed invoice was not attached into CMS, the data was “posted” in CMS and the hard copy was signed out to HART’s Internal Controls Analysts for the OCA team to review. The invoice was never missing.  Delay claim data was incomplete. More specifically, delay claims totaling nearly $64.2 million were not reported. Most notably, the $8.7 million delay claim 7 filed by Ansaldo Honolulu Joint Venture8 (eventually settled in October 2015) was not reported. Other delay claim data was outdated and not updated for two months. (See Exhibit 2.6, pending and possible changes). Response: This statement is inaccurate. HART tracks all change order and delay impacts via CMS and CHERPS. The auditor is characterizing a spreadsheet, which is a working document that was provided to them as backup for the delay cost impacts outlined in the December 18, 2014 cost update presented to the HART Board of Directors, as a management tool and comparing it against updated data out of CMS. This delay claim spreadsheet was never intended nor was it characterized to be used as a HART management tool to track any and all delay claims so it is incorrect to comment on the incompleteness of a working spreadsheet at a specific point in time.  The number of executed and pending utility agreements were inconsistent and unreconciled. More specifically, the CMS report showed a total of 48 utility contracts. In contrast, the tracking spreadsheet identified 54 contracts (including 40 active utility contracts, 7 closed contracts, and 7 pending contracts). 7 According to HART, the $8.7 million delay claim filed by Ansaldo Honolulu Joint Venture (See Exhibit 2.6, Core Systems Design Build O&M Contract pending claim) was included in the $10$20 million escalation costs increase (See Exhibit 2.7). However, HART was unable to provide details to support the $10-$20 million cost estimate. 8 AnsaldoBreda and Ansaldo STS became a part of the Hitachi Group Company on November 2, 2015 and November 3, 2015, respectively. AnsaldoBreda is now Hitachi Rail Italy. Attachment 1: HART’s Response to Audit Report Content Page 12 of 62 Appendix H: Management Response Response: Due to the departure of several key personnel previously tasked with managing utilities contracts, HART was in the process of verifying and reviewing all utility contract information. This review has since been completed and the CMS will be updated. The tracking spreadsheet was a working tool that was used to reconcile the data and not an official reporting tool.  Six utility agreements were missing in a key report. Six pending contracts valued at $107.9 million were excluded from the HART CMS forecast report. Response: The key report referred to is the Forecast Costs by Contract. The Forecast Costs by Contract report contains only executed contracts and pending changes on the executed contracts. The six pending contracts were not executed and therefore were properly excluded from the Forecast Costs by Contract report.  Differences in the contingency balance did not match external PMOC monthly reports. HART balances were higher than the figures reported by the PMOC. Appendix D compares the differences between HART’s contingency balances with the figures reported by the PMOC. The differences between the HART balances and the PMOC balances ranged from $149 million to $254 million. Response: The auditor is comparing (2) distinctly different contingency balances represented in the HART Monthly Progress Report and the PMOC Monthly Report. The HART Monthly Progress Report provides an actual remaining balance of contingency which takes the original contingency allocation of $644M and subtracts all reductions of contingency due to executed change orders or transfers of contingency into project scope. The details of these transfers are provided in Appendix B of the report. The PMOC Monthly Report provides a projected remaining balance of contingency which takes the actual remaining balance of contingency then subtracts all changes identified on the Forecast Cost report which results in a forecasted contingency balance. The auditor is misrepresenting the delta between the HART contingency balance and the PMOC contingency balance as inconsistent project cost information. The auditors should not be comparing an actual value to a projected value and making a conclusion about the consistency of the data because they are different. This comment should be removed and Appendix C should be supplemented with context that describes the two balances are different representations.  State of Hawai`i GET balances did not match PMOC monthly reports. Appendix E compares the differences between the actual quarterly GET receipts with amounts reported by the PMOC. Although there may have been a timing difference between collection and reporting, the variances we found were significant. Reporting differences between the HART GET receipts and the PMOC reports ranged from $25 million to $492 million. Response: HART disagrees with this assertion for the following reasons: Attachment 1: HART’s Response to Audit Report Content Page 13 of 62 Appendix H: Management Response  All GET surcharge receipts have been properly and accurately reported in the City’s financial accounting system and agree to the amounts remitted by the State of Hawaii Department of Taxation. These receipts have been correctly reported on HART’s Monthly Progress Reports.  GET surcharge receipts are remitted by the State Department of Taxation (DoTAX) on a quarterly basis, and is received on the last day of the month following the end of each quarter. Appendix E is incorrect since it does not properly align the quarterly reports to the quarter when the GET surcharge is actually received and reported. The GET Surcharge revenue information in the PMOC reports lag what is received and recorded in the City’s financial system C2CHERPS. To illustrate: the GET surcharge for the quarter ending June 30, 2013 would be received on July 31, 2013 and would be reported in the quarter ending September 30, 2013.  The Project Management Oversight Committee (PMOC) is an independent entity which prepares its own monthly reports for the Federal Transit Administration. The GET surcharge information reported by the PMOC in their independent monthly report was compiled by the PMOC and may not reflect what HART has been reporting on its monthly progress reports. Any errors in the PMOC’s monthly report should be directed to the PMOC and not to HART. Exhibit 2.5 Contract Balance Comparison (dollars in millions) Source: OCA analysis of HART’s List of Awarded Contracts Summary as of June 22, 2015, Forecast Report for June 2015, and HART Facts as of June 30, 2015. Attachment 1: HART’s Response to Audit Report Content Page 14 of 62 Appendix H: Management Response Exhibit 2.6 Incomplete Delay Claims Summary (dollars in thousands) Source: OCA analysis based on HART Project Delays Cost Summary Spreadsheet, August 28, 2015, June 26, 2015, January 31, 2015, and December 18, 2014; CMS forecast report, September 25, 2015. (*) Numbers may not add up due to rounding. Project cost estimates lacked supporting documentation Cost estimating, by nature, is imprecise. Therefore, it is important to develop cost estimating methodologies and document key assumptions for the estimates. The Guide to the Project Management Body of Knowledge (PMBOK Guide) states that supporting documentation should provide a clear and complete understanding of how the cost estimate was derived. Supporting detail should include:  A description of the project’s scope of work.  Documentation of the basis of the estimate (i.e., how it was developed);  Documentation of all assumptions made;  Documentation of any known constraints; and  An indication of the range of possible estimates. We did not find documentary evidence to support $450 million of the additional cost increases as it relates to the above basic requirements. Attachment 1: HART’s Response to Audit Report Content Page 15 of 62 Appendix H: Management Response Response: Per the auditor’s request, HART staff provided supporting documents including but not limited to; Forecast and cost reports from our Contract Management System, Independent Cost estimates (ICE), Change Order reports, market analysis, and other documents which supports the Project Cost Estimate presented in October 2015 to the HART Board of Directors. While it is unclear how the auditor comes up with their $450 million value, HART contends the report does not fairly represent the supporting documentation provided. HART staff found the auditors struggled to understand that in addition to definitive cost estimates and budget estimates, Rough Order of Magnitude (ROM) estimates are also utilized in developing the overall Project Cost Estimate. ROM estimates, due to the lack of details available are not expected to have the same level of documentation provided with an ICE for example; however they seem to characterize any value with less supporting documentation than would be expected for a definitive estimate as insufficient to support a cost projection which demonstrates their limited understanding of sound Project Management practices. Utilization of ROM estimates are considered a best practice in Project Management for establishing a cost basis for scopes of work that are early in the stages of development or roughly defined. There are several elements of the Project Cost estimate where limited information was available and ROM estimates were provided such as; cost escalation caused by prior delays and related to future construction packages, alternative resolutions to relocate 138kV HECO lines, and various project enhancements being contemplated just to name a few. HART did communicate to the auditors which cost elements were based off ROM estimates, definitive cost estimates, or were budget estimates. HART can only assume the utilization of ROM values in the Project Cost Estimate is the basis for their statement because they did not provide any details in the report nor did they communicate anything to HART to substantiate their finding regarding the $450M being referenced. HART cannot demonstrate it has an effective method of tracking, monitoring, and reporting on delay claim costs HART stated delay claims totaled $190 million and were attributed to lawsuits, escalation costs, and other delays. Of the $190 million, $146 million were executed through change orders funded by the project contingency reserves. We found that the remaining costs of approximately $44 million were either unsupported or the claim amount changed because HART does not have an effective method in tracking, monitoring, and reporting on delay claim costs. Exhibit 2.7 quantifies the delay claim costs. Response: HART has been consistent and clear that the $190M for delays attributed to lawsuits, escalation, and other delays is an estimate and negotiations to settle these costs with the contractors are ongoing. The escalation element remains the most variable factor in the overall estimate for delay cost. Negotiations to come to an agreed upon settlement for escalation with the contractor were unsuccessful so HART proceeded with executing a Attachment 1: HART’s Response to Audit Report Content Page 16 of 62 Appendix H: Management Response “Provisional” change to the contractor where HART will pay only on actual escalation incurred on eligible cost. The benefit to HART is that the agency will pay only what the contractor is owed for escalation as opposed to a settlement value that could have potentially been higher than actual escalation incurred. However, until all of these costs are approved there will be a potential for the total claim amount to change. Exhibit 2.7 Reporting Comparison of Delay Claim Costs Source: OCA analysis based on HART Project Risks Update, December 18, 2014; Project Cost Update, October 15, 2015; and HART Project Delays Cost Summary, August 2015 In the December 2014 board meeting, HART provided a one-page Project Delays Cost Summary (spreadsheet) in support of the $190 million in delay claim costs. As of October 2015, HART’s spreadsheet remained unchanged. However, we identified changes that were not updated because HART does not separately track and monitor delay claim costs. By updating the claim information, we found:  $12.2 million in additional potential delay costs related to the West O`ahu/Farrington Highway Guideway Design-Build contract9 that increased the delay claim from $6.8 million to $19 million.  $1 million in additional potential delay costs related to the Kamehameha Highway Guideway Design Build contract (Escalation Due to Schedule Impacts) that increased the delay claim from $3.5 million to $4.5 million.  $825,000 in overstated delay claims related to the CMS forecast report. The report removed an $825,000 delay escalation claim for the maintenance and storage facility, but the update was not reflected in the tracking spreadsheet and not entered for two months.  $670,184 in overstated delay claims in the tracking spreadsheet. The tracking spreadsheet identified a $7.5 million delay escalation claim for the West O`ahu Farrington Highway Guideway Design-Build contract, but the forecast report showed $6,829,816. 9 West Oahu/Farrington Highway Guideway Design-Build contract (Delay of NTP 2, 3, & 4 – Escalation Costs) Attachment 1: HART’s Response to Audit Report Content Page 17 of 62 Appendix H: Management Response  We also did not find details to support litigation costs of approximately $6.9 million because HART tracks them separately as ineligible project costs. Response: HART updates its project cost information monthly to reflect the most current information. This entire section, including sub-bullets, mischaracterizes a spreadsheet that was identified as a working document as a management tracking tool and identifying variances from the CMS. The auditor requested support for the cost impact of delay claims highlighted in the December 18, 2014 presentation. HART provided a spreadsheet which is a working document that tracks delay costs as a snapshot at a particular point in time. That spreadsheet was never intended to be used as a HART management tool to track all delay claims. All projected changes and claim values are tracked via CMS and CHERPS which are the agency’s management tracking tools for cost data and are updated monthly. Despite our finding, the manager of Project Controls stated that the spreadsheet was never intended to be used as a management tracking tool. The Project Controls manager also stated that HART tracks and monitors delay claim costs using CMS and C2HERPS. HART provided CMS forecast reports to show how delay claims are tracked and monitored. We found that these reports are inclusive of all contracts and change orders. In one report, we identified over 490 line items that consisted of contracts with executed change orders; pending, probable, and possible changes; and claims in dispute. We question how HART can accurately identify, track, monitor, and report on total delay claim costs when these reports do not track them as separate costs. Response: HART clearly understands the information as presented in CMS and C2HERPS reports. HART acknowledges the auditor needed further explanation. The documentation should not be disregarded simply because it was not understood by and not in a format desired by the auditor. HART lacked adequate support for $120 million in utility costs Utility relocation costs were not included in the $910 million project deficit reported by HART in December 2014. In October 2015, however, HART provided project cost updates that showed utility costs increased from $50 million to $120 million. See Exhibit 2.8. Attachment 1: HART’s Response to Audit Report Content Page 18 of 62 Appendix H: Management Response Exhibit 2.8 Utility costs increase (dollars in millions) Source: OCA analysis based on HART Project Risks Update, December 18, 2014 and Project Cost Update, October 15, 2015. HART explained that they used the Rough Order of Magnitude (ROM) 10 methodology to value the utility cost estimates. Although projects in the early phase generally have limited information to produce quality cost estimates, at a minimum, we expected a description of the estimate, scope and assumptions, data sources, estimating methodology and rationale, risk analysis results, and a conclusion about whether the cost estimate was reasonable. When we requested supporting documentation for how HART reached the total cost estimates, we were told that detailed estimates were still being developed and were not available for review. HART eventually provided a draft three-page document that listed five options that totaled $99 million. The cost estimates did not provide detailed documentation describing how it was derived; showed no evidence of any review or approval; and did not identify the factors used to estimate the $120 million in utility costs. As a result, we question the credibility of the estimate. Response: The auditors were informed that the $120M (increase of $70 million) for utility costs was based off of rough order of magnitude (ROM) estimates and that detailed estimates are still being developed. Subsequent to providing that information, HART provided limited details in draft form to support the cost for relocating the high voltage HECO lines which represents most of the $120M cost, but was never intended to support the full value. For example, the ROM estimates were developed and carried in the budget to indicate the expected resolution value of HECO clearance issues, but do not specify whether the solution will be to relocate all aerial overhead transmission lines on new poles or to underground these same transmission lines. As of the date of 10 According to HART’s Cost Estimating Procedure, a Rough Order of Magnitude Estimate (ROM) is an estimate developed to facilitate project budget and feasibility determinations. The order of magnitude estimate information is based on parametric units (e.g. route feet, lane miles, gross square feet, number of parking stalls) and other quantifiable data. Pricing is based on historical cost caps that are adjusted for project location, size or capacity differences, and cost escalations. The GAO Cost Estimating and Assessment Guide states that a ROM is developed when a quick estimate is needed and few details are available. Based on historical information, it is typically developed to support what-if-analyses, and can be developed for a particular phase or portion of an estimate to the entire cost estimate. The analysis is helpful for examining differences in alternatives to see which are the most feasible. Because it is developed from limited data and in a short time, a ROM analysis should never be considered a budget-quality cost estimate. Attachment 1: HART’s Response to Audit Report Content Page 19 of 62 Appendix H: Management Response this response, negotiations continue with HECO to determine the most appropriate and economical solution. Further, HART never characterized this cost as anything more than ROM estimates, yet the auditor is representing their findings as if they were definitive estimates. The definition of a ROM estimate, as provided in their footnote, is to facilitate the project budget and determine feasibility, and to examine alternative approaches to an issue. In short, these are costs identified in the relatively early stages of development. It is premature to characterize these ROM values as “not credible” before they have been fully developed into a definitive cost estimate. The auditors should revise their statement to be reflective of the level of estimate that was provided. We subsequently discovered the project manager responsible for the utility contracts did not know about the $120 million cost increase. Response: The increase was actually $70 million (the change is from $50 million to $120 million as stated in this same section of the report, three paragraphs earlier and shown in Exhibit 2.8 above as well). HART informed the OCA of this error in the first draft, however, OCA never made corrections as requested. Secondly, in the statement above, the auditor infers that the PM didn’t know about the $70 million cost increase based on a single interview with a project manager (PM) because she wouldn’t express an off-the-cuff opinion based upon the contents of a presentation to the HART Board in which she did not attend nor prepare. During the interview, on November 24, 2015, the auditor asked the PM if she agreed with the contents of the Project Cost Update Presentation to the board in October 2015. The PM explained that she hadn’t attended the board meeting and was not sure what numbers or budget had been presented to the board. The auditor relied upon the oral questioning without providing documentation for the PM to review prior to opining whether she agreed with the details of a presentation to the Board. $46 million in estimated cost overruns for project enhancements were not supported Like the utility relocation costs, project enhancements were not discussed when HART reported the $910 million cost overrun and revenue shortfall in December 2014. In October 2015, however, HART reported project enhancements costs that increased from $75 million to $130 million. The additional $55 million cost estimate increase included $35 million in public highway improvements and $20 million in additional escalators. The HART project controls manager told us that project enhancements were created to present change orders differently to stakeholders and the public. (See Exhibit 2.9 below.) Attachment 1: HART’s Response to Audit Report Content Page 20 of 62 Appendix H: Management Response Exhibit 2.9 Project Enhancement Costs Increase (dollars in millions) Source: OCA analysis based on HART Project Risks Update, December 18, 2014; Project Cost Update, October 15, 2015; and CMS forecast report, September 2015. We found that HART could not support $46 million of the $130 million in project enhancement cost estimates. Discrepancies existed between the cost estimates presented to the HART board and the documentation supporting the estimates because HART did not follow its cost estimating procedures. For example, we did not find detailed descriptions or support for:  $18.1 million for public highway improvements  $5.2 million for the emergency backup generators  $2.7 million for fare collection We questioned the estimated $20 million for additional escalators. HART provided a handwritten proposed costs document that showed two estimates which varied by roughly $6 million with a low of $17 million and a high of $25 million. Higher estimates can overstate the total project costs while lower estimates can potentially result in cost overruns. Response: All of the costs on the ROM were broken down by component of cost and the ROM was reviewed by the General Engineering Consultant. The ROM met all of the documentation requirements. It was hand written because it was a preliminary estimate of cost (which is what a ROM really is), required for budgeting purposes. The document cannot be considered wrong merely because it was not in the format that the auditor wanted to see it in. According to the GAO Cost Estimating and Assessment Guide, it is imperative that all assumptions are documented so that management fully understands the conditions the cost estimate was structured on. The GAO further states that failing to do so can lead to overly optimistic assumptions that heavily influence the overall cost estimate, to cost overruns, and to inaccurate estimates and budgets. While cost estimates are only estimates, absent detailed information, we were unable to identify all the cost elements included in the total amount. More importantly, we could Attachment 1: HART’s Response to Audit Report Content Page 21 of 62 Appendix H: Management Response not assess the reasonableness and appropriateness of the methodology and assumptions used to develop some of the cost estimates. Response: For budgeting purposes, some of the estimates were adjustments to the EAC calculation which requires minimal documentation as recommended in the GAO Best Practice Manual. The PMBOK actually allows EAC to be formulaically driven and requires no formal documentation. As HART identifies the contracting mechanism in which to solicit services to design and/or build these particular components of work (highway improvements, emergency backup generators, escalators, fare collection, etc.), detailed estimates (Independent Cost Estimates (ICEs)) are generated prior to the release of that particular solicitation (future work) or negotiated change order (current work). The auditor was provided several examples of HART’s ICE format and the level of detail provided meets industry-standard estimating procedures and quality standards. Because these components of work are included in a contract package (the Airport Guideway and Stations contract or the West Oahu Farrington Highway contract for example), the detailed ICEs are included within those packages. Escalation cost11 estimates were not calculated by using a specific factor HART increased its escalation cost estimates by $195 million, from $45 million in December 2014 to $240 million in October 2015. HART attributed the increase to extraordinary market conditions. During the audit, HART staff was unable to explain the methodology used to support these cost estimates. We were told that HART did not use a specific factor to calculate the $240 million in escalation costs and that these costs are subsets of the total escalation within the project costs. HART claims that it has procedures and methodologies in place to forecast escalation that is based upon historical data as well as using the latest bid results, other agency procurement results, market studies, and independent economic reports, etc. When we asked for evidence to substantiate the $240 million escalation cost estimates, however, there was no documentation to support how the $240 million was derived. HART was also unable to provide a detailed breakdown of escalation costs for the total $6.5 billion project cost. Consequently, we were unable to verify HART’s methodology to forecast escalation costs and the reasonableness of the additional escalation costs because the amounts were not calculated by using a consistent and specific factor. The exhibit below shows the unsupported cost increase. 11 Escalation costs represent cost increases projected by a contractor or HART when estimating work to be completed at a time in the future. Attachment 1: HART’s Response to Audit Report Content Page 22 of 62 Appendix H: Management Response Exhibit 2.10 Escalation Costs Increase (dollars in millions) Source: OCA analysis based on HART Project Risk Update, December 18, 2014; and Project Cost Update, October 15, 2015. Response: HART has procedures and methodologies in place to prepare cost estimates and forecast escalation (a difficult task given ongoing construction market fluctuations in Honolulu), based upon historical data as well as using the latest bid results, other agency procurement results, market studies, and independent economic reports (e.g., Rider Levett Bucknall Quarterly Reports). The statement that “HART staff was unable to explain the methodology used to support these cost estimates” is not a true statement. HART provided detailed explanations and extensive documentation to the auditor. HART underreported GET county surcharge forecasts In 2014, HART projected a GET revenue shortfall in the range of $80 to $100 million and attributed the shortfall to a coding error by the State of Hawai`i Department of Taxation. The error resulted in a $9.9 million distribution error which was compounded annually over 10 years for a total of $100 million. We requested information related to the $100 million GET revenue shortfall and analyzed the spreadsheets provided by HART. The HART spreadsheets showed how GET county surcharge receipts were tracked and how revenues were projected. We found that HART’s revenue forecasts were higher than reported. Consequently, HART underreported its projected shortfall amount by approximately $41 million. HART executive management knew the amount was higher than the $100 million reported, but did not report its projections accurately to the board in December 2014. Response: HART disagrees with this statement. The spreadsheet HART provided to the City Auditor was a comparison of actual GET surcharge receipts to the amounts per the financial plan of the Full Funding Grant Agreement. The City Auditor’s recalculation of the “plan” amount based upon corrected information received from the Hawaii State Department of Taxation (DoTAX) resulted in a reduction of $141 million to the original “plan” amount. In HART’s analysis of the impact of the DoTAX error to the planned GET surcharge receipts, it reviewed the actual GET amounts received to date, which were increasing year-over-year. It also considered economic information on Hawaii’s economy as Attachment 1: HART’s Response to Audit Report Content Page 23 of 62 Appendix H: Management Response reported by the Council of Revenues, State Department of Economic Development and Tourism, and the Hawaii Tourism Authority which all indicated that Hawaii’s economy was strong and tax revenues would continue to grow going forward. Based on its analysis, executive management conservatively reduced the estimated impact of the DoTAX error by $41 million, which represented only 1.2% of the original $3.291 billion plan amount. Therefore, the decision by HART executive management was a financially prudent updated estimate of GET Surcharge revenue going forward. Improved Financial Management and Planning Are Needed While the FTA does not require submittal of updated financial and operating plans after the award of a full funding grant agreement, it does retain the right to ask for updated financial and operating plans if any significant changes to the project occur after the funding grant agreement is signed. According to the FTA Guidance for Transit Financial Plans, sound financial planning ensures the financial health of transit agencies and affects the quality of service provided. Financial and operating plans serve as a fundamental tool for management and policy makers to make critical decisions, especially for a project of this magnitude. Consistent with best practices, the plans should therefore be regularly updated to reflect the most current financial condition of the project. A HART executive stated that the outdated plans are not an impediment to HART operations. According to the executive, HART is fulfilling its reporting requirements; updating the financial and operating plans are contingent upon the passage of the GET county surcharge extension12 by the city; and that FTA has not established a specific timeframe requirement for the financial updates. Another HART executive stated a formal update requires HART to go through a lengthy process that is subject to the review and approval of key stakeholders, including the HART Board of Directors, the project management oversight consultant, and the FTA.13 We believe this reasoning should not delay HART’s efforts to update its financial and operating plans. Without current financial and operating plans, HART management, policy makers, and decision makers will be unable to make cost-effective decisions to ensure the project is completed efficiently, effectively, and economically. Response: The FTA instructed HART to update the Financial Plan. The deficiency related to the lack of projected funds. Therefore, the FTA wanted a revised Financial 12 At the time of our interview, the GET county surcharge extension was still subject to the adoption of bill 23 by the city council. 13 According to HART, the FTA instructed the agency to submit a revised financial plan because of a deficiency related to the lack of projected funds. In order to revise the plan, HART noted that it sought to extend the GET surcharge as a viable finding source. HART also noted that the FTA held off any further action until HART could demonstrate that it had the financial capacity to complete the project. As a result of the GET extension, the FTA has scheduled a full budget review of HART’s updated financial plan at the next risk refresh meeting on March 30, 2016. Attachment 1: HART’s Response to Audit Report Content Page 24 of 62 Appendix H: Management Response Plan. In order to revise the financial plan, HART needed a viable funding source. HART sought to extend the GET and informed the FTA of this. The GET extension would provide sufficient funding to complete the project. The process of getting the extension was lengthy and final city approval took nearly a year from City Council introduction on 3/12/2015 (City Council approved the extension on 1/28/2016) through Mayoral approval on 2/1/2016. FTA held off of any further action until HART could demonstrate that it had the financial capacity to complete the project. As a result of the extension being approved, the FTA has scheduled its next risk refresh (that includes a full budget review) for March 30, 2016 and HART is able to submit for review an updated Financial Plan. Attachment 1: HART’s Response to Audit Report Content Page 25 of 62 Appendix H: Management Response Chapter 3: HART Needs to Improve Project Management and Contract Administration Summary Project management and contract administration controls can be improved. More specifically, Honolulu Authority for Rapid Transportation’s (HART) Project Management Plan (PMP) and its subsidiary plans are outdated and unreliable as decision-making tools. HART made concessions to a single contractor; did not perform quantitative analysis to justify a major decision to repackage a bid for nine stations in the Westside Stations Group; and paid $1.5 million in stipends to unsuccessful bidders without knowing the bidders’ actual costs. HART is also paying for vacant office space. Contract administration controls need to address invoice payments, procurement file documentation, and prevent improper payments. Background On December 18, 2014, HART’s chief executive officer (CEO) reported to the HART Board of Directors that the total project costs will increase to $5.8 billion, an increase of $600 million in additional costs and a $310 million funding shortfall. HART attributed project cost increases to three separate events: lawsuits that resulted in delay claims, higher than expected bid for the construction of the nine Westside stations, and the unfavorable general excise tax (GET) county surcharge revenue receipts. Ten months later, on October 15, 2015, total project costs increased to $6.5 billion. The exhibit below provides details. Attachment 1: HART’s Response to Audit Report Content Page 26 of 62 Appendix H: Management Response Exhibit 3.1 Factors that Contributed to the Project Cost Increase and Revenue Shortfall (dollars in thousands) Source: Office of the City Auditor (OCA) analysis based on HART Project Risks Update, December 18, 2014 and Project Cost Update, October 15, 2015. (*) Numbers may not add up due to rounding. Construction cost overruns are not unusual Our comparison of construction costs for other capital rail projects indicated cost overruns frequently occur. The following data table features a comparison of capital costs among 20 heavy and light rail projects.  The first 8 services (Atlanta, Baltimore, Buffalo, Miami, Pittsburgh, Portland, Sacramento, and Washington, D.C.) were rail projects that initially established heavy and light rail services in the respective urban areas. While one project was completed under budget, the others had cost overruns that ranged from 13 percent to 83.1 percent  The remaining 12 heavy and light rail projects were extensions and built to integrate the new projects into already established heavy rail transit services. One heavy rail project was completed under budget. The other 11 projects had cost overruns that ranged from 3.3 percent to 35.8 percent. Attachment 1: HART’s Response to Audit Report Content Page 27 of 62 Appendix H: Management Response Exhibit 3.2 Capital Costs Comparison of Heavy and Light Rail Projects Source: Office of the City Auditor (OCA) analysis based on information obtained from the Urban Transit Rail Projects: Forecast Versus Actual Ridership and Cost, 1990, prepared by Dr. Don H. Pickrell for the Urban Mass Transit Administration and the FTA’s Predicted and Actual Impacts of New Starts Projects, 2003 and 2007. (*) All dollar amounts shown represent adjusted real values. (**) The FFGA program was established after the projects featured in the 1990 study. Additional HART rail project cost overruns and shortfalls are likely One of HART’s stated goals is to preserve the stewardship of resources by maintaining public trust through the prudent and transparent use of financial, human, and environmental resources. HART can demonstrate good public stewardship by wellsupporting its management decisions. In addition to deficiencies in financial management and planning, we found shortcomings that could impact the project’s long-term financial viability, increase the likelihood of additional cost overruns, and reduce the funds available to the rail project. Specifically, we found:  HART does not have an adequate contingency reserve.  Cost controls were insufficient to control cost increases. Attachment 1: HART’s Response to Audit Report Content Page 28 of 62 Appendix H: Management Response  Updating the project management plan, including the subsidiary plans, was never made a priority.  Inappropriate concession (retainage payments) with a contractor was made.  Cost estimates and potential savings were lacking in the decision to repackage the Westside Stations Group.  HART lacks policy and procedures on administering stipends and has paid $1.5 million without documentation of unsuccessful bidders’ actual costs.  HART is paying for vacant office space.  Contract administration needs to be improved. The project management oversight consultant (PMOC) meets with HART management and staff monthly to discuss increased costs and to ensure that cost issues are proactively addressed. In December 2014, the PMOC reported it provided numerous cost mitigation recommendations that HART should consider implementing. Despite the recommendations, PMOC noted that minimal cost containment measures had been accomplished by HART and the trend of minimal cost containment was alarming. Response: Regarding the implementation of primary and secondary mitigation measures, HART and the PMOC have discussed dozens of options each and every month and continue to do so as a matter of standard practice. The audit report cites the December 2014 PMOC Report wherein is says “the PMOC reported that it provided numerous cost mitigation recommendations that HART should consider implementing.” HART agrees that the PMOC and HART discussed several cost mitigation measures during the fall of 2014 after the WSSG bid, most of which had already been developed by HART before the PMOC provided their renewed list of items. However, many of these items were discounted or eliminated due to several reasons (a deletion of scope, e.g., deleting Ho’opili Station, is considered a breach of the FFGA contract; eliminating architectural features, e.g., the embossed guideway columns at the stations would be a violation of the promises made during public design outreach sessions; or removing all escalators at all stations was considered too great of an impact to the traveling public to have been worth the cost savings). The PMOC Report did not cite specific cost mitigation measures that HART refused to implement – because this never happened. To the contrary, even now HART and the PMOC continue to look for ways to mitigate rising costs even after the approval and implementation of the GET extension. HART does not have an adequate contingency reserve Normally, the amount of contingency required for a project decreases with the project’s progress. Over time, as costs become more definitive, the contingency amount should Attachment 1: HART’s Response to Audit Report Content Page 29 of 62 Appendix H: Management Response decrease. However, HART increased its allocated and unallocated14 contingency funds by $539.4 million because it does not have an adequate contingency reserve.  In June 2012, HART reduced its original $866 million contingency by $222 million to $644 million after it submitted its 2012 Full Funding Grant Agreement (FFGA) financial plan to the Federal Transit Administration (FTA). Response: When HART reviewed the first draft of this report HART informed the auditor that this statement mischaracterized the change in contingency in June 2012. However, the auditor refused to incorporate HART’s corrections. Normally, the amount of contingency required for a project decreases with the project’s progress. The $222 million reduction in contingency that the auditor mentioned was actually a routine reduction in contingency required in line with the normal evolution of a New Starts project moving from Final Design to entering an FFGA where a reduction of contingency is expected. As required of a potential new starts grantee the FTA will perform a risk refresh to review the scope, schedule, and cost documentation to assess their ability to manage the budget and schedule of the project before entering into a FFGA. This process took place from April 2012 through September 2012, as required by the FTA the PMOC provided a comprehensive Risk and Contingency Report under Operating Procedure (OP) 40 where the PMOC states among many other instances supporting HART’s cost estimate, “The grantee’s total project estimate of $5,122 million, including $644 million in total contingency and $173 million in finance charges, is acceptable to support an FFGA.”  In July 2012, HART established a $76.1 million Known Changes15 contingency account to separately fund costs that would have been covered by unallocated contingency reserves.  In October 2012, the PMOC noted that “significant contingency reduction occurred, to a point where contingency was below accepted control levels” upon HART’s submittal of the updated cost estimate in 2012 to support the FFGA application. Subsequently, the PMOC acknowledged that HART “has implemented efforts to recover contingency levels through cost reduction measures, value engineering, and revised project delivery strategies.” Unallocated contingency provides a funding source to cover unknown but anticipated additional project execution costs and uncertainty due to risk factors such as unresolved design issues, market fluctuations, unanticipated site conditions and change orders. It also covers unforeseen expenses and variances between estimates and actual costs. 15 HART tracks Known Changes separately from the Project contingency established under the FFGA. Known Changes are executed through budget transfers. According to HART, Known Changes are recognized as project scope and not contingency. HART management explained that the Known Changes were identified as pending changes that were subject to final negotiations with contractors. Upon our review of Known Changes, we found that HART used these reserves to fund change orders that included a $20.1 million delay claim, $6.8 million in non-rail escalation and rail mark-up costs, and $2.6 million budget transfer into allocated contingency, contrary to the intended use of this fund. 14 Attachment 1: HART’s Response to Audit Report Content Page 30 of 62 Appendix H: Management Response Response: When HART reviewed the first draft of this report HART informed the auditor that this statement mischaracterized the PMOC’s comments made in the October 2012 Monthly Report. However, the auditor refused to incorporate HART’s corrections. The auditor is misrepresenting the PMOC’s comments made in the October 2012 Monthly Report and inaccurately describing the genesis of the “known” changes contingency. In the October 2012 Monthly Report the PMOC states “Upon submittal of the updated cost estimate by the grantee in spring 2012 to support the FFGA application, it was observed that significant contingency reduction occurred, to a point where contingency was below accepted control levels.” As required of a potential new starts grantee the FTA will perform a risk refresh to review the scope, schedule, and cost documentation to assess their ability to manage the budget and schedule of the project before entering into a FFGA. This process took place prior to April 2012 even though the auditors are characterizing these comments as being made in October 2012. Subsequent to this Risk Refresh, HART implemented cost containment strategies, value engineering measures, and improved project controls relating to cost and schedule as noted in the same October 2012 PMOC Monthly report. Prior to this report, in September 2012, as required by the FTA the PMOC provided a comprehensive Risk and Contingency Report under Operating Procedure (OP) 40 where the PMOC states among many other instances supporting HART’s cost estimate, “The grantee’s total project estimate of $5,122 million, including $644 million in total contingency and $173 million in finance charges, is acceptable to support an FFGA.” The auditor’s allegation that HART’s contingency reserve was insufficient at the time the FFGA was executed is false and any comments made implicating otherwise should be removed from the report.  From May 2013 through July 2013, the PMOC expressed concerns “with the adequacy of the remaining contingency given the anticipated costs due to the project delays.” In August 2013, the PMOC noted a concern on “whether there is sufficient contingency remaining, given the status of the project.” Response: When HART reviewed the first draft of this report HART informed the auditor that this statement mischaracterized the PMOC’s comments made in the Monthly Reports referenced. However, the auditor refused to incorporate HART’s corrections and changed the dates of the reports referenced in the final report draft to make it more difficult for HART to respond to the final draft. The auditor is misrepresenting the statements made from the various PMOC Monthly Reports from April 2013 to August 2013 and the recommendation referenced from the November 2014 as support for their assertion HART underfunded contingency in 2012. The auditor correctly points out the PMOC’s concern with contingency however they misrepresent these comments by ignoring the context in which these statements were made in the PMOC Monthly Attachment 1: HART’s Response to Audit Report Content Page 31 of 62 Appendix H: Management Response Report. The implication that HART underfunded contingency in 2012 is false and that point has been addressed in responses provided in other areas of this report. The reality is during the referenced time frame of these comments HART experienced some unforeseen issues with significant cost implications that are relevant to the comments made by the PMOC at those times. Prior to the April 2013 report, HART was suspended from all ground disturbing activities in order to complete the Archeological Inventory Survey (AIS) along the alignment of the Project. This caused a significant delay to the Project that was unanticipated and not accounted for in the baseline budget established under the FTA. As mentioned in the PMOC Monthly Reports from April 2013 through August 2013, the PMOC is stating concerns with the adequacy of the remaining contingency with regards to the anticipated cost due to project delays. Quantifying the cost impact of the AIS delay was a complicated endeavor because ultimately HART had very little control on when the suspension would be lifted as that authority resided with the State Supreme Court. This was a significant risk item during that time period that the PMOC and HART appropriately address this in their Monthly Reports respectively. It is also clear the PMOC comments are not related to an alleged funding shortfall in the contingency level in 2012. The auditors should revise their statement in order to not misrepresent the PMOC’s comments and remove any implication that contingency was underfunded in 2012.  In the September 2014 PMOC monthly report, the PMOC recommended that “strong controls must be put in place immediately to avoid future rapid contingency reductions.” It also added “the frequency and the levels of project management to which these statistics are reported should be improved and monitored monthly.” Response: When HART reviewed the first draft of this report HART informed the auditor that this statement mischaracterized the PMOC’s comments made in the Monthly Reports referenced. However, the auditor refused to incorporate HART’s corrections and changed the dates of the reports referenced in the final report draft to make it more difficult for HART to respond to the final draft. The recommendation made by the PMOC in September 2014 is also unrelated to contingency established in 2012. In August 2014, the FTA completed their 2014 update Risk Refresh report and according to their risk model they concluded the Project would end up approximately 5% over budget. One of their recommendations as a result of their risk modeling was to implement strong controls to mitigate against rapid contingency usage. This recommendation is repeated in the November 2014 PMOC Monthly Report and 2014 Risk Refresh is cited in the as origin of this recommendation. Further, in August 2014, the Westside Station Group construction bids came in over 60% higher than budgeted. This signaled there had been significant changes in the construction Attachment 1: HART’s Response to Audit Report Content Page 32 of 62 Appendix H: Management Response bidding market that would potentially impact the remaining construction work to be procured. The PMOC also notes this as a key issue in the November 2014 report specifically with regards to contingency.  In the December 2015 PMOC monthly report, the PMOC estimates $303 million remaining in total contingency. HART’s forecast report, as of December 2015, identifies $330 million in executed change orders and projects $301 million in pending, probable, and possible changes. Response: This analysis of the PMOC’s December 2015 monthly report was not included in the OCA’s first draft of this report. It was added in the final report draft to make it more difficult for HART to respond to the final draft. In addition to improperly adding the new analysis, the OCA has repeatedly misunderstood and misquoted data in other PMOC reports. The OCA is comparing the PMOC’s reported contingency to HART’s internal management reports without understanding the basis of either report or explaining how the two reports should be related. HART did not communicate the need for additional contingency until December 2014. In December 2014, HART increased the underfunded contingency reserves to $884 million. In October 2015, HART increased its reserves to $1.18 billion. The HART contingency increases from $644 million (13 percent) in June 2012 to $884 million (15.9 percent) in December 2014 to $1.18 billion (19.2 percent) in October 2015 were part of the reported project shortfalls. The last increase of $539.4 million (allocated and unallocated) was more than the allocated contingency under the final FFGA, and may have been excessive. Response: There are several errors in the audit report when characterizing the Contingency Percentage of Project Capital Costs. The statement with regards to the “last increase of $539.4M” being more than the contingency under the FFGA is false. Further, the statement is inaccurate and contradictory to other statements weaved throughout the report regarding contingency. The report acknowledges, in several sections including in Exhibit 3.3, the contingency under the FFGA Financial Plan is $643.6M. Then the report states the additional contingency budgeted in the last cost update of $539.4M is more than the contingency under the FFGA. Statements regarding contingency and Exhibit 3.3 should be revised to be factually accurate or removed. Attachment 1: HART’s Response to Audit Report Content Page 33 of 62 Appendix H: Management Response Exhibit 3.3 Contingency Reserves Increase (dollars in thousands) Source: OCA analysis based on the FFGA financial plan, June 2012; HART Project Risks Update, December 18, 2014; and Project Cost Update, October 15, 2015; PMOC Monthly Report, May 2013. (*) Numbers may not add up due to rounding. Cost controls were insufficient to control cost increases We found deficiencies related to HART’s cost controls16 that, in our opinion, partly contributed to the significant cost increases. In March 2014, the PMOC performed an initial review of information provided by HART and suggested a possible recommendation for HART to develop aggressive cost containment measures. In December 2014, HART announced the $910 million project cost overrun and revenue shortfall. The PMOC reported that, in February 2015, HART started implementing some cost containment measures. Although HART claims that evaluating and developing cost containment opportunities and cost reduction strategies were ongoing activities, we believe that HART could have taken a more proactive approach in implementing cost containment measures. Instead, HART reacted by requesting more funding. Response: The auditor provides no proof that cost reduction efforts were not in place. The auditor did not review any of HART’s cost containment documents. The recommendation is quite broad and unsupported by evidence. Instead, the auditor is misrepresenting the comments made by the PMOC in the February 2015 Monthly Report and states an objectionable position that HART sought more funding instead of being proactive in cost containment. In the February 2015 PMOC report referenced, the PMOC states cost containment and cost reductions measures were “recently implemented” and they continue with providing 16 According to the PMBOK guide, control costs is the process of monitoring the status of the project, to update the project budget, and managing changes to the cost baseline. Attachment 1: HART’s Response to Audit Report Content Page 34 of 62 Appendix H: Management Response a bulleted list of said cost containment measures. This should not be interpreted as a complete list of all cost containment measures implemented but what the PMOC states are recently implemented measures that were developed, some of which in coordination with PMOC. The PMOC’s decision to provide the list of cost containment measures recently implemented in the February 2015 report should not suggest there was a delay in implementing other cost containment prior to that report. By stating the PMOC reported HART “started” implementing cost containment measures is untrue and the auditor should remove this comment from the report. HART has been and continues to be very proactive in evaluating and implementing cost containing and cost mitigating measures on the Project. For example, the PMOC states in their Monthly Reports as early as 2012, “…the grantee has implemented efforts to recover contingency levels through cost reduction measures, value engineering, and revised project delivery strategies. In addition, strong controls have been put in place to avoid future rapid contingency loss”. In 2013, in the midst of the AIS delay, the PMOC acknowledges in their Monthly Report HART’s efforts to maximize economies of scale and reduce interface needs to achieve cost and schedule benefits by repackaging contract packages. HART’s decision to cancel and repackage the Westside Stations Group procurement in August 2014 was a repackaging strategy that resulted in over $40M in savings to the project that would not have been realized had HART proceeded with the award of the original contract package. In September 2014 HART provided a comprehensive response to the FTA’s 2014 Risk Refresh update detailing a number of cost and risk mitigation actions HART has taken, cost containment measures implemented or to be implemented, and provided a response to each of FTA’s recommendations with an action plan. Despite any of the previously mentioned points, this disputes the implication HART has not taken a proactive approach to cost containment and cost mitigation, the auditors are entitled to their opinion. However, it is erroneous and deceitful to suggest HART’s primary reaction to address the Project’s cost increases was to request more funding. The auditor should remove this assertion from the report. We also found that not all project managers compare actual costs against their budgets even though this comparison is an important cost control mechanism. One project manager said that she relies on project controls to monitor her contract costs. The project is at risk of additional cost overruns when project managers do not pay attention to project costs and budgets. In our opinion, project managers should continuously compare actual costs against budget amounts, should analyze any variances, and take corrective actions before costs go higher than expected. Response: Project Controls provides support to the Project Managers through preparation and issuance of various cost reports which the Project Managers use to manage contract costs. Management of contract costs includes monitoring actual costs to contract amounts, identification of issues that may have a cost impact, and managing the change process. The auditor is using evidence obtained from a single interview with one project manager to support this claim. The auditor inappropriately asserts that this issue applies to more than one project manager although the issue was not investigated further. Attachment 1: HART’s Response to Audit Report Content Page 35 of 62 Appendix H: Management Response Updating the project management plan, including subsidiary plans, was never made a priority A Project Management Plan (PMP) is a formal, approved document that guides how a project is to be executed, monitored, and controlled. It includes subsidiary plans that provide guidance and direction for cost management planning and control. According to the FTA Project and Construction Management Guidelines, a PMP is required by statute for major capital projects, provides a functional, financial, and procedural route map for the grantee to effectively and efficiently manage on-time, within-budget, and at the highest quality level in its unique project environment. The FTA requires PMPs to be submitted initially prior to preliminary engineering and updated through subsequent project phases. Moreover, best practice recommends continuous updates to the PMP because it can provide greater precision with respect to schedule, costs, and resource requirements to meet the defined project scope. We found that HART has not updated its PMP, including the subsidiary plans, despite considerable changes in project schedule, costs, and staffing since the federal FFGA was issued by the FTA in December 2012. Response: When HART reviewed the first draft of this report HART informed the auditor that this statement was not accurate regarding the FTA’s Best Practices. However, the auditor refused to incorporate HART’s corrections. The FTA best practice doesn’t recommend continuous updates to the PMP. The FTA best practice is to update the PMP as the project enters a new phase (i.e. from final design into FFGA). Updating the PMP and its subsidiary plans17 has not been a high priority for HART. According to the PMOC, the PMP update has been in progress prior to March 2013. The PMOC indicated that it was critical for HART to update the PMP and its subsidiary plans. We believe that an outdated PMP and its subsidiary plans could hinder management’s ability to effectively guide the project to completion in an economical, effective, and efficient manner. Response: Many of the plans identified in the audit report were identified to be updated in August 2014. This happened to be right before the 9 station bid package bid proposals were received and greatly exceeded the original estimates. At this time HART, working with its partners, had to analyze and restructure the program execution method in order to contain cost. Re-structuring the Contract Packaging Plan (one 9staiton package into three 3-station packages) resulted in numerous changes including 17 Subsidiary plans include the quality management plan; real estate acquisition and management plan; bus fleet management plan; rail fleet management plan; safety and security management plan; safety and security certification plan; configuration management plan; staffing and succession plan; risk and contingency management plan; operating plan; force account plan; mitigation monitoring program; interface management plan; contract packaging plan; claims avoidance plan; construction management plan; contract resident engineer manuals; and project procedures. Attachment 1: HART’s Response to Audit Report Content Page 36 of 62 Appendix H: Management Response re-organization of staffing and consultant services to manage the separate contracts, reallocation of funding to cover higher costs, update of the master schedule and the risk and contingency management plan. It was also during the 2015 year that the FTA performed the Triennial Review of HART and DTS. Also, during this time, City Council took up a resolution to remove section 5307 monies from the HART project funding. Staff has been working to update the program plans and keep up with the changes. Draft revisions have been created and shared with the PMOC and FTA, but are awaiting final sign-off for a variety of measures. For example, the Financial Plan update was onhold until it became apparent the GET would be extended. The update of the financial plan impacted the contract packaging plan as HART was now facing a budget deficit that would require major scope reductions had the GET not been extended. Staff created numerous draft schedules to match the contract packaging plan options along with estimated costs. The project management plan has been updated in draft form, but pending the update to the CPP and Financial Plan, the draft is waiting for final approval. HART will be entering the Risk Refresh Workshop with the FTA at the end of March and believes that comments and reports resultant of that workshop will help finalize the plans. Attachment 1: HART’s Response to Audit Report Content Page 37 of 62 Appendix H: Management Response Exhibit 3.4 Key Management Plans Source: HART Standard Terms Definitions and Acronyms, April 19, 2012; HART Monthly Progress Report, December 2015; and PMOC Monthly Report, December 2015. Attachment 1: HART’s Response to Audit Report Content Page 38 of 62 Appendix H: Management Response Waiver concessions were made Pursuant to the Hawai`i Revised Statutes §103-32.1, city contracts allow HART and the city to withhold up to 5 percent of a contract amount to ensure that the contractor’s performance is satisfactory and acceptable. Once the city is satisfied with the project or is satisfied with any re-work the contractor is asked to do, the city will release the retained amount (retainage) to the contractor. For one contract, the rail prime contractor, Kiewit, requested a partial release of the retained amount. On March 6, 2013, HART waived the five percent retainage for three construction contracts with Kiewit18. This concession was made for three specific change orders totaling $26 million and the total amount waived was about $1.3 million. Actual expenditures totaled $23 million, of which the total amount actually waived was $1.1 million. According to HART managers, they approved the waiver as an incentive for Kiewit to stay on the job. The approval reduced Kiewit costs during the period when the rail project work was suspended. The suspension occurred after an August 2012 Hawaii Supreme Court decision caused the project work to halt so archaeological surveys could be completed. In September 2013, project work restarted.19 By waiving the 5 percent retainage, HART and the city lost some leverage to ensure Kiewit performed satisfactorily under its contracts. The city also risked being accused of favoritism or bias towards one contractor. Response HART did not make concessions to a single contractor. In the examples cited, HART determined that retention did not apply. AIS Suspension Delay Change Orders During the court-ordered suspension, no construction work could be performed. HART paid the contractor for only actual, incurred costs for them to remain mobilized during the suspension. Notice-to-Proceed Delay – Cost Escalation Change Orders HART withheld retention for the work completed in accordance with the General Conditions. The value of that work completed was then subject to cost escalation. The escalated costs did not represent payment for work completed and was therefore not subject to retention. 18 The contract documents indicate Kiewit requested a partial release of retainage on only one contract prior to HART issuing the approval memorandum for all 3 Kiewit contracts. 19 According to HART managers, the term waived was an error and the wording should have been retention will not be withheld. Attachment 1: HART’s Response to Audit Report Content Page 39 of 62 Appendix H: Management Response Per HART General Conditions, 5% retainage is required to be withheld for progress payments to the Contractor for work completed up to 50% of the value of the contract. As of this date, HART has retained over $15.7 million on invoices submitted by Kiewit. That is almost 15 times more than the $1.1 million retainage that the city auditor cites. The $15.7 million retainage will only be released when the Kiewit contract has been satisfactorily completed and contract final acceptance is achieved. HART determines whether the contract has been satisfactorily completed and controls when the retained amounts will be released. HART erred when using the term “waived” in the change order. The wording that should have been used is - “retention will not be withheld.” Retainage payments are mandated and controlled by contract terms and conditions. Retainage amounts and subsequent release of retainage (payments) are made to protect the City from being in a position of having insufficient funds to have a complete and usable facility, when the contract terms are fulfilled. Cost estimates and potential savings were lacking in the decision to repackage the Westside Stations Group20 HART was unable to demonstrate that it prepared a reasonable level of cost-benefit analysis to justify its decision to repackage the Westside Stations Group prior to the rebid. Our review of the Westside Stations Group repackage strategy found that HART lacked sufficient documentation to quantify the expected costs and potential savings when the decision was made to cancel the original bid. Consequently, HART assumed significant risks that could have driven contract costs higher and made future savings unattainable. Response: HART provided extensive documentation to support the fact that quantitative analysis was performed within the legal constraints required at each stage of the bid cancellation. However, the auditor lacked to technical knowledge to apply the judgment required to conclude there was a finding. During our audit, we asked HART to provide cost estimates and to identify its expected savings from the repackage. After opening the packages of the three station bids, HART executive management claimed a potential cost savings of $31 million on the decision to divide the nine stations into three packages of three stations. According to HART, the sum of the independent cost estimates totaled $263 million, $31 million less than the original low three bid of $294 million for the nine station package. We found that HART lacked sufficient time to fully evaluate and quantify any potential savings. On September 9, 2014, HART’s CEO, who also serves as the chief procurement officer, authorized HART staff to cancel the nine stations group construction bid prior to the 20 The Westside Stations Group consists of the first nine stations along the rail route. HART repackaged the Westside Stations Group into three rail station groups: Farrington Highway Stations, West O`ahu Stations and Kamehameha Highway Stations. Attachment 1: HART’s Response to Audit Report Content Page 40 of 62 Appendix H: Management Response receipt of all independent cost estimates for the three rail station packages, including a $5.2 million contract for a H2R2 ramp that was originally part of the Kamehameha Highway Stations Group. See Exhibit 3.5 for the timeline. Exhibit 3.5. Westside Stations Group Bid Cancellation Timeline Source: OCA analysis of independent cost estimates and HART Monthly Progress Report, November 2015. According to HART, its consultants performed quantitative analysis to support its decision to repackage the Westside Stations Group bids and provided two whitepapers to substantiate its cost-benefit analysis. While there were discussions of market factors and different procurement strategies, we did not find any quantitative analysis to show the potential costs or savings for the decision to rebid. One of the whitepapers included several pages on value engineering, scope reduction and scope transfer ideas. For the first three stations, a Rough Order of Magnitude (ROM) for cost savings were notated next to each idea. These ROM for cost savings were extrapolated to the remaining six stations. HART was unable to provide how the ROM figures were derived. The independent cost estimates for the three packages totaled $263 million which was $79 million higher than the engineer’s original estimates. Nevertheless, HART reported in its August 2014 Monthly Report that the overall cost of the $5.2 billion project would not change and that additional costs could be covered using a combination of contingency funds and adjusting the contract scope to reduce costs. Attachment 1: HART’s Response to Audit Report Content Page 41 of 62 Appendix H: Management Response Contrary to the August 2014 Monthly Report, HART announced in December 2014 that the cost for all the stations to be constructed and the remaining guideway will exceed the contingency reserves, off-sets, and other funds available by several hundreds of millions of dollars. Response: Both of the preceding two paragraphs are factually inaccurate in that the August 2014 Monthly Report is based upon July 2014 information, prior to the August 2014 WSSG bid opening. To compare the July 2014 information to the information known in December 2014 is misleading. In addition, the $263 million independent cost estimates for the three new station packages were prepared after the WSSG bid opening. To compare the total of those estimates to the information in the August 2014 Monthly Report, as is stated above, is an inaccurate comparison. Fortunately, the actual outcome reduced the original, nine station group bid by $31 million. If the strategy had failed, the decision could have driven contract costs higher than the original nine station bid. Because of the fiscal situation confronting the agency, the CEO told us that he had to make it work. He relied on his professional judgment and consultant’s opinions when he canceled the nine station bid and repackaged the contract into three rail station groups. While professional judgment is important, critical decisions should be supplemented by quantitative analysis and documented analysis, as well as past experience, and current or historical data. Response: The audit report is misrepresenting what the CEO actually said. The auditor took pieces from a conversation and does not portray the full context of what was discussed. It is misleading to imply that the CEO relied only upon professional judgment. He explained, in great detail, the analytical process that HART undertook in responding to the WSSG bid opening. This event started an entire effort to analyze and evaluate the situation. It was only at the end of this topic’s discussion that professional judgment was mentioned. The entire decision-making process did not rely solely on professional judgment, yet required detailed analysis, active discussion and debate among HART staff, and engineering judgment. HART paid $1.5 million to unsuccessful bidders without documenting their actual costs. According to HART managers, issuing stipends to unsuccessful bidders is a common practice, is allowed under state law, and is accepted by the Federal Transit Administration. According to HART, this practice gives the owner title to the proposed design concepts. The stipend payment covers all or part of the cost of preparing bid proposals and encourages competition although the practice is an added cost to the owner. While there is no specific evidence that firms would not submit bids if no stipend was provided, HART offered $3.5 million to compensate unsuccessful bidders for three Attachment 1: HART’s Response to Audit Report Content Page 42 of 62 Appendix H: Management Response design-build contracts21. The stipends were to be divided equally and not to exceed $500,000 for each unsuccessful bidder. HART’s records show that $1.5 million was paid to three unsuccessful bidders as of June 30, 2015. Exhibit 3.6 Stipend Payments Source: OCA analysis based on C2HERPS data provided by the HART. Response: HART previously informed the auditor that stipends were not paid in connection to the Airport Guideway and Stations procurement. However, the auditor did not correct this error in the final draft. We were, consequently, unable to assess whether the stipend payments were excessive or if the stipends covered the unsuccessful bidder’s actual costs because HART had not established any written policy or procedures related to administering the stipend payments. Effective policy and procedures provide staff guidance for issuing proposal stipends to unsuccessful bidders. Without any policies or procedures, it is unclear how HART was able to determine the compensation amount for each of the unsuccessful bidders; assess the documentation needed to support their actual costs; or determine the value or usefulness of the unsuccessful bidders’ proposals. Response Summary: 1. All 3 stipends were paid on City-procured contracts – WOFH and KHG. 2. No stipend has been paid on AGS because it’s an active solicitation. 3. HRS Section 103D-303(i) permits the use of stipends on design-build construction contracts over $1M. 4. Payment based on actual costs would violate FTA Best Practices, which provides that the amount of the stipend is to be “uniform for all competitors.” 5. Payment based on actual costs would likely result in higher stipend payments. As a rule, HART does not pay offerors for the cost of preparing their proposals and bids; stipends are only provided for certain large and complex design-build (DB) contracts as authorized to do so under HRS Sec. 103D-303(i). The stipend amount set by HART (at the time, Rapid Transit Division of City’s Department of Transportation Services (DTS)) is based on industry estimates for the cost to prepare proposals. The industry estimate for preparing proposals is between .5% and 1% of the construction costs. That is, for a $300M contract, the cost for preparing proposals is typically between $1.5M-$3M. Both of the contracts that stipends that had stipend payments were in excess of $300M and 21 West O`ahu/Farrington Highway Guideway, Kamehameha Highway Guideway, and Airport Guideway and Stations Design-Build Contracts. Attachment 1: HART’s Response to Audit Report Content Page 43 of 62 Appendix H: Management Response HART/RTD has never paid more than $500,000 stipend to an unsuccessful offeror, which is well below the actual costs for preparing proposals. A uniform stipend amount is also consistent with Federal Transit Administration’s (FTA) Best Procurement Practices on the subject matter. Furthermore, HART’s solicitation documents expressly stated the terms under which an unsuccessful offer would be entitled to stipends and the not-to-exceed amount to be paid to each unsuccessful offeror; that is, a minimum 41% overall qualitative score was required to receive a stipend. HART agrees to memorialize its policies and procedures into a “policies and procedures” manual. HART is paying for vacant office space HART leases four floors of office space. During our on-site visit at HART, we identified one leased floor of 16,182 rentable square feet that had 12 vacant offices and 15 empty workstations. About 41 percent of the offices and workstations were unoccupied on this floor when we conducted a physical inventory count in July 2015. Although HART stated it has plans to occupy the empty space, the offices and workstations sat vacant throughout our six-month audit. While it was empty, we requested a current inventory listing of its leased offices and workstations. It took HART staff five weeks to provide us this information. Review of its inventory listing revealed that the vacancy rate increased to 44 percent. As of April 2016, HART reported a vacancy rate of 27 percent on the floor in question. Upon verification, we found that the vacancy rate is closer to 32 percent22. HART should evaluate and document its office space requirements and minimize HART operating costs by subletting its surplus office space or renegotiating its leases so that it only pays for space that is needed. Summary: HART does regularly review its office space requirements and negotiates lease agreements to ensure that both current and future needs are in alignment with the project’s current status. HART currently leases 63,927 sq. ft. (55,110 useable sq. ft.) of office space at Ali’i Place. HART conducted a physical inventory on 3/15/2016 and determined its occupancy rate to be 88% comprised of the following breakdown: Suite 150: 31.5 of 34 available workstations in use = 93% Occupancy Rate 11th floor: 63 of 68 available workstations in use = 93% Occupancy Rate 17th floor: 65 of 68 available workstations in use = 96% Occupancy Rate 23rd floor: 48 of 66 available workstations in use = 73% Occupancy Rate As of April 2016, HART reported an aggregate occupancy rate of 88 percent. The aggregate amount included three floors that were 93 percent to 96 percent occupied and one floor that was rd 73 percent occupied. The audit discusses the 23 floor that is currently 32 percent vacant. 22 Attachment 1: HART’s Response to Audit Report Content Page 44 of 62 Appendix H: Management Response Overall: 207.5 of 236 available workstations = 88% Total Occupancy Rate Note: Workstations include locations designed for someone to sit and work on a full time basis. This includes offices, cubicles, desk/work areas, etc. The vacant space is necessary and reasonable given that, as of February 23, 2016, HART had (11) vacant positions and is contractually required to provide office space for multiple contractor/consultant firms; HART currently houses employees from the following firms: HDR (InfraConsult), CH2M Hill, Paragon Partners Ltd., Lea+Elliot, PGH Wong, RM Towill, SSFM, Stantec, Lawson & Associates, and the Solis Group. HART also hires local college students as Engineering and Student Interns on a part time basis in accordance with the City & County of Honolulu Department of Human Resources policies when office space is available and we have substantive work appropriate for Interns. Additionally, it should be pointed out that the practice of providing office space to consultants was a recommendation in a prior City Auditor audit – reference City Auditor report 13-03, recommendation #8 – Lease all office space to reduce consultant overhead rate charges. The numbers reported in the audit appear to have focused solely on the 23rd floor, which is the least occupied floor. The recommendation that HART renegotiates its lease “so that it only pays for space that is needed” is an oversimplified view of how long term leasing works and is not practicable for an organization such as HART. HART is currently responsible for overseeing the construction of the Honolulu Rail Transit Project. As an organization, HART functions differently from a regular City Department and the personnel needs in terms of both quantity and areas of expertise needed vary significantly over time as opposed to an established City Department. Specifically the personnel/workstation needs evolve and change throughout the project lifecycle depending on which phase of the project we are in. A project’s staffing level is dynamic and is directly related to the type and amount of work going on at any given time. In order to accommodate the project’s personnel needs, HART has taken great effort to structure and negotiate the current leases to meet current and future office space requirements. HART’s approach considered the number of personnel required as the project progresses towards completion and optimizes the office space requirement and the resulting cost to the project. Starting in 2018, the office leases begin a structured reduction of office space over the remainder. All while being cognizant that long-term leasing of office space yields the best rates for the lessee and ultimately benefitting the taxpayer. Therefore the recommendation that HART only lease office space it needs implies that HART change the amount of leased space depending on how many people are onboard at any given time. This is an impractical view of office leasing practices and would result in drastically increased costs as well as inefficiencies that would affect the overall project in a negative way. Attachment 1: HART’s Response to Audit Report Content Page 45 of 62 Appendix H: Management Response Exhibit 3.7 Office and workstation vacancy rates Source: OCA physical count of leased office space and HART Work Space Inventory Listing, November 30, 2015 Exhibit 3.8 Photos of Vacant Offices and Workstations Source: OCA Contract administration needs to be improved Current and complete policies and procedures are necessary to provide clear and effective guidance to staff regarding contract management. Strong recordkeeping practices serve to ensure compliance with federal, state, and local laws. We sampled 25 contracts during our audit and found several deficiencies. We found that HART has not developed written policies related to contract administration and invoice payment practices, and procedures were incomplete and not regularly updated. These deficiencies could lead to noncompliant and questionable practices. Response: HART does have written contract administration and invoice payment practices. As mentioned in the response on the next page. Auditor should revise this statement. Attachment 1: HART’s Response to Audit Report Content Page 46 of 62 Appendix H: Management Response Exhibit 3.9 Summary of Contract Types Reviewed Source: OCA sample selection Contract administration policies and procedures were incomplete and outdated We found HART has not developed written policies related to contract administration and invoice payment practices. Policies and procedures were also needed for capital project monitoring and reporting. HART contract administration procedures were last updated in 2012 and need to be updated to reflect current policies and procedures. Outof-date policies and procedures increase the risk of contract mismanagement if guidance is incorrect. Response: HART does have written contract administration procedures and invoice payment practices. HART is in the process of updating existing procedures and has initiated an effort to update contract administration procedures. In September 2015, HART updated the Contract Change Procedure 5.CA-11. Contract administration roles and responsibilities were confusing We found that HART’s contract administration procedures defined confusing roles and responsibilities for its project managers, contract managers, and contract administrators. HART’s project managers acted as contract managers until August 2015. In a separate updated procedure manual, HART delineated the positions into two distinct roles. In addition, the terms contract administrator and contract manager are used interchangeably at HART even though procedures define them differently. As a result, changes to contract administration procedures are confusing. Until August 2015, the project manager filled both the project manager and the contract manager roles. The dual role increased the risk of contract mismanagement between 2012 and 2015. Attachment 1: HART’s Response to Audit Report Content Page 47 of 62 Appendix H: Management Response Response: HART has initiated an effort to update its Contract Administration procedures. A key focus of the update is to clarify and reflect current roles & responsibilities for project managers and contract managers. HART has taken proactive steps to address this issue through the establishment of a contract administration team in 2014, clarifying roles and responsibilities and updating of existing procedures and policies. For the sample of 25 contract files, we found: Response Summary: 1. Of the 25 contract files reviewed by the auditor, 2 were not HART contracts and 8 were procured prior to HART’s inception. 2. Of the 50 contract issues identified by the auditor, 48 were deemed to be unfounded for one or more of the following reasons: a. the alleged missing document was, in fact, properly filed in the contract file; b. the contract was not a HART contract, c. the issue raised by the auditor was not a contract requirement; and d. HART had an electronic copy of the document, but a hard copy was not placed in the contract file. 3. HART acknowledges that there are 2 issues which are outstanding and which HART will remedy. HART informed the OCA that 2 of the legal services contracts included in the OCA’s sampling were procured by the City and, as such, were not HART contracts. Despite HART’s confirmation that they were not contracts procured or administered by HART, the OCA elected to keep the contracts in its sampling. Accordingly, HART respectfully takes exception to any deficiencies identified for contracts that are not HART contracts. Of the remaining 23 contracts, HART notes that 8 were procured prior to HART’s inception. Although HART understands that the scope of the audit includes expenditures that pre-date HART’s inception, it is factually inaccurate to attribute contract deficiencies to HART for contracts which were not procured by HART. Accordingly, HART respectfully takes exception to any deficiency findings attributed to HART for contracts that were procured prior to HART’s inception. When HART brought this issue up with the auditor’s they brushed HART’s concern aside with the claim that “every agency has legacy issues”. With that argument the auditor acknowledges that the problems were not caused by HART’s actions, however, the auditor failed to disclose the fact and extent that the issues that were noted found were just “legacy issues” and not extraordinary or unreasonable problems. Contract files had missing documentation Documentation deficiencies included no complete listings of contract modifications and supporting documentation in the files; 8 files which grant HART access to only prime Attachment 1: HART’s Response to Audit Report Content Page 48 of 62 Appendix H: Management Response contractor records; no files with access to sub-contractor records; 3 contract files that did not include the Scope of Work23; and 2 contract files missing a Letter or Notice of Award. HART respectfully takes exception to the OCA’s finding that there were “no files with access to sub-contractor records.” The OCA’s finding suggests that such access is a contract file requirement when, in fact, it is not a contract file requirement. In footnote 24, the OCA indicates that subsequent to its review, HART presented “Statements of Work for 2 contract files.” HART respectfully takes exception to this finding and footnote because the scope of work was properly filed in the respective contract files at the time of the OCA’s initial review. HART respectfully takes exception to the OCA’s finding that “2 contract files [were] missing a Letter or Notice of Award.” Although HART acknowledges that one of the notifications was filed electronically rather than in the hard copy file, the other contract file properly documented the notification of award. The OCA’s finding that the notification was missing appears to be based on its mistaken belief that there needs to be a document specifically entitled, “Notice of Award.” HART respectfully disagrees that a document title is dispositive of whether its contents provide adequate notification of award. HART’s contract file included a letter transmitting the contract to the successful offeror for execution, notification letters to the unsuccessful offerors that the contract had been awarded to another firm, and documentation of the posting of the award on the City’s website. HART’s contract file therefore included more than adequate documentation of the notification of award. Financial disclosures and conflicts of interest certifications were missing We found no evidence to show that HART required financial disclosures for prime contractors or subcontractors; 7 contract files where the prime contractors did not certify they had no real or apparent conflicts of interest24, and no evidence of subcontractor’s conflicts of interest certifications. HART maintained subcontractors were required to file conflicts of interest statements as part of the solicitation process. HART respectfully takes exception to the OCA’s finding that there was “no evidence to show that HART required financial disclosures for prime contractors or subcontractors.” The OCA’s finding suggests that financial disclosures are necessary to meet the conflict of interest certification requirement when, in fact, they are not necessary. As such, the condition that the auditor observed should not have been a finding based on the criteria that should have been tested. Subsequent to our review, HART presented Statements of Work for 2 contract files. HART noted that the third had been electronically filed, but would be included as a hard copy in the contract file. 24 Subsequent to our review, HART presented Conflict of Interest certifications for 4 contract files. HART contends that one contract was procured prior to its inception, and that the two remaining contracts were funded by HART, but were procured by Corporation Counsel. 23 Attachment 1: HART’s Response to Audit Report Content Page 49 of 62 Appendix H: Management Response Excluded contractor checks were missing Two contract files lacked evidence that an Excluded Parties List System (EPLS)/System for Award Management (SAM) check was performed prior to award to ensure the contractor was not suspended or debarred by the Federal government. These checks may have been made, but were not documented in the procurement files. Although HART maintains that EPLS/SAM checks were not put into routine practice at HART until 2012, our review of contract files suggests these checks were being performed as early as 2007. The compliance requirement with 2 CFR 180.300 was met because HART demonstrated that it had obtained certification from these 2 respective contractors regarding debarment, suspension and other ineligibility and voluntary exclusion from transactions financed in part by the U.S. Government. It is unclear why the OCA asserts that the “excluded contractor checks were missing” when the OCA expressly acknowledges that HART met the compliance requirements. As pointed out by the OCA, the Code of Federal Regulations provides that the compliance requirement is satisfied if contractor certifications are obtained. HART therefore respectfully takes exception to the OCA’s finding when the OCA, itself, acknowledges that the compliance requirement was met and the stated objective of this area of the audit was to “assess compliance of HART’s procurement and contract management practices”. As such, the condition that the auditor observed should not have been a finding based on the criteria that should have been tested. Other contract file deficiencies included no evidence that contract managers conducted performance reviews; 22 contract files did not have designated contract managers; 1 contract file lacked the independent evaluations and scores of the evaluation committee related to the contract award 25; and 3 files had no evidence of a cost or price analysis by HART for the intended award26. A cost or price analysis should be performed for every contract so that the essential objective of a reasonable price is assured. HART respectfully takes exception to the OCA’s findings that there was “no evidence that contract managers conducted performance reviews” and that “22 contract files did not have designated contract managers.” The OCA’s findings suggest that such performance reviews and designations were contract file requirements when, in fact, they are not contract file requirements. In footnote 26, the OCA indicates that subsequent to its review, HART presented the independent evaluations and scores of the evaluation committee. HART respectfully takes exception to this finding and footnote because the evaluations and scores were 25 Subsequent to our review, HART presented independent evaluations and scores of the evaluation committee related to this contract award. 26 Subsequent to our review, HART presented cost or price analysis for 2 contract files. HART produced a waiver for cost or price analysis being performed for the third contract; however, the waiver states that no prices had been submitted. No further analysis was performed. Attachment 1: HART’s Response to Audit Report Content Page 50 of 62 Appendix H: Management Response properly filed in the contract file at the time of the OCA’s initial review. The auditor simply overlooked the documentation in the files and never asked for help locating the documentation. In footnote 27, the OCA indicates that subsequent to its review, HART produced the cost/price analysis for 2 contract files. HART respectfully takes exception to this finding and footnote because the cost/price analyses were properly filed in the respective contract files at the time of the OCA’s initial review. The auditor simply overlooked the documentation in the files and never asked for help locating the documentation. The auditor did not inform HART of the contract file deficiencies until after the first draft of the report was given to HART. After receiving the first draft, HART provided the documentation to the auditor to correct factual misstatements in the report. It is improper for the auditor to insert HART’s corrections in footnotes rather than correcting the errors in body of the report. Invoice payment procedures do not address all contract types We found that HART’s invoice payment procedures do not address all contracts types, such as cost reimbursement, and time and materials contracts, because there were no defined policies and only one procedure27 related to contract payments. HART’s invoice payment procedure was developed in 2012 and was limited in scope to only Firm Fixed Price contracts which uses a Schedule of Milestones to determine monthly progress payments by milestone achieved. Because HART has a variety of contracts and invoices which are not paid based on milestones, we believe that this procedure is insufficient to fully support HART’s invoice payment process. Furthermore, we found that HART’s procedure had not been updated to reflect its current practices with respect to invoices. If practices for paying invoices for contracts and goods and services are not adequately conveyed in guidance, policies, and procedures, it can lead to invoices being paid in spite of insufficient support and questionable expenses. Response: HART concurs with this statement. HART Contractor Progress Payments procedure (5.CA-03 dated 4/19/2012) is limited in scope to only Firm, Fixed-Price (FFP) contracts which use a Schedule of Milestone. However, appropriate HART staff and management reviewed invoices before payment was approved in every case. Invoice review is detailed and includes review for compliance with contract terms, billing rates and appropriateness of supporting documentation. A draft contractor invoice procedure (Procedure no. 5.CA-10) has been prepared to formalize the review process for payments under contracts other than FFP contracts and is under management review. 27 Contractor Progress Payments, 5. CA-03, Rev. 1.0 - April 19, 2012 Attachment 1: HART’s Response to Audit Report Content Page 51 of 62 Appendix H: Management Response Invoice payments had unsupported and unallowable costs A strong invoice payment process prevents improper payments from being made. HART risks making improper payments when there is a lack of proper review and documentation to support the work or services billed. Our review of 50 HART invoices revealed instances of incomplete and improper payments or authorizations. We also found unsupported and unallowable costs. Specifically, our review showed HART paid:  3 invoices, valued at $8,670,112, where checklists and forms were being used by HART which were not described in the invoice payment procedure. No amounts were improperly paid. Response: HART used a more detailed checklist than the example included in the approved procedure. HART is in the process of updating procedures with the new checklists.  3 invoices that lacked the required payment review checklist. The invoices totaled $6,292,325. No amounts were improperly paid. Response: The required checklists were missing, however, HART did have the required routing sheets documenting that the required staff reviewed the invoices.  2 invoices, valued at $18,607,656, had narrative descriptions attached to invoices which were incorrect. No amounts were improperly paid. Response: The errors were typos in the narratives that did not affect the amount paid in any way. A typo is not a reasonable basis to support the finding that the payment was unsupported or unallowable.  2 invoices that had management approvals that totaled $23,288 although the work was performed prior to the execution of the contract agreements and constituted procurement violations. No amounts were improperly paid. Response: The invoices pertain to utility relocation work performed by two utility companies: Oceanic Time Warner Cable ($14,513.08) and Tesoro Hawaii Corporation ($8,775). Utility relocation work is sole source and not a competitive procurement because only the utility company can perform the relocation of its own property/utilities. HART was in the process of negotiating utility relocation agreements with Oceanic and Tesoro when they decided to commence work. HART informed both Oceanic and Tesoro that such work would be performed at risk should negotiations fail, but both utility companies did so in good faith support of the Project because they understood that their cooperation was vital to maintaining the Project schedule. Because work was performed prior to the execution of the utility relocation agreements, HART’s policies and procedures required the documentation and acknowledgement of the circumstances leading Attachment 1: HART’s Response to Audit Report Content Page 52 of 62 Appendix H: Management Response to Oceanic and Tesoro commencing work during negotiations via a “Request for Approval of Procurement Violation.” Thus, HART properly implemented its procedures regarding work performed prior to contract execution.  $11,344 for on-call contractor work performed for the Pig & the Lady restaurant (83 N. King Street), even though the work on the City Center had not officially started. HART contends the work was necessary, supported, and allowable under the terms of the contract. Response: The work was necessary, supported, and allowable under the contract. The work performed at the 83 N. King Street is part of the HART Historic Preservation Committee (HPC) 2014 Funding. The HPC permits owners of historic buildings near the Chinatown station to apply for minor repairs and upgrades to building facades. Pacific Gateway (the owner of 83 N. King Street) applied for and was approved for removal of piping on front façade and the replacement of an awning. This work is done in advance of Guideway construction.  $1,863 for unallowable travel agent fees. This was due to a conflict between HART and the City’s respective travel policies. Response: The $1,863 in “travel agent fees” was actually 50 individual pcard transactions with booking fees paid to various online vendors such as Expedia from 2008 – 2014. HART is confused by this finding because it is not related to any of the stated objectives of the audit. This was allowable under HART’s travel policy at the time and it is an ordinary and customary business practice to use sites like Expedia to book travel. The condition observed should not have been a finding as it was not related to the criteria tested during the audit.  $740 for vacation travel expense paid for by HART. HART contends the work was allowable under the terms of the contract. Response: The auditor says that HART contents that the work was allowable under the terms of the contract as if the fact is disputable. HART provided documentation to the auditor supporting that the fact that the terms of contract allowed a certain number of trips per year to include travel for holidays. It was necessary because HART had to import real estate professionals from the mainland with considerable experience with relocations and acquisitions under the uniform act. Attachment 1: HART’s Response to Audit Report Content Page 53 of 62 Appendix H: Management Response Chapter 4: Better planning is needed to address and manage future rail project costs Summary Once the rail system is completed and operational, other rail systems indicate annual and ongoing operations and maintenance costs must be addressed. Other rail systems in the nation indicate subsidies will be needed to fund rail operations and maintenance costs after the rail is constructed. HART has not planned for the operations and maintenance of the rail system or the costs of operating the system after it is completed. HART needs to improve planning and oversight to effectively address and manage future operations and maintenance needs; maximize fare box recoveries and ridership; minimize city subsidies; and fill operations and maintenance positions. Background Due to project delays, HART reports interim rail service will begin in late 2018 and full service operations are projected to begin in 2021. The original and updated time schedule is shown below. Exhibit 4.1 Project Schedule Source: Office of the City Auditor (OCA) analysis based on HART documents HART needs to plan for annual operations and maintenance of the rail system HART reports it is only responsible for constructing the rail project and is not responsible for the ongoing, annual operations and maintenance of the rail system. As a result, we found HART plans were outdated and did not adequately address how to operate and maintain the rail system once it is completed. We also did not find plans that addressed the ongoing costs of operating the system. Response: HART is responsible for ongoing operations and maintenance for the rail system. The Charter of the City & County of Honolulu states “The public transit Attachment 1: HART’s Response to Audit Report Content Page 54 of 62 Appendix H: Management Response authority shall have the authority to develop, operate, maintain, and expand the city guideway system”. Moreover, the core systems contract is a design, build, operate and maintain contract (DBOM) that details the operations of the rail system. HART records show the audit work related to operating costs consisted of (1) interview with the Core Systems Project Manager and (1) email information request. A review of HART Board minutes, available on the website, evidences HART and its Board’s commitment in fulfilling its Charter responsibilities for operations and maintenance for the rail system. The following table is a partial listing of various Board (and Staff activities) related to rail operations, fare policies, and subsidy reduction measures. This illustrates HART’s actions, thus far, regarding rail O&M costs and fare policy. Date 7/1/2011 7/7/2011 9/8/2011 9/29/2011 10/6/2011 10/13/2011 11/3/2011 11/17/2011 12/1/2011 3/16/2012 6/28/2012 8/9/2012 8/9/2012 8/9/2012 8/9/2012 8/9/2012 8/30/2012 8/30/2012 10/4/2012 11/29/2012 12/6/2012 12/6/2012 12/6/2012 2/7/2013 10/17/2013 12/19/2013 12/19/2013 12/19/2013 12/19/2013 Description HART Board sworn in Financial Plan Discussed DBOM Core Systems Contractor Capacity Safety and Security Presentation Update on DBOM Core Systems Contract DBOM Core Systems Contract Discussion Presentation on Scope of DBOM Core Systems Update on DBOM Core Systems Contract Update on DBOM Core Systems Contract Bus and Handi-Van Operating Costs FFGA Financial Plan Update Discussion on Fare Gates Discussion on Rail/Bus Synergies Discussion on Train Seating Discussion on Financial Plan Operating Budget FFGA Financial Plan Update Fare Gates Train Seating FFGA FCA Update / Operating Budget Discussion FFGA Financial Plan Discussion FFGA Financial Plan Discussion Review of HART Operating Statement Discussion of DBOM Core Systems Update on DBOM Core Systems Contractor Capacity Discussion on Maintenance Yard Automation Discussion of Operating Budget Admin Costs DTS presentation on Fare Collection Study Formation of Fare Policy Permitted Interaction Group Four-Car train Presentation Attachment 1: HART’s Response to Audit Report Content Page 55 of 62 Group Board Finance Finance/POC POC Board Board Board Board Board Finance Board Finance Finance Finance Finance Finance Board Board Finance Board Finance/POC Finance/POC Finance/POC Finance/POC Finance/POC HR Board Board Board Appendix H: Management Response 1/16/2014 1/16/2014 2/13/2014 2/13/2014 4/24/2014 8/14/2014 10/9/2014 10/23/2014 11/13/2014 3/12/2015 5/21/2015 11/24/2015 11/24/2015 2/18/2016 Update on HART/DTS/OTS/Ansaldo Working Group Fare Study Workshop Presentation Automated Maintenance Yard Configuration change order Fare Study Workshop Presentation Presentation on Safety and Security HART/Ansaldo/DTS/OTS Working Group Update Report on Fare Policy Permitted Interaction Group Report on Fare Policy Permitted Interaction Group Core Systems Fare Vending Machines Interim Report of the Fare Policy Permitted Interaction Group Update on HART/DTS/OTS/Ansaldo Working Group Presentation on Sustainability and Photovoltaic Programs Update on Integrated Fare Systems Formation of Financial Plan Permitted Interaction Group Board Board Finance/POC Board Board Board Board Board Finance/POC Board Board Board Board Board According to HART, however, the board and HART are committed to fulfilling its charter responsibilities for the operations and maintenance for the rail system. Attachment 1: HART’s Response to Audit Report Content Page 56 of 62 Appendix H: Management Response Other rail systems indicate annual and ongoing operations and maintenance costs must be addressed. Our city comparisons indicated that all rail services throughout the nation require some form of subsidy to make up for the costs of operating and maintaining the rail. Our comparisons showed that fares paid by riders were insufficient to cover the entire cost of operating and maintaining the rail systems. (See Exhibit 4.2) Exhibit 4.2 Subsidy Comparisons (dollars in millions) Source: OCA analysis based on data obtained from the National Transit Database (NTD), 2013 The comparisons indicated that fixed guideway projects often resulted in significant transit service realignments, such as the creation of a transit agency to oversee and administer the operations of rail and other modes of public transit. To ensure a smooth transition from construction to operations, HART needs to update its operations and maintenance plan, establish operations and maintenance policies, develop fare policy details, identify subsidy sources, determine ridership and sources of revenues, and fill critical positions related to the operations and maintenance of the system after it is completed. Attachment 1: HART’s Response to Audit Report Content Page 57 of 62 Appendix H: Management Response Subsidies will be needed to fund rail operations and maintenance costs Fare revenues are earned through carrying riders through regularly scheduled rail service. While fare revenues will cover a portion of the operations and maintenance costs, our comparison of other systems showed that the fare revenues will not be sufficient to fully support total operating and maintenance costs. According to HART’s chief executive officer (CEO), fare revenues will cover about 30 percent of the operations and maintenance costs. The remaining 70 percent will require subsidies from the city. Although city subsidies will be needed, HART has not clearly defined how rail operations and maintenance will be subsidized in its 2012 Full Funding Grant Agreement (FFGA) financial plan. Our comparison of other cities showed that other revenue sources28 are available to offset the cost of operating the rail system. For example, operating costs not funded by fare revenues can be supported by a combination of federal, state, and local government taxes. Exhibit 4.3 identifies state, federal assistance, and other fund sources for other cities. Exhibit 4.3 Sources of Operating Funds by Transit Agency* Source: OCA analysis based on information from the NTD, 2013 (*) Percentages do not total 100 percent due to rounding. 28 Excise taxes, special assessments for cities and towns, and property taxes. Attachment 1: HART’s Response to Audit Report Content Page 58 of 62 Appendix H: Management Response (**) Fare revenues based on the farebox recovery rate for entire transit agency in addition to rail service, often including but not limited to the following services: bus, commuter bus, commuter rail, light rail, heavy rail, ferry boat, and paratransit. Other forms of funding include parking fees, selling surplus land and property, retail space rental to vendors, utility company rentals of rights of ways, and advertising. Although these alternative sources of funds exist, HART has not identified other revenue sources for its rail operations and maintenance in its financial plan. As a result, HART and the city cannot ensure the city subsidies are minimized. HART needs to improve planning to maximize fare box recoveries. The farebox recovery rate is the percentage by which the fare revenues collected account for the total operating costs of the service. It is calculated by dividing the total fare revenue by its total operating costs. Our city comparison of farebox recovery rates of other transit agencies showed that fare revenues alone will not be sufficient to fund all rail operations and maintenance costs. Exhibit 4.4 shows selected rail services across the metropolitan areas of the United States. All of these rail services had farebox recovery rates which varied greatly and were consistently less than 100 percent. Fares were either variable29 or flat rate30. Of the selected rail comparisons, farebox recovery rates ranged from 77 percent for San Francisco’s Bay Area Rapid Transit (BART) service to 24 percent for Houston. The average farebox recovery rate for all selected rail services for 2013 was 43 percent. 29 A fare cost that varies in relation to the level of operational activity (time of day, distance travelled). 30 A fare cost that remains fixed irrespective to the level of operational activity. Attachment 1: HART’s Response to Audit Report Content Page 59 of 62 Appendix H: Management Response Exhibit 4.4 Rail Farebox Recovery Rates Comparison Source: OCA analysis of farebox recovery rates based on information from the National Transit Database (NTD), 2013 Our comparison indicated HART needs to plan for potential fare revenues, farebox recovery rates, and anticipated ridership levels to ensure the rail operations and maintenance costs are covered. HART needs to improve planning to maximize ridership. Our nationwide comparison of comparable rail systems indicated ridership levels have generally fallen short of forecasted levels. As part of the application process for the federal FFGA, HART developed ridership estimates and forecasts which were incorporated into the 2012 operations and maintenance plan (OMP). Our nationwide comparison indicated, however, that actual ridership fell short of the forecasted levels. (See Exhibit 4.5 below.) Attachment 1: HART’s Response to Audit Report Content Page 60 of 62 Appendix H: Management Response Exhibit 4.5 Ridership Forecast vs. Actual Sources: OCA analysis based on information from the Urban Transit Rail Projects: Forecast Versus Actual Ridership and Cost (1990), prepared by Dr. Don H. Pickrell for the Urban Mass Transit Administration; Predicted and Actual Impacts of New Starts Projects: Capital Cost, Operating Cost, and Ridership Data (2003), prepared by the Federal Transit Administration; The Predicted and Actual Impacts of New Starts Projects: Capital Cost and Ridership (2007), prepared by the Federal Transit Administration. (*) The federal FFGA program was established after the projects featured in the 1990 study. (**) As some forecasted years were beyond the scope of the 2003 and 2007 the FTA reports, “actual ridership” figures represent most recent numbers at the time the report was conducted. In our opinion, HART should prepare for a scenario in which actual ridership, or the number of passengers actually using rail, falls short of forecasted estimates. Its operations and maintenance plan is outdated and may result in inadequate revenues to cover the annual rail operating and maintenance costs. Attachment 1: HART’s Response to Audit Report Content Page 61 of 62 Appendix H: Management Response HART needs to fill a critical operations and maintenance position. HART’s Operations and Maintenance (O&M) director provides oversight related to the operations and maintenance of the rail system. The director is required to work closely with capital programs to provide operational and technical guidance during the design and construction phases. HART has not filled its O&M director position since the last occupant left the agency in August 2015. The city contract with Hitachi Rail Italy31 authorizes the company to operate and maintain the rail system and its passenger trains. According to HART, the Operations and Maintenance director will be responsible for administering the $1.4 billion operations and maintenance contract. The director will also be in charge of hiring a team of consultants to support the operations and maintenance function. Currently, no staff positions are assigned to support the operations and maintenance division. The vacant position and lack of support staff could leave HART unprepared when the rail becomes operational. Response: HART conducted a very diligent and extensive search for the candidate with the required high level experience and background to fill this position. This thorough search has paid off and HART has hired an individual with outstanding credentials who started on March 28, 2016, as the HART Director of Operations and Maintenance. Additionally, HART will be modifying the organizational structure to include a Deputy Director position under the Director of Operations and Maintenance. The expansion of this area and the reduction of other areas have already been identified in the staffing and succession planning since the beginning of the project. There is no “city contract” with Hitachi Rail Italy. HART’s contract continues to be with AHJV. NOTE: the footnote #31 below is incorrect: AHJV continues to be AHJV; “formerly”, therefore, is not correct. 31 Formerly Ansaldo Honolulu Joint Venture Attachment 1: HART’s Response to Audit Report Content Page 62 of 62 This page intentionally left blank.