United States Government Accountability Office Report to Congressional Addressees March 2020 MILITARY HOUSING DOD Needs to Strengthen Oversight and Clarify Its Role in the Management of Privatized Housing GAO-20-281 March 2020 MILITARY HOUSING DOD Needs to Strengthen Oversight and Clarify Its Role in the Management of Privatized Housing Highlights of GAO-20-281, a report to congressional addressees Why GAO Did This Study What GAO Found Congress enacted the Military Housing Privatization Initiative in 1996 to improve the quality of housing for servicemembers. DOD is responsible for general oversight of privatized housing projects. However, private-sector developers are responsible for the construction, renovation, maintenance, and repair of about 99 percent of military housing in the United States. Recent reports of hazards, such as mold and pest infestation, have raised questions about DOD’s oversight of privatized military housing. The Office of the Secretary of Defense (OSD) and the military departments conduct a range of oversight activities, but some of these activities have been more extensive than others. Specifically, GAO found that: Conference Report 115-952 included a provision for GAO to review ongoing issues within privatized housing. This report assesses, among other things, the extent to which OSD and the military departments (1) conduct oversight of privatized housing and (2) have developed and implemented initiatives to improve privatized housing. GAO reviewed policies and guidance; visited a nongeneralizable sample of 10 installations; analyzed work order data; and interviewed DOD officials and private partner representatives. • The military departments conduct some oversight of the physical condition of housing, but some efforts have been limited in scope. Military departments have authority to conduct oversight of the condition of privatized housing. That oversight generally consists of reviewing a sample of work order requests, visually inspecting housing during change-ofoccupancy, and conducting other point-in-time assessments. However, GAO found that these efforts are limited in scope. For example, annual interior walk-throughs are limited to just a few homes at some installations, which may not comprehensively reflect the condition of the housing units at those installations. • The military departments use performance metrics to monitor private partners, but the metrics do not provide meaningful information on the condition of housing. OSD has recently issued guidance to the military departments to ensure consistency in the framework used to measure project performance. However, the specific indicators used to determine if the metrics are being met may not accurately reflect performance related to the condition of the home. For example, a common indicator is how quickly the private partner responded to a work order, not whether the issue was actually addressed. • The military departments and private partners collect maintenance data on homes, but these data are not captured reliably or consistently. The Department of Defense (DOD) is expanding its use of work order data to monitor and track the condition of privatized housing. However, based on GAO’s analysis of data provided by all 14 private partners, these data cannot reliably be used for ongoing monitoring of privatized housing because of data anomalies and inconsistent business practices in how these data are collected. • DOD provides reports to Congress on the status of privatized housing, but some data in these reports are unreliable, leading to misleading results. DOD provides periodic reports to Congress on the status of privatized housing, but reported results on resident satisfaction are unreliable due to variances in the data provided to OSD by the military departments and in how OSD has calculated and reported these data. What GAO Recommends GAO is making 12 recommendations, including that DOD take steps to improve housing condition oversight, performance indicators, maintenance data, and resident satisfaction reporting as well as to assess the risk of the initiatives on project finances. DOD generally concurred with the recommendations and identified actions it plans to take to implement them. View GAO-20-281. For more information, contact Elizabeth A. Field at (202) 512-2775 or fielde1@gao.gov. OSD has made progress in developing and implementing a series of initiatives aimed at improving privatized housing. In addition, Congress established several requirements addressing privatization housing reform. However, DOD officials and private partner representatives have identified challenges that could affect implementation of these various initiatives. These include concerns that implementation could have unintended negative impacts on the financial viability of the privatized housing projects. However, DOD has not assessed the risk of the initiatives on project finances. United States Government Accountability Office Contents Letter 1 Background DOD Conducts Some Oversight of the Condition of Privatized Housing, but Efforts Are Limited in Key Areas Military Housing Offices Have Not Effectively Communicated Their Role as a Resource for Servicemembers Experiencing Challenges with Privatized Housing DOD and Private Partners Are Implementing Initiatives to Improve Privatized Housing, but May Face Challenges Conclusions Recommendations for Executive Action Agency Comments 42 49 50 52 Appendix I Objectives, Scope, and Methodology 56 Appendix II List of Privatized Military Housing Projects as of September 30, 2017 62 Appendix III Comments from the Department of Defense 69 Appendix IV GAO Contact and Staff Acknowledgments 75 Related GAO Products 5 11 35 76 Tables Table 1: Examples of Various Indicators That Metrics Measuring Private Partner Performance Comprise Table 2: Key Department of Defense (DOD) Initiatives Aimed at Improving Privatized Housing and Selected Related National Defense Authorization Act (NDAA) for Fiscal Year 2020 Requirements Page i 19 43 GAO-20-281 Military Housing Table 3: Privatized Military Housing Projects as of September 30, 2017 62 Figures Figure 1: Typical Funding Allocation for a Privatized Housing Project Figure 2: Military Departments’ Roles and Responsibilities in the Military Housing Privatization Initiative (MHPI) Figure 3: Examples of Completed and Ongoing Initiatives by Military Department to Improve the Oversight of Privatized Housing Figure 4: Examples of Differences in How Reports of Mold Were Recorded in Various Work Orders within and across Partner Work Order Tracking Systems Figure 5: Examples of Varying Levels of Detail in Military Housing Office Signage Figure 6: Flyer Advertising Marine Corps’ Three-Step Dispute Resolution Process 7 9 16 27 38 39 Abbreviations MHPI DOD OSD NDAA Military Housing Privatization Initiative Department of Defense Office of the Secretary of Defense National Defense Authorization Act This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page ii GAO-20-281 Military Housing Letter 441 G St. N.W. Washington, DC 20548 March 26, 2020 Congressional Addressees In 1996, Congress enacted the Military Housing Privatization Initiative 1 (MHPI) in response to Department of Defense (DOD) concerns about the effect of inadequate and poor quality housing on servicemembers and their families. 2 Since then, private-sector developers and property management companies, hereafter referred to as private partners, have assumed primary responsibility for military family housing in the United States. They are currently responsible for the construction, renovation, maintenance, and repair of about 99 percent of domestic military housing—more than 200,000 homes on and around military bases—in the continental United States, Alaska, and Hawaii. Over the last few years, reports of the presence of lead-based paint and other hazards, such as mold and pest and rodent infestation, have raised questions about DOD’s management and oversight of privatized military housing. We have previously reported on DOD’s privatized housing program. In 2018, we reviewed the financial condition of DOD’s privatized housing projects and found that DOD should take steps to improve monitoring, reporting, and risk assessment. 3 We recommended that DOD revise guidance to improve the consistency and comparability in terms of the time periods of the information reported on the financial condition of its privatized housing projects, fully assess the effects of reductions in basic allowance for housing on the projects, clarify when project changes 1National Defense Authorization Act for Fiscal Year 1996, Pub. L. No. 104-106, §§ 28012841 (1996), codified as amended at 10 U.S.C. §§ 2871-2885. 2DOD does not require servicemembers, other than certain key personnel and junior unaccompanied personnel, to live on an installation and thus, in privatized military housing. About a third of eligible servicemembers generally live in military housing and are provided with a basic allowance for housing to cover their living expenses. The basic allowance for housing payment is designed to enable servicemembers to live off-base comparably to their civilian counterparts. Therefore, DOD’s privatized housing competes with available housing options in the local market. Active-duty servicemembers are given priority for privatized military housing. However, projects can advertise and lease to tenants other than active-duty servicemembers, including civilians in some cases, once occupancy falls below a specific level. For example, the Air Force has approved leasing to other tenants when any given project’s occupancy rate falls below 98 percent. 3GAO, Military Housing Privatization: DOD Should Take Steps to Improve Monitoring, Reporting, and Risk Assessment, GAO-18-218 (Washington, D.C.: Mar. 13, 2018). Page 1 GAO-20-281 Military Housing require notice, and define tolerances for project risks. 4 DOD concurred with our recommendations and, as of 2019, was taking steps to address them. Specifically, DOD revised its reporting guidance to the military departments to ensure that financial data were consistent and comparable. In addition, DOD planned to update guidance to include a requirement to report on the risk of changes in the basic allowance for housing and to provide notification of project changes. A list of related products is included at the end of this report. The Conference report accompanying a bill for the Fiscal Year 2019 Department of Defense Appropriations Act included a provision for us to review ongoing issues within privatized military housing. 5 We assessed the extent to which the Office of the Secretary of Defense (OSD) and the military departments (1) conduct oversight of privatized military housing for servicemembers and their families, (2) have communicated their roles and responsibilities to servicemembers and their families, and (3) have developed and implemented initiatives to improve privatized housing. For each of our objectives, we included all privatized housing projects in each military department. We reviewed OSD and military department policies and guidance related to the implementation of the MHPI program and interviewed OSD and military department officials and representatives from each of the 14 private partners that are currently responsible for privatized housing projects. We visited a nongeneralizable sample of 10 installations selected to represent each of the military departments and six private partners—including the five largest who own the majority of privatized military housing—as well as 4DOD’s Office of the Deputy Assistant Secretary of Defense for Military Personnel Policy annually calculates rent and utility rates for locations across the United States based on estimates of local market conditions, which are then adjusted for an individual’s pay grade and dependency status. These calculations, which can fluctuate from year to year, are then used to determine individual servicemembers’ monthly basic allowance for housing payments. Servicemembers’ rent is paid—whether living on the installation or off—with basic allowance for housing payments. 5H.R. Rep. No. 115-952 (2018) (Conf. Rep.), accompanying a bill for the Department of Defense Appropriations Act for Fiscal Year 2019. Page 2 GAO-20-281 Military Housing geographic and climate diversity. 6 We reviewed the ground leases and other MHPI project documents for housing projects at each of these locations, and at each installation we interviewed local military department and housing office officials, as well as local private partner representatives. 7 In order to obtain resident perspectives on the condition of privatized housing at the 10 installations we visited, we facilitated 15 focus groups with over 70 residents. In addition, we developed and administered a publicly available online tool that provided a voluntary opportunity for any resident of privatized military housing to submit information on his or her experiences. We received and reviewed over 650 responses and eliminated those that did not meet our criteria. Information obtained from our focus groups and online tool are not generalizable to all privatized military housing residents. For objective one, we also collected information from the private partners on the structure of the performance incentive fees that they can receive under the terms of their ground leases. 8 We reviewed the performance incentive fee structures to determine the extent to which the indicators underlying them would allow the military departments to assess the private partners’ performance as it relates to the condition of the privatized housing units they maintain. To assess the extent to which private partner work order data could be used to monitor and track the condition of privatized homes, we identified and analyzed available private partner data from each of the 79 MHPI family housing projects on work orders from October 2016 through April 2019. We requested data from the private partners in April and May 2019, selecting a time frame that would include complete data for fiscal years 2017 and 2018 and the most comprehensive data available at the time for fiscal year 2019. To 6The 10 installations in our non-generalizable sample included three Army installations— Fort Bragg, North Carolina; Fort Huachuca, Arizona; and Fort Sill, Oklahoma; two Navy installations—Naval Station Norfolk, Virginia and Naval Base San Diego, California; two Marine Corps installations—Marine Corps Base Camp Lejeune, North Carolina, and Marine Corps Base Camp Pendleton, California; and three Air Force installations—DavisMonthan Air Force Base, Arizona; Langley Air Force Base, Virginia; and Tinker Air Force Base, Oklahoma. 7For a typical project, a military department leases land to a developer for a 50-year term and conveys existing housing units located on the leased land to the developer for the duration of the lease. The documents establishing these arrangements are referred to as “ground leases.” 8As of the end of fiscal year 2017, 74 of the 79 privatized housing projects had established incentive fee plans. Page 3 GAO-20-281 Military Housing assess the reliability of the data, we contacted representatives from each of the 14 private partners to discuss each company’s data system and potential limitations for using the data. We then performed manual testing on initial data files received from each partner to identify issues that would impact the validity and reliability of using these data for ongoing monitoring and tracking of the condition of privatized housing units. For example, we identified instances of duplicate records. While we found these types of data anomalies, as discussed later in the report, we determined that the data were sufficiently reliable for addressing our reporting objectives. To evaluate resident satisfaction data reported in OSD’s reports to Congress on privatized housing, we reviewed the processes for collecting, calculating, and reporting these data for the three most recently issued reports for fiscal years 2015, 2016, and 2017, focusing our detailed analysis on DOD’s fiscal year 2017 report. For objective two, we reviewed military department policies and guidance related to their roles and responsibilities for working with residents of privatized housing. We also reviewed ground leases and other MHPI project documents for the 10 installations we visited to identify guidance that articulated private partners’ and military departments’ roles and responsibilities. During our site visits to 10 installations, we interviewed military department housing office officials and private partner representatives to discuss their specific roles and responsibilities. We also asked questions soliciting information about residents’ understanding of the roles and responsibilities of the military housing office and the dispute resolution process during our 15 focus groups, information that we also sought through our publicly available online tool. For objective three, we interviewed OSD and military department officials to discuss ongoing initiatives developed over the course of our audit work that were aimed at improving MHPI. Following the passage of the National Defense Authorization Act for Fiscal Year 2020 (Fiscal Year 2020 NDAA), we reviewed provisions of the statute designed to improve the condition of privatized housing and evaluated the extent to which these provisions would impact ongoing or planned DOD initiatives or provide new oversight roles and responsibilities for OSD and the military departments. 9 9Pub. L. No. 116-92 (Dec. 20, 2019). Page 4 GAO-20-281 Military Housing We evaluated the extent to which the evidence we collected aligned with OSD policy and guidance for the oversight and management of privatized housing. We also compared OSD and the military departments’ policies and practices for management and oversight of MHPI with principles in Standards for Internal Control in the Federal Government focused on data collection and management accountability. Further details on our objectives, scope, and methodology can be found in appendix I. We conducted this performance audit from November 2018 to March 2020 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background Military Housing Privatization Authorities and Project Structures DOD’s policy is to ensure that eligible personnel and their families have access to affordable, quality housing facilities and services consistent with grade and dependent status, and that the housing generally reflects contemporary community living standards. 10 From the inception of MHPI, the military departments were provided with various authorities to obtain private-sector financing and management to repair, renovate, construct, and operate military housing in the United States and its territories. These authorities included the ability to make direct loans to and invest limited amounts of funds in projects for the construction and renovation of housing units for servicemembers and their families. 11 The projects were generally financed through both private-sector financing, such as bank loans and bonds, and funds provided by the military departments. The Army and the Navy generally structured their privatized housing projects as limited liability companies in which the military departments formed partnerships with the developers and invested funds in the partnership. 12 10Department of Defense (DOD) Instruction 4165.63, DOD Housing (July 21, 2008) (incorporating Change 2, Aug. 31, 2018). 11The authorities also provided DOD with the ability to provide loan and rental guarantees, make differential lease payments, and convey or lease property or facilities to eligible entities, among other things. 12A limited liability company is a company in which the liability of each shareholder or member is limited to the amount individually invested. Page 5 GAO-20-281 Military Housing The Air Force generally provided direct loans to the developers. Because privatized housing projects involve budgetary commitments of the federal government, each project was scored at inception by the Office of Management and Budget to determine the amount of funds that needed to be budgeted for that particular project. 13 The military departments have flexibility in how they structure their privatized housing projects, but typically the military departments lease land to developers for a 50-year term and convey existing housing located on the leased land to the developer for the duration of the lease. The developer then becomes responsible for renovating and constructing new housing and for the daily management of the housing units. At the end of fiscal year 2017, 14 private partners were responsible for 79 privatized military family housing projects—34 for the Army, 32 for the Air Force, and 13 for the Navy and the Marine Corps. 14 See appendix II for a list of all of these housing projects. Each privatized housing project is a separate and distinct entity governed by a series of legal agreements that are specific to that project, hereafter referred to as business agreements. 15 These agreements include, among other things, an operating agreement, a property management agreement, and an agreement that describes the management of funds in the projects, including the order in which funds are allocated within the project. However, while each project is distinct, there are some common elements in how projects invest and use funds. Every project takes in revenue, which consists mostly of rent payments. Projects then pay for operating expenses, including administrative costs, day-to-day maintenance, and utilities, among other things. After that, projects generally allocate funds for taxes and insurance, followed by debt payments. Figure 1 shows a typical funding structure for a privatized housing project. 13The Office of Management and Budget uses scoring to determine the amounts to be recognized in the budget when an agency signs a contract or enters into a lease. 14As of November 2019, these privatized housing projects were owned by the same 14 private partners. For the purpose of this report, we focused on privatized family housing projects and not housing for unaccompanied military personnel. 15Business agreements are alternately referred to as transaction documents or closing documents. Page 6 GAO-20-281 Military Housing Figure 1: Typical Funding Allocation for a Privatized Housing Project Note: If a project has received a loan from the government, as is the case with many Air Force projects, the next use of funds after operating expenses, taxes and insurance, and debt payments is the payment on the government loan debt. This step is not depicted in the figure, as Army and Navy projects generally do not include government loans. In the typical privatized housing project depicted in figure 1, once debt payments are made, funds are allocated to accounts that fund scheduled maintenance, such as repair and replacement of items like roofs, heating and cooling systems, and infrastructure. After that, funds are allocated to a series of management incentive fees, such as the property management fee. Finally, the project divides these remaining funds according to a fixed percentage between accounts that (1) fund major renovations and rebuilds and (2) are provided to the developer. The percentages may vary across agreements, but according to military department documentation, typically, the majority of funds go toward the accounts funding major renovations and rebuilds. Most of the projects’ business agreements also include the option for the private partners to receive performance incentive fees based on achieving the performance metrics established in each individual project’s business agreement. These fees are intended to incentivize private partner performance. The incentive fees can be paid to private partners Page 7 GAO-20-281 Military Housing on an annual or quarterly basis and can be withheld in part or in total if the private partner fails to meet the established metrics. The weight each performance metric and underlying indicator carries toward the incentive fee varies by project, so incentive fees for some projects may be heavily dependent on financial performance, while others may be more heavily weighted toward resident satisfaction. DOD Goals, Roles, and Responsibilities in the Privatized Housing Program The Deputy Assistant Secretary of Defense for Facilities Management, under the authority, direction, and control of the Assistant Secretary of Defense for Sustainment, is responsible for all matters related to MHPI and is the program manager for all DOD housing, whether DOD-owned, DOD-leased, or privatized. 16 In this capacity, the Deputy Assistant Secretary is to provide both guidance and general procedures related to military housing privatization, as well as required annual reports to Congress on privatized military housing projects. 17 However, it is the responsibility of the military departments to execute and manage the privatized housing projects, including conducting financial management and monitoring their portfolio of projects. Each military department has issued guidance that outlines its responsibilities for privatized housing, such as which offices are responsible for overseeing privatized housing projects. 18 See figure 2 for details on each military department’s roles and responsibilities in the MHPI program. 16Almost all DOD family housing in the United States has been privatized; however, DOD is still responsible for overseas family housing and most housing for unaccompanied military personnel in the United States. 17Section 2884(c) of Title 10 of the United States Code requires the Secretary of Defense to report semiannually an evaluation of the status of oversight and accountability measures for military housing privatization projects, including, among other things, information about financial health and performance and the backlog of maintenance and repair. According to DOD officials, although the statute requires semiannual reporting, due to the effort involved, DOD aims to produce one report for each fiscal year, rather than two. 18Air Force Instruction 32-6007, Privatized Housing Management (Sept. 19, 2012) (incorporating AFGM 2019-01, July 25, 2019); Department of the Army, Portfolio and Asset Management Handbook (Version 5.0) (Sept. 11, 2014); Commander, Navy Installations Command Instruction 11103.7A, Navy Housing Roles and Responsibilities for Program Management and Finance (Jan. 16, 2014); and Marine Corps Order 11000.22, Marine Corps Bachelor and Family Housing Management (July 14, 2014) (incorporating Change 1, Jan. 22, 2018). Page 8 GAO-20-281 Military Housing Figure 2: Military Departments’ Roles and Responsibilities in the Military Housing Privatization Initiative (MHPI) Department of the Army, Portfolio and Asset Management Handbook (Version 5.0) (Sept. 11, 2014). a Chief of Naval Operations, OPNAV Instruction 5009.1, Responsibility for Navy Housing and Lodging Programs (Dec. 26, 2007). b c Commander, Navy Installations Command, Instruction 11103.7A, Navy Housing Roles and Responsibilities for Program Management and Finance (Jan. 16, 2014). d Marine Corps Order 11000.22, Marine Corps Bachelor and Family Housing Management (July 14, 2014) (incorporating Change 1, Jan. 22, 2018). e Air Force Instruction 32-6007, Privatized Housing Management (Sept. 19, 2012) (incorporating AFGM 2019-01, July 25, 2019). Page 9 GAO-20-281 Military Housing Prior GAO Work We have previously reported on DOD’s privatized housing program. In 2002, we reported that although military installation officials were participating with developers in making improvement decisions for selected projects, DOD and military department headquarters oversight of those decisions appeared limited. 19 We recommended, among other things, that DOD implement several changes to enhance government protections in the privatization program, such as requiring service headquarters and the OSD to review and approve privatization project reinvestment account expenditures over an established threshold. DOD generally agreed with our recommendations and took steps to implement them. Specifically, DOD revised guidance to establish new rules and thresholds for review and approval of project reinvestment expenditures, among other things. In addition, in 2006, we reported that although DOD and the individual military departments implemented program oversight policies and procedures to monitor the execution and performance of privatized housing projects, opportunities existed for improvement. 20 Specifically, we reported that the value of DOD’s semiannual report to Congress was limited because it lacked a focus on key project performance metrics to help highlight any operational concerns. We also reported that data collected on servicemember satisfaction with housing, important for tracking satisfaction over time, were inconsistent and incomplete because DOD had not issued guidance for the standardized collection and reporting of such information. We recommended, among other things, that DOD streamline its report to Congress to focus on key project performance metrics and to provide guidance to the military departments to ensure the consistent collection and reporting of housing satisfaction information from all servicemembers. DOD generally agreed with our recommendations and took steps to implement them. For example, DOD took steps to streamline its report to Congress and update its guidance directing the services to ensure consistent reporting using a numerical rating system to rank housing satisfaction information. 19GAO, Military Housing: Management Improvements Needed as the Pace of Privatization Quickens, GAO-02-624 (Washington, D.C.: June 21, 2002). 20GAO, Military Housing: Management Issues Require Attention as the Privatization Program Matures, GAO-06-438 (Washington, D.C.: Apr. 28, 2006). Page 10 GAO-20-281 Military Housing DOD Conducts Some Oversight of the Condition of Privatized Housing, but Efforts Are Limited in Key Areas OSD and each of the military departments conduct a range of activities to oversee both the condition of privatized housing and performance of the private partners and have recently implemented initiatives to improve this oversight—such as increasing the frequency of the physical inspection of homes and issuing guidance to ensure consistency in the framework used to measure project performance. However, we found that these oversight efforts remain limited. Specifically, our review showed (1) the scope of oversight of the physical condition of privatized housing has been limited; (2) performance metrics focused on quality of maintenance and resident satisfaction may not accurately reflect private partner performance related to the condition of privatized housing; (3) there is a lack of reliable or consistent data on the condition of privatized housing; and (4) past DOD reports to Congress on resident satisfaction are unreliable due to the inconsistent handling and calculation of the data and therefore may be misleading. Military Departments Conduct Some Oversight of the Physical Condition of Privatized Housing, but the Scope of These Efforts Is Limited The military departments have taken steps to oversee the condition of their privatized military housing inventory and each has issued guidance that outlines their respective oversight roles and responsibilities, but the scope of these oversight efforts has been limited. Military department oversight activities generally fall into two categories—(1) daily oversight of management and operations and (2) periodic reviews of compliance with each project’s business agreements. Daily oversight of management and operations. Each installation has a military housing office that is responsible for conducting daily oversight of a project’s management and operations. Military housing officials told us that activities to monitor the physical condition of housing units generally include reviewing sample work order requests, following up with a sample of residents to check on their experience with recently completed work, and inspecting homes during the change-of-occupancy process. However, the implementation and scope of these activities varies and can be limited. For example, during our site visits conducted from June through August 2019, we identified the following installationspecific practices: • The rate of inspections of homes following change-of-occupancy maintenance at the installations we visited varied. For example, at the time of our site visits, military housing office officials at Tinker Air Force Base, Oklahoma, told us that they inspect 100 percent of homes that have completed change-of-occupancy maintenance, while officials from Langley Air Force Base, Virginia, stated that they inspect 10 to 20 percent of these homes. In November 2019, Air Force Page 11 GAO-20-281 Military Housing officials told us that they are moving to a 100-percent inspection policy. Similarly, the Army issued an order in March 2019 directing military housing office officials to inspect 100 percent of homes where change-of-occupancy maintenance has been completed. Officials from Army installations we visited noted that this was an increase from previous practices, and for one installation was a change in practice from conducting inspections only during the move-out process, which occurs prior to change-of-occupancy maintenance. • According to Department of Navy officials, the Navy’s business agreements stipulate that Navy and Marine Corps installations have access to all work order information. However, practices for following up on work order records varied among some of the Navy and Marine Corps installations we visited. For example, military housing office officials at Camp Pendleton, California, told us that for one of the two partners that own housing on the base, they had access to only 3 percent of completed work orders from the previous month. For the other partner that owns housing on the base, military housing office officials noted that the partner provided them with nine work orders of varying priority each month to review. One military housing office official added that these were the minimum requirements needed for monthly reporting and that they were working with the private partner to increase their access to work order records. Following a different practice, military housing office officials at Naval Station Norfolk, Virginia, told us that they had access to the private partner’s maintenance record system and would pull reports on homes that had made six or more maintenance calls in a 30-day period. Periodic reviews of compliance with each project’s business agreements. Periodic reviews of compliance with a project’s business agreements are a joint effort between the local military housing office, the private partners, military department installation commands, and other echelons of command. 21 These reviews can include neighborhood tours to view project amenities such as community centers, playgrounds, and pools, all of which are owned, maintained, and operated by the private partner companies, as well as exteriors of housing units. However, similar to the daily oversight activities, these annual reviews have been narrow in 21In addition to installation-level chain of command, other echelon commands from each service involved in privatized housing projects include: the Air Force Civil Engineer Center; the Air Force Material Command; the Department of Army; the Army Materiel Command; the Army Installation Management Command; the U.S. Army Corps of Engineers; the Commander; Navy Installation Command; the Naval Facilities Engineering Command; and the Marine Corps Installations Command. Page 12 GAO-20-281 Military Housing the scope of their assessment of the physical condition of the housing units, as interior walk-throughs were, at times, focused on just a few homes at each installation. 22 For example: • The Air Force Civil Engineer Center is the primary oversight and governance body for the Air Force’s privatized housing projects. 23 The Air Force oversight process includes periodic compliance reviews of all privatized housing projects. To accomplish this task, the Air Force is to use a compliance checklist to review the private partner’s compliance with a project’s business agreements. In addition to the compliance reviews, guidance states that Air Force Civil Engineer Center officials visit projects annually, and officials told us that they tour a sample of homes and interview private partner representatives, military housing office staff, and residents during these visits. However, according to selected annual site visit reports we reviewed and a discussion with an Air Force official, annual site visit reports typically include only an evaluation of three to four housing units on an installation and can be restricted to empty units or units that have completed change-of-occupancy maintenance, limiting the robustness of the assessment of the installation’s housing units’ physical condition. • According to Department of the Navy officials, the Navy and the Marine Corps provide oversight of privatized housing projects through a tool called the monitoring matrix. Officials from the various organizational entities involved with privatized housing—to include the Commander, Naval Installation Command; the Naval Facilities and Engineering Command; and the military housing office—are to use this monitoring matrix to periodically review private partner compliance with a project’s business agreements. The matrix contains a condition assessment component, which includes a tour of privatized housing neighborhoods and a visual inspection of individual privatized housing units. However, similar to the Air Force, according to select assessments we reviewed and a discussion with a military 22In spring 2019, according to military department officials, each department conducted a “100 percent” review of privatized housing by directing installation commanders to contact all residents of privatized housing and offering a visual inspection of their privatized housing unit. 23In October 2012, the Air Force Center for Engineering and the Environment and the Air Force Real Property Agency merged to create the Air Force Civil Engineer Center. While Air Force Instruction (AFI) 32-6007 still refers to the Air Force Center for Engineering and the Environment as having oversight of privatized military housing, the Air Force Guidance Memorandum updating the instruction makes clear that those responsibilities have been assumed by the Air Force Civil Engineer Center. Page 13 GAO-20-281 Military Housing housing office official, the visual inspections are typically focused on two to three homes in each neighborhood on an installation and to homes that have recently undergone change-of-occupancy maintenance. • Army guidance calls for the U.S. Army Corps of Engineers to conduct an annual ground lease inspection to review private partner compliance with a project’s business agreements. The guidance also calls for the Army’s program manager to conduct an annual installation visit to each project to evaluate performance and ensure a project’s compliance with the business agreements. The visit is to include a recommended site tour, described in guidance as a brief visual inspection tour of community elements, and a walk-through visual inspection of at least four housing units—two renovated and two recently built—including one unit designated as an accessible home under federal guidelines. However, according to a May 2019 report by the Army Inspector General, these requirements were inconsistently met, and the results did not include a follow-up process and were not communicated to senior commanders. 24 Through the recent housing reviews that they have conducted, each military department’s internal oversight body has recognized that the departments’ oversight guidance has been limited in addressing the condition of privatized homes and provides little clarity to housing officials about their roles and responsibilities in assessing the physical condition of homes. For example, in May 2019, the Department of the Army Inspector General reported that senior commanders and garrison staffs expressed confusion concerning the roles, responsibilities, and authorities regarding privatized housing and that oversight, governance, and synchronization were insufficient to identify current housing challenges. Similarly, an April 2019 report from the Air Force Inspector General noted that ambiguous guidance had resulted in inconsistent action and uneven performance across Air Force housing projects. 25 In addition, a November 2019 report by the Naval Audit Service identified nine separate guidance documents for the oversight of privatized housing and found that personnel at 24Department of the Army Inspector General, Department of the Army Inspector General Special Interest Item, Assessment of the Residential Communities Initiatives (RCI)(May 15, 2019). 25Air Force Office of Inspector General, Directed Inspection Report: Military Privatized Housing Health and Safety Complaint Resolution, (Apr. 9, 2019)(U//FOUO). Page 14 GAO-20-281 Military Housing installation and regional levels were unclear on the guidance and requirements for performing oversight of privatized housing. 26 According to military department officials, each department has completed initiatives and is undertaking initiatives to revise guidance and standardize daily oversight activities in an effort to provide consistent oversight across projects and installations and to increase the focus on the physical condition of housing. In addition, the military departments have initiatives to increase staffing levels, improve training for military housing office officials, and ensure that military department housing officials have independent access to work order data to strengthen their oversight activities. Figure 3 outlines examples of completed and ongoing initiatives by military department to improve the oversight of privatized housing. 26Naval Audit Service, Audit Report, Navy Public-Private Venture Military Base Housing (Nov. 6, 2019). Page 15 GAO-20-281 Military Housing Figure 3: Examples of Completed and Ongoing Initiatives by Military Department to Improve the Oversight of Privatized Housing However, each military department is working to implement servicespecific initiatives with minimal guidance from OSD on the level of oversight expected as it relates to the condition of privatized housing. OSD guidance as it pertains to the condition of privatized housing is limited compared with the guidance OSD provides for monitoring the condition of military-owned housing. Page 16 GAO-20-281 Military Housing Specifically, OSD guidance is focused on the oversight of the implementation of projects, the construction of new housing units, and project financial monitoring. The guidance stipulates that after privatized housing projects are awarded, monitoring should include descriptions of deal structure and strategies for project monitoring. In contrast, OSD guidance for military-owned housing provides clearly defined objectives to the military departments for oversight, including the physical condition of the homes. For example, the DOD manual for housing management directs the military departments to provide managerial oversight of DOD’s government-owned family housing to ensure that (1) the required inventory is being provided and maintained in good condition, (2) the program is being operated in an effective and cost-efficient manner, and (3) servicemembers and their families have adequate housing choices. 27 Further, the manual provides specific objectives for the condition of DOD’s government-owned family housing, stating that for DOD family housing to be considered adequate overall, it must meet minimum standards for configuration, privacy, condition, health, and safety. It also states that military service condition assessments shall use private-sector housing industry and DOD standards or codes as a basis for assessing inventory adequacy. The manual adds that for DOD government-owned family housing to be considered in adequate condition, the construction cost for all needed repairs and improvements cannot exceed 20 percent of the replacement cost. According to DOD’s housing manual, program assumptions for privatized housing are that privatization allows the military departments to work with the private sector to generate housing built to market standards. While the military departments’ policies provide for some measureable oversight activities, such as requiring a certain number or type of home to be inspected, OSD has not provided guidance to the military departments clearly defining oversight objectives for monitoring the physical condition of privatized housing units. DOD’s housing manual further states that because privatization creates a long-term governmental interest in privatized housing, it is essential that projects be attentively monitored. The 50-year term for the ground leases creates a long-term interest in monitoring the privatized housing assets, to include the physical condition of the housing units. However, unless DOD updates its guidance on the oversight of privatized housing with 27DOD Manual 4165.63, DOD Housing Management, (Oct. 28, 2010) (incorporating Change 2, Aug. 31, 2018). Page 17 GAO-20-281 Military Housing objectives for overseeing the physical condition of housing units, it cannot be assured that the military departments’ oversight activities will be sustained over time or be sufficiently consistent across projects, raising the risk that private partners may not provide adequate quality housing. Notably, the military departments have entered into privatized housing agreements with some of the same companies, and members of different military services may live at installations managed by military services different than their own. As such, it is important that oversight expectations generally be consistent across the military departments and the projects they manage. Moreover, all military departments have an interest in ensuring that residents feel confident that the private partners will be held to a consistent standard for maintaining the condition of their homes. Participants in 8 of our 15 focus groups stated that they will no longer live in privatized housing following their current experience, and participants in 6 of our 15 focus groups stated that their current experience with privatized housing will affect the future career decisions for their family. One participant stated that he plans to exit the service after 8 years, noting that his decision is largely based on his experience with privatized housing. In addition, in our online tool we asked residents if their experience with privatized housing would impact their future career and housing decisions. For those residents that responded to these questions, the majority said their experience will make them less likely to continue to live in privatized housing in the future. For example, one respondent stated that while living in privatized housing is a benefit to being in the military, living in housing that is subpar and where nothing seems to be getting fixed or at least acknowledged makes the family hesitant to live in privatized housing again. Some residents also indicated that their experience would impact their future career decisions. DOD Uses Several Metrics to Monitor Private Partner Performance, but the Indicators Underlying Those Metrics May Not Provide Meaningful Information on the Condition of Privatized Housing The military departments each use a range of project-specific performance metrics to monitor private partner performance. However, the indicators underlying the metrics designed to focus on resident satisfaction and on the quality of the maintenance conducted on housing units may not provide meaningful information or reflect the actual condition of the housing units. For example, in April 2019 the Air Force Inspector General reported that the current incentive structure measures Page 18 GAO-20-281 Military Housing many things with precision, but does not measure the right things. 28 Private partner performance is commonly measured through four key metrics—resident satisfaction, maintenance management, project safety, and financial management. 29 To determine how well the private partners are performing under these metrics, military housing office officials told us that they rely on a range of indicators established in the project business agreements. Table 1 provides examples of various indicators that the performance metrics comprise. Table 1: Examples of Various Indicators That Metrics Measuring Private Partner Performance Comprise Metric Indicator Maintenance management • • • Resident satisfaction • • • • • Project safety • • • Financial management • • • Timeliness of change-of-occupancy maintenance Timeliness of response to service orders Achievement of preventative maintenance requirements Management of relationship with the residential community office Yearly customer satisfaction survey results Work order survey results Resident move-in survey results Resident move-out survey results Partner performance in safety and property management Completion of monthly safety inspections Safety meetings/training Timely and reliable financial reporting Management of the project’s net operating income Management of controllable expenses Source: GAO analysis of various performance incentive fee plans. GAO-20-281 According to officials from each military department, the performance metrics and their underlying indicators are a key tool that each military department uses to hold private partners accountable for providing quality management of the privatized housing projects. However, we found that the indicators themselves may not reflect how the private partner is 28Air Force Office of Inspector General, Directed Inspection Report: Military Privatized Housing Health and Safety Complaint Resolution, (Apr. 9, 2019)(U//FOUO). 29Most, but not all, of the private partners are eligible to receive performance incentive fees based on achieving the performance metrics established in each individual project’s business agreement. As of the end of fiscal year 2017, we identified 5 of the 79 privatized housing projects that did not have an established incentive fee plan. The projects are Fort Carson, Colorado; Hill Air Force Base, Utah; Dyess Air Force Base, Texas; Robins I Air Force Base, Georgia; and Robins II Air Force Base, Georgia. Page 19 GAO-20-281 Military Housing performing in terms of providing servicemembers and their families with quality services and housing. For example: • Maintenance management: One commonly used indicator of performance in maintenance management measures how often the property manager’s response time to work orders meets required time frames established in the project’s business agreements. 30 While this indicator measures the timeliness of the private partner’s response, it does not measure or take into account the quality of the work that was conducted or whether the resident’s issue was fully addressed. As such, a property manager may fully meet the metric for maintenance management, even if a given repair has not been adequately completed. Residents in 13 of our 15 focus groups noted that they typically have had to submit multiple work order requests before an individual maintenance issue has been fully addressed. For example, a resident who participated in one of our focus groups provided us with a copy of work orders she had submitted related to a single maintenance issue in her home. The first work order was marked completed on time, yet the resident had to submit a work order for the same issue a week later. Further, an official at one Army installation told us that since the incentive fee for the project is awarded on a quarterly basis, judging property managers only on the basis of work orders completed on time for that quarter could mask persistent ongoing housing problems. This is because many smaller work orders get closed out each quarter, while work orders for more complicated issues might stay open over multiple quarters. Some projects include indicators that aim to more directly measure quality, such as the number of work orders placed during the first 5 business days of residency. This type of indicator may more clearly indicate the extent to which change-of-occupancy maintenance was complete on a given home. • Resident satisfaction: One example of an indicator of resident satisfaction is whether a project has met target occupancy rates established in the project’s business agreements. An OSD official and private partner representatives told us they use occupancy as an 30Work orders may be categorized as emergency, urgent, or routine and have preestablished required time frames that the private partners response is evaluated against based on a specified amount of time as outlined in the projects business agreement. For example, the performance incentive fee plan for the project at Camp Lejeune, North Carolina, stipulates that the private partner is required to respond to work orders classified as emergency within ½ hour during normal working hours and within 1 hour after business hours, to urgent work orders within 4 hours, and to routine work orders within 3 days. Page 20 GAO-20-281 Military Housing indicator of satisfaction, based on the assumption that residents would move if they are dissatisfied with their home’s condition. However, according to the Army’s Portfolio and Asset Management Handbook, occupancy rates are not a recommended metric to monitor private partner performance because occupancy rates already impact project finances. 31 Our focus groups and the responses we received to our online tool also indicate that this may not be a reliable assumption. Although most residents are not required to live in military housing, residents in each of our 15 focus groups and responses to our online tool indicated a variety of reasons for choosing to live in privatized housing, many of which do not have to do with their satisfaction with the quality or condition of their homes. For example, residents in our focus groups cited other factors influencing their decision to live in privatized housing, such as living in close proximity to military medical or educational services for children or other family members that are part of the military’s Exceptional Family Member Program, a lack of safe and affordable housing in the surrounding community, and access to quality schools. 32 Volunteers that responded to our online tool also cited accessibility to base services, commute time, and safety as reasons for choosing to live in privatized housing. 33 31U.S. Army, Portfolio and Asset Management Handbook: Residential Communities Initiative Privatized Army Lodging, (Version 5.0) (Sept. 11, 2014). Despite the Army’s Portfolio and Asset Management Handbook’s recommendation that occupancy not be used as a metric, several Army projects still use occupancy as a metric to monitor private partner performance. 32The Exceptional Family Member Program provides servicemembers and their dependents with special medical and educational needs. In each of our 15 focus groups we asked residents to tell us “What were the factors that led you to make the decision to live in privatized housing vs. a rental home in the community?” In 4 out of our 15 focus groups, residents indicated that accessibility to services for an Exceptional Family Member Program family member played a role in their decision to live in privatized housing versus a rental home in the community. In these 15 groups, we also asked residents to tell us “What were the factors that led you to make the decision to live in privatized housing vs. a rental home in the community?” Residents in 11 out of 15 focus groups indicated that affordability and 10 out of 15 indicated that safety played a role in their decision to live in privatized housing versus a rental home in the community. Additionally, in each of our 15 focus groups and in our online tool, we asked residents to tell us “What were the factors that led you to make a decision to live in privatized housing vs. a rental home in the community?” In our focus groups, residents in 9 out of 15 focus groups indicated that schools played a role in their decision to live in privatized housing versus a rental home in the community. 33For our online tool, residents were specifically asked about several factors that lead them to live in privatized housing. Of the residents that responded to commute time, accessibility to base services, and safety as factors, a majority indicated that these played a role in their decision. Page 21 GAO-20-281 Military Housing Another commonly used indicator of resident satisfaction is the results of various resident satisfaction surveys, such as maintenance surveys and leasing surveys, as well as the annual satisfaction survey. The military departments and the private partners use these survey tools to gauge resident satisfaction with the maintenance conducted on their homes, service provided by property managers, and amenities provided in their community, among other things. However, residents in 4 out of our 15 focus groups indicated that the surveys they receive related to maintenance performed on their homes do not ask questions about the quality of maintenance work. For example, residents told us that maintenance surveys, which they generally receive after maintenance work is completed on their homes, ask if the maintenance worker was courteous, but not about the quality of the work performed on the home. We reviewed maintenance surveys from 3 of the 10 installations we visited and found that the surveys asked residents to provide feedback on the quality of the work, with questions asking them to rate their satisfaction with the quality of the maintenance work completed. In addition, we reviewed a quarterly Army survey from one of the installations we visited and found that this survey asked residents about their satisfaction with the courteousness and professionalism of the maintenance team and the responsiveness and timeliness of maintenance work, but did not specifically ask about their satisfaction with the quality of the maintenance work completed. We also found that the information used to support the indicators can vary. For example, officials at one Army installation—Fort Huachuca, Arizona—use quarterly resident surveys, the Army’s annual survey, and action plans on Army annual survey results as indicators of resident satisfaction. However, officials at another Army installation—Fort Knox, Kentucky—use residential community office relationship management and point of service surveys. Similarly, we found differences in the information used as indicators of the maintenance management metric. For example, officials at both Hickam Air Force Base, Hawaii, and DavisMonthan Air Force Base, Arizona, rely on the timeliness and quality of change-of-occupancy maintenance as an indicator of maintenance management. However, officials at Hickam Air Force Base also use work order response and completion times as indicators of the maintenance management metric, whereas officials at Davis-Monthan Air Force Base, Arizona, only use work order response times. Standards for Internal Control in the Federal Government state that management should evaluate performance and hold individuals Page 22 GAO-20-281 Military Housing accountable for their internal control responsibilities. If management establishes incentives, management should recognize that such actions can yield unintended consequences and evaluate incentives so that they align with the entity’s standards of conduct. 34 The standards further state that management should use quality information to achieve the entity’s objectives, including relevant data from reliable sources. In October 2019, OSD, in collaboration with the military departments and private partners, issued new guidance standardizing the performance incentive fee framework across the military departments. The new guidance provides a framework for standardizing the minimum and maximum percentages of the fee that each metric can account for, allowing for some flexibility in the weight each metric will carry for an individual project. Specifically, maintenance management and resident satisfaction can account for between 60 and 90 percent of the fee, project safety can account for between 5 and 15 percent of the fee, and financial performance can account for between 5 and 15 percent of the fee. However, despite DOD’s efforts to ensure more focus on the condition and quality of, and resident satisfaction with, privatized housing through the standardization of metrics across the military departments, the metrics may be misleading if the specific underlying indicators used to determine whether a metric has been reached are not reevaluated on an ongoing basis to ensure they are accurate measures of the private partners’ performance and an accurate reflection of the condition and quality of privatized homes. OSD and military department officials have recognized that the current indicators for measuring performance do not consistently focus on or prioritize the private partners’ performance with maintaining housing units and ensuring resident satisfaction. For example, Army officials told us they are no longer using occupancy rate as an indicator of resident satisfaction and have taken steps to standardize performance indicators across all Army projects, while still allowing for flexibility at the installation level to modify the weight of indicators to provide incentives reflective of the specific needs of the installation. Limitations to the current indicators may hinder the military departments’ ability to accurately determine private partner performance. OSD and the military department officials told us they have not yet reevaluated the specific indicators used to determine whether a private partner has met a specific metric because doing so will require negotiation with each of the private partners for each project. However, without reviewing the specific indicators used to award 34GAO-14-704G. Page 23 GAO-20-281 Military Housing performance incentives, OSD and the military departments do not have assurance that the information the military departments are using to award these incentives reflects the actual condition of the housing. DOD and Private Partners Collect Maintenance Data on Privatized Housing, but These Data Are Not Captured Reliably or Consistently for Use in Ongoing Monitoring of Housing Units Maintenance data collected by the private partners are not captured consistently or reliably across projects for use in ongoing monitoring of the condition of privatized housing units over time. The privatized housing projects’ business agreements typically include a requirement for the private partner to maintain a records management system to record, among other things, maintenance work requested and conducted on each housing unit. According to private partner representatives from all 14 companies, each company uses commercial property management software platforms for activities such as initiating maintenance work orders and dispatching maintenance technicians. 35 Some private partner representatives stated that while data from the work order tracking systems are primarily used to prioritize and triage maintenance work, the data were never intended to monitor the overall condition of privatized housing units. Military department officials told us that efforts are underway to monitor work order data from the private partners’ work order tracking systems in an effort to increase the military departments’ oversight and accountability of the private partners for providing quality housing to servicemembers. For example, the Army and the Navy are taking steps to create data dashboards to track installations’ work orders by priority, status, and category. 36 However, while data from these work order tracking systems may be useful for point-in-time assessments of work order volume at a given installation, we found that these data are not captured reliably or consistently for use in ongoing monitoring of the condition of privatized housing units across projects and over time. We received and reviewed data from each of the 14 private partners’ work order tracking systems covering each of the 79 privatized housing 35As of February 2019, according to private partner representatives, they primarily used commercial property management software platforms from two companies. For the purposes of this report, we will refer to these software platforms as “work order tracking systems” and data from these systems as “work orders.” 36Work orders may include an associated priority level, such as “emergency,” “urgent,” or “routine;” a status, such as “open,” “completed,” “in-progress,” or “cancelled;” and a category, such as “electrical” or “plumbing.” Page 24 GAO-20-281 Military Housing projects. 37 Based on our review of these data and discussions with private partner representatives, we found two primary factors that would limit the reliability or consistency of using these data for ongoing monitoring of the condition of privatized housing units over time—(1) inconsistent use of terminology in work order records and (2) differing practices for opening and closing work orders. Inconsistent Use of Terminology in Work Order Records Data in these work order tracking systems include information such as records of resident requests for service, history of work conducted on specific housing units, change-of-occupancy maintenance performed, and work completed on common areas. Residents may request service for a broad range of issues, such as lost keys, broken appliances, ceiling or wall damage, lack of hot water, or water leaks or floods. According to private partner representatives, work orders can be entered into the system by property management office staff, maintenance technicians, or call center representatives for those companies that use offsite call centers to process resident service request calls. At some installations, residents can also enter work orders into the work order tracking system through online portals or mobile applications. However, we noted cases where work orders were inconsistently entered into the work order tracking systems with respect to two primary factors— (1) how the request is described by the resident or interpreted by the individual entering the data, which can differ for each work order; and (2) the existing range of pre-established service category options in the private partner’s work order tracking system, which differ among the partners. According to private partner representatives, the individual responsible for entering the work order into the system—property management office staff, maintenance technicians, call center representatives, or residents—makes a judgment on how to categorize the work order. These factors create challenges for looking at the data across projects. Private partner representatives from one installation we met with stated that the quality of the work order data is dependent on the data input into the system. In some cases, the data input can be inaccurate or imprecise, depending on the specificity with which a resident describes his or her maintenance issue or how a staff person enters the data into the system. A private partner representative from another installation we visited stated 37The private partners provided us with over 8 million work order records from fiscal year 2017, fiscal year 2018, and the first 7 months of fiscal year 2019. Page 25 GAO-20-281 Military Housing that reporting on data from the work order tracking system can be challenging because individuals across installations inputting data may have a different interpretation of a resident’s reported issue. Private partner representatives from another installation noted that the work order tracking system they used could not be easily updated with a new category if needed, making it more difficult to identify systemic issues. For example, there is one category for all exterior repairs, but no way to break that category down into what the specific repairs are, such as roofs. In the event that there is an issue with several roofs in the same area, the private partner representative said it would be hard to identify the issue because the only option available is to look through the notes section. According to this individual, the regional maintenance technicians, not the work order tracking system, are the best resource for identifying trends or recurring issues. This inconsistent entering of information into the work order tracking systems, which occurs both within and across installations, means that the military departments cannot readily use the data to capture the prevalence of a particular issue, such as mold, among the homes in a reliable manner. For example, if someone wanted to use work order data to track instances of mold, he or she would find that these may be represented in the work order systems under a variety of service categories, such as mold or mildew, plumbing and bath, heating and cooling, or general. To isolate service requests related to mold, one may have to rely on using the service comments for each request, which can vary in their level of detail. In addition, service requests for mold issues may be entered into the work order systems under different priority levels, such as routine, urgent, or emergency. As a result of the variation in the type and amount of information collected in the work order tracking systems, work order data alone cannot be used to determine the validity of a service request, the severity of the problem, or whether the work was completed to a quality standard. Figure 4 shows examples of differences in how a perceived mold issue can be captured in these systems based on our review of the data provided by the private partners. Page 26 GAO-20-281 Military Housing Figure 4: Examples of Differences in How Reports of Mold Were Recorded in Various Work Orders within and across Partner Work Order Tracking Systems Military department officials found similar limitations when analyzing the work order data. According to some officials, one challenge in using the work order data for oversight is that, while there are good data in the individual records, people report and record things differently. Specifically, Page 27 GAO-20-281 Military Housing a Navy official working with these data told us they have to consider these differences and create unique algorithms to query data for each partner. Differing Practices for Opening and Closing Work Orders At some installations we visited, private partners noted changes in practices for opening or closing work orders, limiting the usefulness of data in monitoring the status of work orders over time and thus the condition of privatized housing. For example, according to private partner representatives at one installation we visited, a practice for tracking emergency work orders in the work order tracking system had changed in 2013. Work that comes in under an emergency priority may take several steps to complete: A maintenance technician may first have to stop the emergency, then clean up any resulting damage, before repairing the root cause and completing any finishing work. Prior to 2013, maintenance technicians would open and close new work orders for each step in the process. Under the new practice, the original work order is kept open until completion. Representatives from a different private partner described a similar change in practices, noting that if a work order was closed or recategorized before the work was finished there could be issues for how it is tracked, such as getting dropped out of the system and the work not getting done. A third partner noted the same practice, but added that an emergency work order can be downgraded to urgent or routine status during the time that the work is taking place. As a result, work order data alone may not accurately identify the number of open work orders at any given time, the time it took to address a maintenance issue, or if a maintenance request has been fully completed. Additionally, we identified anomalies in the work order data provided to us from each of the 14 partners. For example, we identified instances of, among other things, duplicate work orders, work orders with completion dates prior to the dates that the resident had submitted the work order, and work orders still listed as in-progress for more than 18 months. According to military department officials, they have increased their efforts to review data from the private partners’ work order tracking systems and have found similar anomalies. For example, a Navy official working with work order data found that a couple of homes had six or seven unique work order records in the system, but each contained identical information in the various data fields. Officials from both the Navy and Air Force have come across work order records that were marked as complete within minutes of being entered into the system or marked as complete with a date prior to the work order being open, which signaled the need for further scrutiny. Page 28 GAO-20-281 Military Housing Each military department has efforts underway to monitor private partner work order data in an effort to increase oversight of the quality of privatized housing. However, because neither OSD nor the military departments have identified minimum data requirements, established consistent terminology or practices for data collection, or developed processes for the military departments to validate the work order data collected by the private partners, data from these work order tracking systems are not reliable for use in the ongoing monitoring of the condition of privatized homes. Further, military department data monitoring efforts are department-specific, even though the departments have entered into privatized housing agreements with some of the same companies. Standards for Internal Control in the Federal Government state that management should use quality information to achieve the entity’s objectives and design information systems and related control activities to achieve objectives and respond to risks. 38 Information, among other things, should be complete and accurate. The standards also state that management should define the identified information requirements at the relevant level and requisite level of specificity for appropriate personnel. Without direction from OSD to establish minimum data requirements and consistent terminology or practices for data collection, as well as a requirement for the military departments to validate the data, the military departments’ ability to use data from the private partners’ work order tracking systems to monitor the condition of privatized homes over time is limited and may vary across projects. DOD Provides Reports to Congress on Resident Satisfaction with Privatized Housing, but Data in These Reports Are Unreliable, Leading to Misleading Results DOD has provided periodic reports to Congress on the privatized housing program; however, reported results on resident satisfaction have been unreliable and are misleading due to (1) variances in the data the military departments collect and provide to OSD and (2) OSD’s calculation and presentation of the data. DOD is statutorily required to provide reports to Congress that include, among other things, information about military housing privatization projects’ financial health and performance and the backlog, if any, of maintenance and repairs. 39 These reports have also included information on resident satisfaction based on the results of the annual military department satisfaction surveys. In May 2019, DOD issued its report for fiscal year 2017, which stated that overall resident 38GAO-14-704G. 39See, 10 U.S.C. § 2884(c). Page 29 GAO-20-281 Military Housing satisfaction for calendar year 2017 was 87 percent. 40 However, this number is misleading due to issues associated with the collection and calculation of the data DOD used. 41 The military departments provide data on resident satisfaction to OSD for inclusion in DOD’s submission to Congress based on information from the annual resident satisfaction surveys. Specifically, OSD’s instructions to the military departments for the fiscal year 2017 report required the military departments to report the following: • The month and year of the most recently completed tenant satisfaction survey. • The number of residents surveyed and the total number of tenants who completed the survey during the reporting period. • Resident responses to the question that asks: “Would you recommend privatized housing?” Results should indicate how many tenants responded “Yes,” “No,” or “Don’t Know.” However, instead of asking whether residents would recommend privatized housing, the military departments’ annual resident satisfaction survey asked residents the following: “How much do you agree or disagree with the following statement, ‘I would recommend this community to others.’” The difference in the wording between the question asked of residents and the question reported to Congress is notable, as a resident’s satisfaction with his or her community and inclination to recommend it to 40U.S. Department of Defense, Office of the Assistant Secretary of Defense for Sustainment, Military Housing Privatization Initiative Program Evaluation Report Reporting Period: October 1, 2016–September 30, 2017 (May 2019). While the reporting period for this report was fiscal year 2017, with respect to reporting data on resident satisfaction, DOD reported information for calendar year 2017. 41We also reviewed DOD’s reports for fiscal years 2015 and 2016, which did not report on overall resident satisfaction, but did report on satisfaction by home type. Therefore, we did not assess DOD’s collection and calculation of data for these years. U.S. Department of Defense, Office of the Assistant Secretary of Defense for Sustainment, Military Housing Privatization Initiative Program Evaluation Report for Reporting Periods: October 1, 2014– September 30, 2015, and October 1, 2015–September 30, 2016 (Aug. 13, 2018). Page 30 GAO-20-281 Military Housing others may not be reflective of satisfaction with either the privatized housing unit or privatized housing in general. 42 We also found differences in how the military departments interpreted responses to the question they asked. When asked whether they would recommend their community to others, residents were provided the following response categories on a scale of five to zero: (5) strongly agree, (4) agree, (3) neither agree nor disagree, (2) disagree, (1) strongly disagree, and (0) not applicable, no opinion, don’t know, or no answer. However, we found that the ways in which the military departments translated these responses into the “yes,” “no,” or “do not know” categories differed across the military departments, and in the case of the Army differed from year to year. Specifically: • For the fiscal years 2015 through 2017 reports, Navy officials told us they counted responses reported in categories 5 (strongly agree) and 4 (agree) as “yes,” responses in categories 2 (disagree) and 1 (strongly disagree) as “no,” and responses in categories 0 (not applicable, no opinion, don’t know, or no answer) and 3 (neither agree nor disagree) as “don’t know.” • For the same time period, Air Force officials told us they counted responses in categories 5 (strongly agree), 4 (agree), and 3—neither agree nor disagree—as “yes,” responses in categories 2 (disagree) and 1 (strongly disagree) as “no,” and responses in category 0 (not applicable, no opinion, don’t know, or no answer) as “don’t know.” If 3 had not been counted as “yes,” the reported resident satisfaction rate would have been lower. For example, for Lackland Air Force Base, Texas, if officials had not counted responses in category 3 as “yes,” the resident satisfaction rate for newly constructed units would have been more than 20 percent lower than what was reported. The Army calculated responses differently for fiscal years 2015, 2016, and 2017. Specifically: 42A resident might recommend his or her community to others for factors other than the condition of his or her privatized housing unit such as proximity to base services, commute time, access to DOD schools, neighborhood amenities, childcare, safety, community support, affordability, and services provided by DOD’s Exceptional Family Member Program. In 2019, according to the company that developed the annual satisfaction survey, the military departments’ annual satisfaction survey added a question that asks residents how they would rate the satisfaction with the condition of their homes. Page 31 GAO-20-281 Military Housing • For the fiscal year 2017 report, the Army counted responses in categories 5 (strongly agree) and 4 (agree) as “yes,” responses in categories 2 (disagree) and 1 (strongly disagree) as “no,” and responses in categories 0 (not applicable, no opinion, don’t know, or no answer) and 3 (neither agree nor disagree) as “don’t know.” • For the fiscal year 2016 report, the Army counted responses in categories 5 (strongly agree) and 4 (agree) as “yes,” responses in categories 2 (disagree), 1 (strongly disagree), and 0 (not applicable, no opinion, don’t know, or no answer) as “no,” and responses in category 3 (neither agree nor disagree) as “don’t know.” • For the fiscal year 2015 report, the Army counted responses in categories 5 (strongly agree), 4 (agree), and 3 (neither agree nor disagree) as “yes,” responses in categories 2 (disagree) and 1 (strongly disagree) as “no,” and responses in category 0 (not applicable, no opinion, don’t know, or no answer) as “don’t know.” In addition, we identified errors and inaccuracies in how OSD calculates these data and reports them to Congress. Specifically, we found missing data points and incorrect formulas, among other errors, in OSD’s calculation of the data submitted by the military departments. For example: • The formula used by OSD to calculate overall resident satisfaction for fiscal year 2017 did not include data for several projects, including four Army projects—Fort Bragg, North Carolina; Fort Knox, Kentucky; Joint Base Lewis-McChord, Washington; and Presidio of Monterey/Naval Postgraduate School, California. As of September 30, 2017, these four projects accounted for over 18 percent of the Army’s total housing inventory. • The formula used by OSD to calculate resident satisfaction by project double counted resident satisfaction data for new and unrenovated homes for Vandenberg Air Force Base, California, by incorrectly using the Vandenberg Air Force Base data for both the Vandenberg and for the Fort Huachuca/Yuma Proving Ground project. As a result, incorrect data were reported for the Fort Huachuca/Yuma Proving Ground project for some categories of homes. • OSD did not include resident satisfaction data for New Orleans Naval Complex, Louisiana, in its fiscal year 2017 report to Congress, even though the Navy had included data for this project when submitting its data to OSD. OSD also reported identical resident satisfaction data for Wright-Patterson Air Force Base, Ohio, in fiscal years 2015, 2016, and 2017, despite the fact that Air Force officials noted in their Page 32 GAO-20-281 Military Housing submissions to OSD that the annual resident satisfaction data was from the annual resident satisfaction survey for Wright-Patterson Air Force Base conducted December 2013. • Further, Army data provided to OSD had calculation errors that OSD did not reconcile. Specifically, the Army provided OSD the total number of surveys received for a project, as well as the number of surveys broken out by different housing categories. However, we found instances where the sum of the data broken out by different housing categories was not equal to the reported total number of surveys received. For example, when we reviewed data for Fort Rucker, Alabama, the calculated sum of surveys broken out by different housing categories was 1,372, but the Army reported a total of 530 surveys received, a difference of 842 surveys. Further, the presentation of data in OSD’s report to Congress is misleading because OSD did not explain the methodology it used to calculate the overall resident satisfaction percentage or include caveats to explain limitations to the data. Specifically, OSD did not include information on overall response rates to the annual satisfaction survey for each military department, nor did it include response rates by project. Low response rates can create the potential for bias in survey results. For example, in its fiscal year 2017 report, OSD reported that 25 percent of residents living in renovated homes at the MHPI project including Fort Detrick, Maryland/Walter Reed Army Medical Center, Washington, D.C., were satisfied with their housing. However, only four residents provided responses to this question, meaning that just one resident reported being satisfied. In addition, we found that OSD did not include an explanation in the report for why five projects were listed as not applicable. 43 According to OSD officials, this error was a quality control issue that they plan to address, but the officials told us there are no plans for quality control in development at this time. The Fiscal Year 2020 NDAA includes a provision requiring each military installation to use the same satisfaction survey for tenants of military housing—including privatized military housing—the results of which are not to be shared with private partners until reviewed by DOD. 44 The statute also states that DOD’s reports to 43In DOD’s fiscal year 2017 report to Congress, resident satisfaction data were listed as not applicable for the following projects: Elmendorf Air Force Base I, Alaska; New Orleans Naval Complex, Louisiana (NOLA); Southeast Regional, Navy; Joint Base ElmendorfRichardson, Alaska; and Elmendorf Air Force Base II, Alaska. 44Pub. L. No. 116-92, § 3058 (Dec. 20, 2019). Page 33 GAO-20-281 Military Housing Congress shall include additional information, such as the results of residence surveys, as well as assessments of maintenance response times, completion of maintenance requests, the dispute resolution process, overall customer service for tenants, and other factors related to the condition of privatized housing. 45 OSD’s report to Congress states that, given DOD’s objective of improving the quality of life for its servicemembers, the degree of satisfaction military families experience in privatized housing is a critical indicator of overall program success and the military departments and private partners use tenant surveys to help assess the quality of privatized housing. 46 Additionally, Standards for Internal Control in the Federal Government state that management should obtain relevant data from reliable internal and external sources in a timely manner based on identified information requirements. Relevant data have a logical connection with, or bearing upon, the identified information requirements. Reliable internal and external sources provide data that are reasonably free from error and bias and faithfully represent what they purport to represent. Management should evaluate both internal and external sources of data for reliability, and obtain data on a timely basis so they can be used for effective monitoring. 47 However, the errors we identified in OSD’s data calculations, as well as the differences in how the military departments translate data provided to OSD, indicate the need for better internal controls, including a process for collecting and calculating resident satisfaction data from the military departments, and explanation of the data collected and reported on resident satisfaction to ensure they are reasonably free from error and bias and represent what they purport to represent. According to an OSD official responsible for preparing the reports to Congress, her office inherited the MHPI report process from its predecessors and had to quickly catch up on reports because DOD was behind on its reporting requirement. However, she noted her office is working with the military departments to review the resident satisfaction survey questions and will be identifying and implementing measures to ensure an accurate and reliable process to compile, calculate, report, and compare MHPI 45Pub. L. No. 116-92, § 3016(d). 46U.S. Department of Defense, Office of the Assistant Secretary of Defense for Sustainment, Military Housing Privatization Initiative Program Evaluation Report Reporting Period: October 1, 2016–September 30, 2017 (May 2019). 47GAO-14-704G. Page 34 GAO-20-281 Military Housing residents’ satisfaction by military department and across DOD. Additionally, for future survey data reporting, OSD officials told us they plan to research the possibility of directly collecting resident survey data from the survey administrator to minimize data transcription errors. Until OSD makes these changes to the data collection and calculation efforts that make up the department’s report to Congress and provides explanations of the data in the reports, OSD will not be able to provide Congress with an accurate picture of resident satisfaction with privatized housing. Military Housing Offices Have Not Effectively Communicated Their Role as a Resource for Servicemembers Experiencing Challenges with Privatized Housing Military housing offices located at each installation are available to provide resources to servicemembers experiencing challenges with their privatized housing, among other services, but these offices have not always clearly and systematically communicated this role to residents of privatized housing. Military housing office officials noted that servicemembers living in privatized military housing primarily interact with their installation’s military housing office when they first receive orders to move to an installation. The military housing office provides new residents with information on their local housing options, to include referral services for housing options. However, military department guidance calls for the military housing office to provide continued assistance to servicemembers and their families living in privatized housing. For example, each military department has guidance that establishes the role of its housing offices in the resident dispute resolution process—specifically, if servicemembers are experiencing a dispute with the private partner: • Army policy states that each installation should have an official tasked with providing support to servicemembers regarding resident issues that cannot be resolved by the private property manager. 48 This individual is also in charge of resolving every resident complaint, and the military housing office, if required, can request mediation by the garrison commander. • Air Force policy directs installation commanders to establish regular meetings with the private partners to discuss resident disputes and develop resolutions for residents’ issues. 49 Also, the Air Force business agreements for each project are to establish Management 48Department 11, 2014). of the Army, Portfolio and Asset Management Handbook (Version 5.0, Sept. 49Air Force Instruction 32-6007, Privatized Housing Management (Sept. 19, 2012)(incorporating Air Force Guidance Memorandum 2019-01, July 25, 2019). Page 35 GAO-20-281 Military Housing Review Committees, in which the private project owner, Air Force housing office officials, and the Air Force Civil Engineer Center meet quarterly to review and facilitate the resolution of prevalent issues. • The Navy announced a standardized two-step resolution process in May 2019 for housing residents who have issues or concerns with their current homes. 50 The first step is to report any issue to the local property manager. If the issue is not resolved in either a timely manner or to quality standards, residents are asked to contact their local Navy housing service center, which directly reports to the installation commanding officer, or the servicemember’s chain of command. Prior to the standardization of this process, Navy guidance established a general responsibility to assist residents in the dispute resolution process and each project’s tenant lease includes specific dispute resolution processes. 51 • The Marine Corps has established a three-step dispute resolution process for residents to follow when they are experiencing a dispute with the private partner. Further, Marine Corps policy calls for each of the private partners to establish standard operating procedures that should include complaint resolution procedures. 52 Despite established military department guidance, we found that residents were sometimes confused and lacked awareness of the availability of the military housing office to assist them with issues they were experiencing with privatized housing. For example, residents who participated in our focus groups and responded to our online tool expressed the following concerns: • At least one resident in each of our focus groups noted being sometimes confused about the military housing office’s roles and responsibilities with regard to the maintenance of their home. These residents indicated they did not know the military housing office existed or could serve as a resource. Further, some individuals that responded to our online tool indicated that they did not know they 50Commander, Navy Installations Command Public Affairs, Press Release NNS190514-08 (May 14, 2019) https://www.navy.mil/submit/display.asp?story_id=109591. 51Commander, Navy Installations Command, Instruction 11103.4A, Responsibility for Housing Programs in the Navy (Jan. 31, 2014). 52Marine Corps Order 11000.22, Marine Corps Bachelor and Family Housing Management (July 14, 2014)(incorporating Change 1, Jan. 22, 2018). Page 36 GAO-20-281 Military Housing could reach out to military housing office officials or their chain of command with issues related to the condition of their home. • Residents in at least three of our focus groups indicated they perceived that the military housing office was not working independently of the partner or in the residents’ best interest. For example, residents in at least three focus groups noted that they viewed the military housing office as an extension of the private partner. Other residents noted that they did not know what the military housing office was or what role the office plays in managing privatized housing. In addition, residents we solicited information from through our online tool indicated that they felt they have not had any recourse in resolving issues and disagreements with private partners. For example, one individual who responded to our online tool stated that she was glad she moved off post because she now has legal recourse if the landlord does not meet maintenance requirements. The military department oversight agencies have found that the military departments have not clearly and systematically communicated their roles to residents, and resident confusion and a lack of awareness regarding the role of the military housing offices is an issue. In April 2019, the Air Force Inspector General reported that less than half of the residents interviewed used their military housing office to resolve complaints, and at some installations officials visited many residents did not know the military housing office had an oversight role. 53 Similarly, in May 2019, the Army Inspector General reported to the Secretary of the Army that at 82 percent of Army installations with privatized housing, residents did not know how to escalate issues with either the private partner or the Army housing office. 54 Additionally, the Army Inspector General reported that installation command teams and staff cited multiple circumstances where military housing offices and tenant advocacy roles and responsibilities were unclear. Further, military housing office officials with whom we spoke during our site visits acknowledged the gap in resident awareness regarding the existence and purpose of the military housing office. Officials also noted that at times residents were unaware of the difference between the military housing office and the private partner office due, in part, to their 53U.S Air Force, Office of Inspector General, Directed Inspection Report: Military Privatized Housing Health and Safety Complaint Resolution, (Apr. 9, 2019)(U//FOUO). 54The 82 percent is based on 1,180 surveys collected by the Department of the Army Inspector General. Surveys were provided to 1,493 residents at 49 Army locations. Page 37 GAO-20-281 Military Housing physical co-location and unclear building signage. For example, a military housing office official at Fort Bragg, North Carolina, told us the military housing office was the best kept secret on the installation. Moreover, residents that participated in our four focus groups at Fort Bragg expressed confusion in differentiating the Army military housing office officials from private partner representatives. Similarly, officials at the military housing office at Tinker Air Force Base, Oklahoma, told us that many residents were confused by their office’s role because the private partner office goes by the name “Tinker Housing Office.” Further, we observed that both private partner representatives and some military housing office officials are located in the same building, and signage does not distinctly indicate that the office houses both military officials and private partner representatives. In contrast, the military housing office at Camp Pendleton, California, is intentionally branded as the “Camp Pendleton Joint Housing Office” and signage indicates the office houses officials from both the Marine Corps and the installation’s private partners. See figure 5 for examples of the varying level of detail in military housing office signage. Figure 5: Examples of Varying Levels of Detail in Military Housing Office Signage Some military housing office officials told us they have taken steps to improve resident awareness, such as increasing advertising of the military housing office’s role and contact information, using town hall meetings to inform residents of their roles and responsibilities, and rebranding their military housing offices to differentiate them from the private partners. For example, the Army housing office at Fort Sill, Oklahoma, changed its Page 38 GAO-20-281 Military Housing name from the “Residential Communities Initiative Housing Office” to the “Garrison Housing Office” to more clearly denote that the military housing office is not associated with the private partner. In addition, a Marine Corps housing office official provided us with a flyer, which is distributed to residents by the private partner, informing residents of housing office contact information and the service’s three-step dispute resolution process. See figure 6 for a copy of the flyer. Figure 6: Flyer Advertising Marine Corps’ Three-Step Dispute Resolution Process According to DOD officials, the military departments generally decreased their staffing and oversight of daily privatized housing operations after the Page 39 GAO-20-281 Military Housing MHPI was enacted, which led to less ongoing resident interaction. For example, Army officials we spoke with in January 2019 told us they typically filled 80 percent of available military housing office positions across their installations. Additionally, officials stated that housing offices were generally staffed with two or three officials responsible for assisting servicemembers with housing needs both on the installation as well as in the local community. Further, the officials told us that the team at Fort Bragg, North Carolina, was decreased from about 15 to 3 positions. According to OSD officials, while housing offices should generally not require the number of personnel that were necessary prior to privatization, reductions following sequestration reduced housing staff below the level necessary to fully perform required privatized housing oversight as it was originally envisioned at the outset of the program. 55 OSD has also recognized that the military departments’ communication with residents about their role as a resource for residents has not been clear or systematic. In February 2019, the Assistant Secretary of Defense for Sustainment testified before Congress that a way forward in addressing resident concerns would require focus in three key areas: communication, engagement, and responsiveness. In support of this, OSD and the military departments are collaborating with each of the private partners on several initiatives aimed at improving the residents’ experience with privatized housing and ensuring a consistent resident experience across installations. These initiatives include: 56 • establishing a tenant bill of rights that will clearly define tenants’ rights and responsibilities; • establishing a resident advocate position that is planned to provide advice, education, and support to the resident and advocate on the resident’s behalf in disputes with private partners; 55In fiscal year 2013, the absence of legislation to reduce the federal budget deficit by at least $1.2 trillion triggered the sequestration process in section 251A of the Balanced Budget and Emergency Deficit Control Act of 1985, as amended. On March 1, 2013, the President ordered sequestration of budgetary resources across non-exempt federal government accounts, which resulted in reductions to DOD’s resources of about $37 billion in discretionary appropriations and about $37.4 million in direct spending. Spending reductions under sequestration affected DOD’s civilian workforce and many programs and functions, and required DOD to take actions including furloughing most civilian employees for 6 days. GAO, Sequestration: Observations on the Department of Defense’s Approach in Fiscal Year 2013, GAO-14-177R (Washington, D.C.: Nov. 7, 2013). 56As of January 2020, DOD was working to incorporate related statutory provisions in the Fiscal Year 2020 NDAA that impact these efforts. We provide more detail on the status of these efforts in Table 2. Page 40 GAO-20-281 Military Housing • developing a common lease that provides a common framework and language in residential leases across all privatization projects; and • developing a standardized formal dispute resolution process to ensure the prompt and fair resolution of disputes that arise between privatized housing landlords and residents. Despite the development of initiatives aimed at improving the resident’s experience with privatized housing and various ad hoc efforts to better brand and advertise the roles and responsibilities of some military housing offices, the military departments have not systematically or clearly communicated these efforts to residents, and military officials we met with acknowledged that there still appears to be a gap in residents’ awareness of the military housing office and its role in the dispute resolution process. Standards for Internal Control in the Federal Government state that management should externally communicate the necessary quality information to achieve the entity’s objectives. Management communicates this externally through reporting lines so that external parties can help the entity achieve its objectives and address related risks. 57 Moving forward, having plans in place to clearly and systematically communicate the difference between the military housing office and the private partners— including the military departments’ roles, responsibilities, and military housing office locations and contact information—will better position the military departments to achieve the intended objectives of the initiatives they are currently developing with OSD. 57GAO-14-704G. Page 41 GAO-20-281 Military Housing DOD and Private Partners Are Implementing Initiatives to Improve Privatized Housing, but May Face Challenges DOD and Private Partners Are Implementing Initiatives to Improve MHPI OSD, the military departments, and the private partners have identified and begun collaborating on a series of initiatives aimed at improving residents’ experience with privatized housing. According to an OSD official, a series of initiatives have been identified and are currently in various phases of development and implementation. Tri-service working groups, each chaired by a designated military department and including officials and legal counsel from each military department as well as private partner representatives, are leading efforts to develop and implement the initiatives. In addition, in the Fiscal Year 2020 NDAA, Congress established several requirements aimed at addressing military privatization housing reform. 58 Several of the statutory requirements provide specific provisions that DOD will need to incorporate into its development and implementation of existing MHPI initiatives, as well as additional requirements aimed at improving the oversight of privatized housing. Table 2 outlines key initiatives aimed at improving privatized housing, as well as additional selected requirements mandated by the Fiscal Year 2020 NDAA. 59 58Public Law No. 116-92 (Dec. 20, 2019). 59In addition to the provisions discussed in this report, the Fiscal Year 2020 NDAA also included provisions related to other elements of MHPI such as the reporting of medical issues, use of nondisclosure agreements, and others. Page 42 GAO-20-281 Military Housing Table 2: Key Department of Defense (DOD) Initiatives Aimed at Improving Privatized Housing and Selected Related National Defense Authorization Act (NDAA) for Fiscal Year 2020 Requirements DOD initiatives Selected related Fiscal Year 2020 NDAA requirement Develop a resident bill of rights and resident responsibility document (companion document to bill of rights). The resident bill of rights is to provide clarity to residents on their rights and responsibilities while living in privatized military housing. • In February 2020, the Secretary of Defense signed the resident bill of rights and issued the resident responsibility document. The bill of rights states that 15 of the 18 residents’ rights mandated by the fiscal year 2020 NDAA will be provided to residents by May 1, 2020. • The bill omitted three rights—access to maintenance history, process for dispute resolution, and withholding of rent until disputes are resolved. On March 3, 2020, the Acting Assistant Secretary of Defense (Sustainment) testified before Congress that DOD will continue to work with the Military Housing Privatization Initiative (MHPI) companies and, as necessary, Congress to ensure the benefits of these three rights are fully available as quickly as possible. The resident bill of rights, among other things, must ensure residents have the right to: • have their basic allowance housing payments segregated and held in escrow, with approval of a designated commander, and not used by the property owner, property manager, or landlord pending completion of the dispute resolution process (Pub. L. No. 116-92, § 3011(b)); • be provided with a maintenance history of the prospective housing unit before signing a lease (Pub. L. No. 116-92, § 3011(b)); and • receive a plain-language briefing, before signing a lease and 30 days after move-in, from the installation housing office on all rights and responsibilities associated with tenancy of the housing unit, including information regarding the existence of any additional fees authorized by the lease, any utilities payments, the procedures for submitting and tracking work orders, the identity of the military tenant advocate, and the dispute resolution process (Pub. L. No. 116-92, § 3011(b)). The statute also lists several resident responsibilities such as timely reporting of issues with their home and maintaining standard upkeep of their home, among others (Pub. L. No. 11692, § 3011(b)). Establish a common lease framework for all MHPI Projects (i.e., tenant leases). The goal of the common lease is to provide residents of privatized housing with similar terms in their leases, regardless of where they are living and which private partner owns their housing unit. The resident bill of rights shall also include the right to expect common documents, forms, and processes for housing units will be the same for all installations of the Department, to the maximum extent applicable without violating local, State, and Federal regulations (Pub. L. No. 116-92, § 3011(b)). The statute also states a landlord providing a housing unit shall agree to have any agreements or forms to be used by the landlord approved by the Assistant Secretary of Defense for Sustainment, including a common lease agreement (Pub. L. No. 116-92, § 3014). Implement a common (enterprise) dispute adjudication process and resident advocate position. The military departments and private partners are collaborating on a common dispute resolution process that would apply to all projects. The resident advocate position, according to an Office of the Secretary of Defense (OSD) official, will be available to provide independent advice, education, and support to residents. However, as of January 2020, according to an OSD official the military departments have not yet determined whether this individual would be active duty or civilian and where the position would fall organizationally—specifically, whether it would be part of the military housing offices. The Secretary concerned shall implement a standardized formal dispute resolution process to ensure the prompt and fair resolution of disputes that arise between landlords providing housing units and tenants residing in housing units concerning maintenance and repairs, damage claims, rental payments, move-out charges, and such other issues relating to housing units as the Secretary determines appropriate (Pub. L. No. 116-92, § 3022). Congress required that each military department Secretary designate the installation or regional commander in charge of oversight of housing as the deciding authority under the dispute resolution process. The report also establishes process elements and timelines (Pub. L. No. 116-92, § 3022). Page 43 GAO-20-281 Military Housing DOD initiatives Selected related Fiscal Year 2020 NDAA requirement Review MHPI resident satisfaction metrics and data collection process. According to OSD officials, they are reviewing the process by which they measure and report on resident satisfaction data. DOD’s reports to Congress must include additional information such as the results of residence surveys, as well as assessments of maintenance response times, completion of maintenance requests, the dispute resolution process, overall customer service for tenants, and other factors related to the condition of privatized housing (Pub. L. No. 116-92, § 3016(d)). Standardize performance incentive fee ranges. In October 2019, OSD issued new guidance standardizing the performance incentive fee ranges across the military departments. DOD shall: • Report, at least annually, on a publicly accessible website, information regarding the use of performance incentive fees to include the applicable incentive fees, the metrics used to determine the incentive fees, whether incentive fees were paid in full or were withheld in part or in full, and if any incentive fees were withheld, the reasons for such withholding (Pub. L. No. 116-92, § 3016(a)). • Take into consideration any decision a commander renders in favor of the tenant in the formal dispute resolution process in determining whether to pay or withhold all or part of any incentive fees for which a private partner may otherwise be eligible for under the contract (Pub. L. No. 116-92, § 3013(a)). Revitalize housing workforce (hiring of additional staff) and establish customer care agents. According to military department officials, each military department is taking steps to hire and train additional staff. DOD must, by June 20, 2020, report to Congress on civilian personnel shortages for appropriate oversight of the management of privatized military housing. The report is to include the number of additional personnel who are required, the installation and headquarter locations at which they will be employed, the employment positions they will fill, and the duties they will perform, including a breakdown of duty requirements by function, such as oversight, home inspectors, and maintenance (Pub. L. No. 116-92, § 3041). Make resident work order status and progress visible and transparent to staff and residents. According to private partner representatives, all military department housing officials have been provided access to work order systems. Additionally, all partners have implemented the use of mobile applications that allow residents submit and track to work order status. Private partners must: • Maintain an electronic work order system that enables access by the tenant to view work order history, status, and other relevant information (Pub. L. No. 116-92, § 3022). • Allow real-time access to work order systems by military housing officials, at the installation, major subordinate command, and service-wide levels; and allow the work order or maintenance tickets to be closed only once the tenant and the head of the housing management office of the installation sign off (Pub. L. No. 116-92, § 3022). Source: GAO review of DOD information and Fiscal Year 2020 NDAA provisions. GAO-20-281 In addition to the provisions noted in table 2, the Fiscal Year 2020 NDAA included requirements for increased oversight of the physical condition of privatized housing. Specifically, the legislation required the following: Page 44 GAO-20-281 Military Housing Initiatives to Improve MHPI May Face Implementation Challenges • The Secretary of Defense is to designate a Chief Housing Officer to oversee housing units, including the creation and standardization of policies and processes regarding housing units. 60 • The Secretary of Defense is required to establish a uniform code of basic standards for privatized military housing and plans to conduct inspections and assessment of the condition of privatized homes. 61 • The military departments are required to create a council on privatized military housing for the purposes of maintaining adequate oversight of the military housing program and serving as a mechanism to identify and resolve problems regarding privatized military housing. 62 • The head of the installation military housing office is responsible for conducting a physical inspection and approving the habitability of a vacant housing unit for the installation before the landlord managing the housing unit is authorized to offer the housing unit available for occupancy; conducting a physical inspection of the housing unit upon tenant move-out; and initiating contact with a tenant regarding the tenant’s satisfaction with his or her housing unit not later than 15 days after move-in, and again 60 days after move-in. 63 • Each installation is required to use the same satisfaction survey for tenants of military housing, including privatized military housing, and results are not to be shared with partners until reviewed by DOD. 64 DOD and private partner representatives have cited several challenges that could affect their ability to implement initiatives aimed at improving MHPI. Specifically, key challenges include the timeliness with which they are able to implement initiatives, a lack of resources needed for implementation, and concerns that implementation could have unintended negative impacts on the financial viability of the privatized housing projects. 60Pub. L. No. 116-92, § 3012(a). 61Pub. L. No. 116-92, § 3051. 62Pub. L. No. 116-92, § 3042. 63Pub. L. No. 116-92, § 3014(a). 64Pub. L. No. 116-92, § 3058. Page 45 GAO-20-281 Military Housing • Timeliness of implementation due to the need to collaborate with and obtain input and agreement from the large number of stakeholders involved in privatized housing. According to DOD officials and private partner representatives, many of the initiatives designed to improve privatized housing require not only agreement between DOD and the private housing partners, but also discussion with and, in some cases, approval by the project bond holders. 65 Because DOD does not have the ability to unilaterally make changes to existing business agreements, this need for stakeholder agreement limits DOD’s control over the implementation timeline of any initiative that requires changes to a project’s business agreement—such as the implementation of a standardized dispute resolution process. Additionally, the private partners noted that the bond holders may be reluctant to agree to changes to the business agreements that could result in higher project costs. • The need for more military department staff with targeted expertise. As noted earlier, the military departments had reduced their involvement in daily privatized military housing operations as part of the overall privatization effort. This included reducing staffing levels at the installations, and military housing office officials at over half of the installations we visited stated that reduced staffing levels had impacted their ability to carry out oversight duties, such as work order data analysis and housing inspections. Further, until recent issues surfaced over concerns of the quality of privatized housing, the military departments had distanced themselves from involvement in daily military housing operations. For example, the Army issued a memorandum in 2013, which has since been rescinded, stating that garrison commanders were not to authorize, direct, or permit Army representatives to initiate health and welfare inspections of privatized housing. 66 Each of the military departments has plans to increase the military housing office staffing at each installation to allow for enhanced oversight. In particular, according to military department officials, these positions will focus on quality control and quality assurance of the maintenance of privatized homes. However, improvements to the condition of privatized housing may not be fully realized until DOD establishes a uniform code of basic standards for 65Privatized housing projects were generally financed through both private-sector financing and funds provided by the military departments. Projects obtained private-sector financing by obtaining bank loans and by issuing bonds. 66Department of the Army Memorandum, Residential Communities Initiative (RCI) Policy for Major Decisions Authority–Policy Memorandum #1 (Feb. 5, 2013). Page 46 GAO-20-281 Military Housing privatized military housing, as required by the Fiscal Year 2020 NDAA, and these new personnel are trained in these standards. • The potential for unintended negative financial impacts on the projects that could outweigh the intended benefits of the initiatives. OSD officials and private partner representatives have expressed concern that some proposed initiatives could result in unintended financial consequences for the housing projects. In particular, private partner representatives noted costs from things such as legal fees associated with the development of a common lease and the various addendums that would be required and the unanticipated costs of hiring outside third party inspections. In particular, some of the private partners noted that the financial impact of unfunded requirements to projects that are already experiencing financial distress could result in even fewer funds available to reinvest in improvements to the current and future physical condition of the homes. Moreover, OSD officials told us they have concerns that some initiatives—such as increased frequency of change-of-occupancy inspections that may result in homes remaining vacant longer than planned and therefore not collecting rent—may unintentionally impact a project’s cash flow. Officials noted that some installations have large-scale housing turn over at the same time and inspections may not be able to be completed in the required time frames. For example, OSD officials said that at Fort Leavenworth, Kansas, the vast majority of homes generally turn over during a 2-week time period. Officials said that in a location like this, new oversight requirements may have a negative impact on residents’ move-in timelines, which could subsequently impact occupancy rates and project cash flow as a result of delays in rent payments. OSD officials also stated that residents’ ability to have their basic allowance housing payments segregated and held in escrow may present financial challenges to both the resident and the project. These officials noted that they did not yet know how the withholding of these payments would be implemented. According to OSD officials, as of January 2020, there are many questions surrounding the implementation of the Fiscal Year 2020 NDAA provisions. Officials told us that they have not yet assessed the impact of increased oversight on the financial viability of the MHPI projects, but stated that as they develop processes to implement each new statutory provision, the financial impact is something that needs to be considered. DOD’s Military Housing Privatization Initiative Performance Evaluation Report for fiscal year 2017 stated that the government’s interests are not Page 47 GAO-20-281 Military Housing always aligned with those of the private sector, and that oversight and engagement are required and expected in a public-private partnership over the long term to ensure success. 67 We have previously reported that the military departments have not defined their risk tolerance levels for privatized housing relative to the program’s objective of providing quality housing that reflects community living. 68 Specifically, we recognized that the Office of Management and Budget guidance on the preparation, submission, and execution of the federal budget suggests that publicprivate partnerships, such as privatized military housing projects, contain some elements of risk to the government. 69 Standards for Internal Control in the Federal Government state that management should identify, analyze, and respond to risks related to achieving defined program objectives. While DOD is in the process of developing and implementing initiatives to improve privatized military housing, OSD and the military departments have not assessed the risk of the proposed initiatives on the financial viability of the privatized housing projects. According to an OSD official, the intention of privatization was to reduce the government’s role in the management of military housing and put more responsibility on the private partners. As described earlier in this report, the military departments have ramped up their oversight efforts in response to recent concerns about the condition of privatized housing by, for example, revising guidance and hiring additional staff. However, OSD has not assessed the impact of these activities on the financial viability of the MHPI projects. For example, OSD has not determined how increasing the frequency of housing office inspections and residents’ withholding of rent could impact the bottom line of some of its privatized projects. Without assessing risks to the financial viability of the MHPI projects associated with the implementation of these initiatives aimed at improving privatized housing, DOD’s efforts to improve the privatized housing program could be compromised. Further, DOD has a long-term interest in ensuring the financial health of the properties privatized under MHPI. As we have reported, typically the titles to the homes that were conveyed to 67U.S. Department of Defense, Office of the Assistant Secretary of Defense for Sustainment, Military Housing Privatization Initiative Program Evaluation Report Reporting Period: October 1, 2016–September 30, 2017 (May 2019). 68GAO, Military Housing Privatization: DOD Should Take Steps to Improve Monitoring, Reporting, and Risk Assessment, GAO18-218 (Washington, D.C.: Mar. 13, 2018). 69Office of Management and Budget Circular No. A-11, Preparation, Submission, and Execution of the Budget (2019). Page 48 GAO-20-281 Military Housing the private partners and any improvements made to these homes during the duration of the ground leases will automatically revert to the military departments upon expiration or termination of the leases. 70 Conclusions DOD’s oversight of privatized housing is critical to ensure that residents are being provided with affordable, quality housing that generally reflects contemporary community living standards. In light of recent concerns about the effect of inadequate and poor quality housing on servicemembers and their families, the military departments have recently implemented steps to increase the oversight of the condition of privatized housing. However, OSD has not provided the military departments with specific objectives for this monitoring. The newly established Chief Housing Officer position, intended to standardize guidance and processes for the oversight of privatized housing, provides DOD with an opportunity to ensure that revised guidance provided to the military departments includes objectives for increased oversight. In addition to oversight of the condition of homes, DOD has taken initial steps to standardize performance incentive metrics across the military departments. However, unless efforts are made to ensure that the indicators driving these metrics accurately reflect private partners’ performance in maintaining the condition and quality of privatized homes, DOD’s ability to hold private partners accountable will remain limited. Further, while the military departments continue to increase the access to and use of work order data to monitor and track the condition of privatized housing, without consistent terminology and practices for work order data collection and processes for validating data collected from the private housing partners, the use of these data may not result in reliable findings. Finally, DOD has frequently reported high customer resident satisfaction rates as a key indicator of the success of the privatization initiative. However, the process used to collect and calculate the data used for determining these rates and limitations in its presentation to Congress raise questions about the reliability of DOD’s reports and their usefulness as an indicator of program success. By improving oversight guidance, mechanisms for measuring private partner performance, the reliability of housing data, and reporting on resident satisfaction, DOD can better ensure that MHPI is providing servicemembers with quality housing. Despite a decreased role in the daily management of privatized housing, the military departments still maintain responsibility for providing 70GAO, Military Housing Privatization: DOD Faces New Challenges Due to Significant Growth at Some Installations and Recent Turmoil in the Financial Markets, GAO-09-352 (Washington, D.C.: May 15, 2009). Page 49 GAO-20-281 Military Housing servicemembers with resources for seeking resolution to any issues identified in their privatized homes. However, without plans in place to communicate military housing office roles, responsibilities, and locations to residents of privatized housing, these individuals may not receive the full benefits of the assistance that the military housing offices provide. In light of the increased focus by DOD and Congress in ensuring that residents are aware of their rights and responsibilities, improved communication with residents about the military housing offices’ roles and responsibilities can help ensure that residents are utilizing the full range of resources available to them if they have issues with privatized housing. As OSD, the military departments, and the private partners take steps to improve the resident experience with privatized military housing and increase the department’s focus on the condition of privatized homes, ensuring that their efforts do not inadvertently harm the financial viability of these projects is key. Without assessing and mitigating the potential risk program improvements may have on the financial viability of the MHPI projects, DOD cannot ensure that these initiatives and the implementation of new statutory requirements will ultimately result in improvements to the condition of privatized housing. Recommendations for Executive Action We are making a total of 12 recommendations—six to the Office of the Secretary of Defense, two to the Secretary of the Army, two to the Secretary of the Air Force, and two to the Secretary of the Navy: The Secretary of Defense should ensure that the Assistant Secretary of Defense for Sustainment, in collaboration with the military departments, provide updated guidance for the oversight of privatized military housing, to include oversight objectives for each service to monitor the physical condition of privatized homes over the remaining duration of the ground leases. (Recommendation 1) The Secretary of the Army should take steps, in collaboration with the Army’s private housing partners, to review the indicators underlying the privatized housing project performance metrics to ensure they provide an accurate reflection of the condition and quality of the homes. (Recommendation 2) The Secretary of the Air Force should take steps, in collaboration with the Air Force’s private housing partners, to review the indicators underlying the privatized housing project performance metrics to ensure they provide an accurate reflection of the condition and quality of the homes. (Recommendation 3) Page 50 GAO-20-281 Military Housing The Secretary of the Navy should take steps, in collaboration with the Navy and Marine Corps’ private housing partners, to review the indicators underlying the privatized housing project performance metrics to ensure they provide an accurate reflection of the condition and quality of the homes. (Recommendation 4) The Secretary of Defense should ensure that the Assistant Secretary of Defense for Sustainment, in collaboration with the military departments and private housing partners, establish minimum data requirements and consistent terminology and practices for work order data collection for comparability across installations and projects and to track trends over time. (Recommendation 5) The Secretary of Defense should ensure that the Assistant Secretary of Defense for Sustainment require the military departments to establish a process to validate data collected by the private housing partners to better ensure the reliability and validity of work order data and to allow for more effective use of these data for monitoring and tracking purposes. (Recommendation 6) The Secretary of Defense should ensure the Assistant Secretary of Defense for Sustainment, in collaboration with the military departments, develop a process for collecting and calculating resident satisfaction data from the military departments to ensure that the data are compiled and calculated in a standardized and accurate way. (Recommendation 7) The Secretary of Defense should ensure the Assistant Secretary of Defense for Sustainment provides additional explanation of the data collected and reported in future reports to Congress, such as explaining the limitations of available survey data, how resident satisfaction was calculated, and reasons for any missing data, among other things. (Recommendation 8) The Secretary of the Army should develop and implement a plan to clearly and systematically communicate to residents the difference between the military housing office and the private partner. At a minimum, these plans should include the Army housing office’s roles, responsibilities, locations, and contact information and should ensure that all residents are aware that they can directly contact Army housing office officials. (Recommendation 9) The Secretary of the Air Force should develop and implement a plan to clearly and systematically communicate to residents the difference Page 51 GAO-20-281 Military Housing between the military housing office and the private partner. At a minimum, these plans should include the Air Force housing office’s roles, responsibilities, locations, and contact information and should ensure that all residents are aware that they can directly contact Air Force housing office officials. (Recommendation 10) The Secretary of the Navy should develop and implement a plan to clearly and systematically communicate to residents the difference between the military housing office and the private partner. At a minimum, these plans should include the Navy housing office’s roles, responsibilities, locations, and contact information and should ensure that all residents are aware that they can directly contact Navy housing office officials. (Recommendation 11) The Secretary of Defense should ensure that the Assistant Secretary of Defense for Sustainment, in collaboration with the military departments, assess the risks of proposed initiatives aimed at improving the privatized military housing program on the financial viability of the projects. (Recommendation 12) Agency Comments We provided a draft of this report to DOD for review and comment. In written comments, reprinted in their entirety in appendix III, DOD concurred with 10 of our recommendations and partially concurred with 2, identifying actions it plans to take to address each of them. DOD also provided technical comments, which we incorporated as appropriate. DOD partially concurred with our recommendation that the Assistant Secretary of Defense for Sustainment, in collaboration with the military departments and private housing partners, establish minimum data requirements and consistent terminology and practices for work order collection. The department noted that neither the Assistant Secretary of Defense for Sustainment nor the military departments could mandate changes to existing privatized housing project ground leases or legal agreements. DOD further noted that it cannot unilaterally make changes to the project ground leases and associated legal documents without concurrence from the private partners. However, the department noted that to the maximum extent practical, it would work to establish minimum data requirements and consistent terminology and practices for work order collection. DOD also partially concurred with our recommendation that the Under Secretary of Defense for Sustainment, in collaboration with the military Page 52 GAO-20-281 Military Housing departments, develops a process for collecting and calculating resident satisfaction data because there is no Under Secretary of Defense for Sustainment. Based on the department’s comments, we revised the addressee of this recommendation, directing action to the Assistant Secretary of Defense for Sustainment. However, the department noted that effective with the survey collection effort for Fiscal Year 2021, it would refine the process for collecting and calculating resident satisfaction data from the military departments to ensure that DOD compiles and calculates data in a standardized and accurate way. We are sending copies of this report to the appropriate congressional committees; Senator Catherine Cortez Masto; Senator Mark Warner; Representative Gus Bilirakis; the Secretary of Defense; and the Secretaries of the Departments of the Army, the Navy, and the Air Force. In addition, the report is available at no charge on our website at https://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-2775 or FieldE1@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix IV. Elizabeth A. Field Director, Defense Capabilities and Management Page 53 GAO-20-281 Military Housing List of Addressees The Honorable James M. Inhofe Chairman The Honorable Jack Reed Ranking Member Committee on Armed Services United States Senate The Honorable Richard Shelby Chairman The Honorable Richard Durbin Ranking Member Subcommittee on Defense Committee on Appropriations United States Senate The Honorable John Boozman Chairman The Honorable Brian Schatz Ranking Member Subcommittee on Military Construction, Veterans Affairs, and Related Agencies Committee on Appropriations United States Senate The Honorable Adam Smith Chairman The Honorable Mac Thornberry Ranking Member Committee on Armed Services House of Representatives The Honorable Pete Visclosky Chairman The Honorable Ken Calvert Ranking Member Subcommittee on Defense Committee on Appropriations House of Representatives Page 54 GAO-20-281 Military Housing The Honorable Debbie Wasserman Schultz Chairwoman The Honorable John R. Carter Ranking Member Subcommittee on Military Construction, Veterans Affairs, and Related Agencies Committee on Appropriations House of Representatives The Honorable Catherine Cortez Masto United States Senate The Honorable Mark Warner United States Senate The Honorable Gus Bilirakis House of Representatives Page 55 GAO-20-281 Military Housing Appendix I: Objectives, Scope, and Methodology Appendix I: Objectives, Scope, and Methodology The Conference Report accompanying a bill for the Fiscal Year 2019 Department of Defense Appropriations Act included a provision for us to review ongoing issues within privatized military housing. 1 This report examines the extent to which the Office of the Secretary of Defense (OSD) and the military departments (1) conduct oversight of privatized military housing for servicemembers and their families, (2) have communicated their roles and responsibilities to servicemembers and their families, and (3) have developed and implemented initiatives to improve privatized housing. We included all privatized housing projects in each military department. For each of our objectives, we reviewed OSD and military department policies and guidance for the implementation of the Military Housing Privatization Initiative (MHPI) program, including guidance on the authority, roles, and responsibilities for oversight and management of privatized housing. We evaluated the extent to which the evidence we collected aligned with OSD policy and stated goals for oversight and management of privatized housing, and whether the evidence adhered to the principles in Standards for Internal Control in the Federal Government. 2 We conducted interviews with officials from the Office of the Assistant Secretary of Defense for Sustainment, Office of Facilities Management; the Office of the Deputy Assistant Secretary of the Army (Installations, Housing and Partnerships); the Army Installation Management Command; the Army Assistant Chief of Staff for Installation Management; the Assistant Secretary of the Air Force for Installations, Environment, and Energy; the Air Force Civil Engineer Center; the Commander, Navy Installations Command; the Commander, Naval Facilities Engineering Command; the Marine Corps Installation Command; and representatives from each of the 14 private partners that are currently responsible for privatized housing projects. We visited a non-generalizable sample of 10 installations selected to represent each of the military departments, six private partners—including the five largest who own the majority of privatized military housing—and geographic and climate diversity. The selected sites in our nongeneralizable sample were three Army installations—Fort Bragg, North Carolina; Fort Huachuca, Arizona; and Fort Sill, Oklahoma; two Navy installations—Naval Station Norfolk, Virginia, and Naval Base San Diego, 1H.R. Rep. No. 115-952 (2018) (Conf. Rep.). 2GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D. C.: September 2014). Page 56 GAO-20-281 Military Housing Appendix I: Objectives, Scope, and Methodology California; two Marine Corps installations—Marine Corps Base Camp Lejeune, North Carolina, and Marine Corps Base Camp Pendleton, California; and three Air Force installations—Davis-Monthan Air Force Base, Arizona; Langley Air Force Base, Virginia; and Tinker Air Force Base, Oklahoma. We reviewed the ground leases and other MHPI project documents for housing projects at each of these locations, and at each installation we met with officials from the installation commander’s office and conducted interviews with officials from both the installation military housing office and representatives from the private partners. 3 To collect input from residents of privatized housing, we facilitated 15 focus groups with a self-selected group of current residents of privatized military housing. During the focus groups, a methodologist led participants through a structured questionnaire, which we pretested with 11 residents of privatized housing prior to the first focus group. To solicit participants for our focus groups, we requested that local military housing office officials email all current residents of privatized housing prior to our visit to inform them of our focus groups. Individuals interested in participating in our focus group sessions were instructed to contact us directly for further information. We had over 70 residents participate in our focus groups. In addition to the 15 focus groups, we conducted an additional five sessions in which fewer than three residents attended. We collected information from these residents, but we did not include their input in our focus group analysis. Comments from focus group participants are not generalizable to all residents of privatized military housing. We also developed and administered a publically available online tool that provided an opportunity for any resident of privatized military housing to voluntarily submit information on their experiences. Participants had the option to remain anonymous and make multiple submissions in order to provide us information on their experience at more than one installation. We developed our tool in conjunction with a survey methodologist to ensure it met our requirements for publically available anonymous data collection instruments, and conducted five pretests of the questions with residents of privatized housing. Our online tool was made available to the 3For a typical project, a military department leases land to a developer for a 50-year term and conveys existing housing units located on the leased land to the developer for the duration of the lease. Each privatized housing project is a separate and distinct entity governed by a series of legal agreements that are specific to that project. These agreements include, among others, an operating agreement, a property management agreement, and an agreement that describes the management of funds. Page 57 GAO-20-281 Military Housing Appendix I: Objectives, Scope, and Methodology public from June 17, 2019, through August 31, 2019. We received a total of 658 responses. In analyzing information provided through the online tool, we took steps to identify responses that did not meet our criteria, including removing 13 responses for reasons such as responses with duplicative usernames or Internet Protocol (IP) addresses that described the same experience or had been started but not fully completed, responses from DOD officials that informed us they had provided responses to test our tool, and responses from residents living on installations outside of the United States. In reporting results from our online tool, we used the following qualifiers in presenting our results— most (to indicate 80 percent or higher); majority (to indicate 51-79 percent); and some (to indicate less than 50 percent). Findings from our focus groups and online tool are not generalizable to all privatized military housing residents. To determine the extent to which DOD conducts oversight of privatized military housing for servicemembers and their families, we conducted the following additional data analysis. Through the steps described in the following bullets, we determined these data to be reliable for the purposes of our findings: • To determine the extent to which performance incentive fee metrics assessed the condition of privatized housing, we collected information on the structure of the incentive fees from private partners for 74 privatized housing projects and received confirmation that there are 5 projects that do not have incentive fee plans as part of their business agreements. We reviewed all of the incentive fee plans and identified commonly used metrics and indicators. We met with officials from the military housing offices, the military departments, and private partner representatives to discuss the administration and measurement of the incentive fee structures. To gain an understanding of how performance incentive fees are used, we reviewed documents and guidance from OSD and the military departments that explains the processes for developing and awarding performance incentive metrics and fees. In addition, we obtained information from residents through focus groups and our online tool and spoke with military housing office officials to obtain anecdotal information regarding the extent to which the metrics are adequately measuring the condition of the housing. • To assess the extent to which private partner work order data could be used to monitor and track the condition of privatized homes, we collected and reviewed private partner work order data from October 2016 through April 2019 from each of the 79 MHPI projects and discussed these data with the private partners and military department Page 58 GAO-20-281 Military Housing Appendix I: Objectives, Scope, and Methodology officials. Given that we requested the work order data from the private partners in April and May 2019, we selected the October 2016 through April 2019 time frame to include complete data for fiscal years 2017 and 2018 and the most comprehensive data available at the time for fiscal year 2019. Prior to requesting these data, we contacted representatives from each of the 14 private partner companies to discuss our forthcoming data request and to better understand each company’s data system and potential limitations for using the data. Subsequently, we requested that each partner provide us with data for all work orders across all data elements for each installation under their management. We received data on over 8 million work orders among the 14 private partners. We performed manual testing on initial data files received by each partner to identify issues that would impact the validity and reliability of using these data for ongoing monitoring and tracking of the condition of privatized housing units. In doing so, we identified instances of anomalies in work order data from each of the 14 partners. For 12 of the 14 partners, we found at least one of the following anomalies in the initial work order data files received for the time period requested: (1) duplicate work orders; (2) work orders with completion dates prior to the dates that a resident had submitted the work order; and (3) work orders still listed as in-progress for more than 18 months. We reviewed work order data from at least one installation for each private partner to check for instances of these anomalies. We also held follow-up discussions with 10 of the 14 private partners to discuss anomalies found in the data and potential factors contributing to the presence of these anomalies. In addition to the initial data collected on all of the work orders, we requested a second data run of work orders over the same time period—October 1, 2016, through April 30, 2019—for service requests related to leadbased paint, mold, and pest/rodent/vermin infestation. As part of this request, we asked that partners provide the criteria used for querying the data they provided us. We reviewed these data to determine how requests for data on specific hazards were getting sorted into the work order tracking systems by category and priority among the various partners. To identify differences in terminology and business practices used by the private partners in their work order tracking systems, we requested and reviewed private partner documentation of data definitions and protocols for managing work order data. In addition, we conducted interviews with military department officials to discuss ongoing efforts by the military departments to collect and analyze work order data. Page 59 GAO-20-281 Military Housing Appendix I: Objectives, Scope, and Methodology • To evaluate resident satisfaction data reported in OSD’s reports to Congress on privatized housing, we reviewed the processes for collecting, calculating, and reporting these data for the three most recently issued reports for fiscal years 2015, 2016, and 2017. We reviewed the instructions OSD provided to the military departments outlining how the military departments are to submit resident satisfaction data to OSD. We also reviewed the question the military departments asked on their annual surveys to gauge resident satisfaction. We then requested the survey data the military departments submitted to OSD to be included in the annual report to Congress for fiscal years 2015, 2016, and 2017. We performed data quality checks and identified inaccuracies on a random sample of data reported by OSD. We reviewed how military departments calculated overall resident satisfaction for each privatized housing project. Further, we discussed these data with OSD and the military departments to assess the validity and reliability of using these data for identifying overall tenant satisfaction with the condition of privatized housing. To determine the extent to which the military departments have communicated their respective military housing office roles and responsibilities to residents, we reviewed military department policies and guidance related to their roles and responsibilities for working with residents of privatized housing. During our site visits to 10 installations, we interviewed military department housing office officials and private partner representatives to discuss their specific roles and responsibilities. We asked questions soliciting information about residents understanding of the roles and responsibilities of the military housing office and the dispute resolution process during our 15 focus groups. We also solicited resident feedback in our online tools regarding residents’ experience reporting maintenance issues and working with military housing offices and private partners to get maintenance issues resolved. To determine the extent to which DOD and private partners have developed and implemented initiatives to improve privatized housing, we interviewed OSD and military department officials to discuss ongoing initiatives developed over the course of our audit work aimed at improving MHPI and reviewed relevant guidance. We met with private partner representatives to discuss their involvement in developing these initiatives, as well as to gain an understanding of any challenges or concerns that may impact the implementation of these initiatives. Following the passage of the National Defense Authorization Act for Fiscal Year 2020, we reviewed provisions of the statute designed to Page 60 GAO-20-281 Military Housing Appendix I: Objectives, Scope, and Methodology improve the condition of privatized housing and evaluated the extent to which these provisions would impact ongoing or planned DOD initiatives or provide new oversight roles and responsibilities for OSD and the military departments. We discussed these provisions with OSD officials and private partner representatives to understand how, if at all, their implementation may impact the privatized housing projects, as well as any potential barriers to implementation in the current legal construct of the program. We conducted this performance audit from November 2018 to March 2020 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Page 61 GAO-20-281 Military Housing Appendix II: List of Privatized Military Housing Projects as of September 30, 2017 Appendix II: List of Privatized Military Housing Projects as of September 30, 2017 In table 3, we provide the complete listing of the Department of Defense’s 79 privatized military housing projects, as of September 30, 2017. 1 This list reflects information that the Office of the Assistant Secretary of Defense for Sustainment provided in its annual report to Congress for the time period of October 1, 2016, through September 30, 2017. The report was provided to Congress in May 2019. The projects can consist of one or multiple installations. Table 3: Privatized Military Housing Projects as of September 30, 2017 Military department Project Installation/state Air Force Air Combat Command (ACC) Group II Davis-Monthan Air Force Base, Arizona Holloman Air Force Base, New Mexico Air Combat Command (ACC) Group III Dyess Air Force Base, Texas Moody Air Force Base, Georgia Air Education & Training Command Group I Altus Air Force Base, Oklahoma Luke Air Force Base, Arizona Sheppard Air Force Base, Texas Tyndall Air Force Base, Florida Air Education & Training Command Group II Columbus Air Force Base, Mississippi Goodfellow Air Force Base, Texas Laughlin Air Force Base, Texas Maxwell Air Force Base, Alabama Joint Base San Antonio-Randolph Air Force Base, Texas Vance Air Force Base, Oklahoma Air Force Academy United States Air Force Academy, Colorado Air Mobility Command (AMC) East Andrews Air Force Base, Maryland MacDill Air Force Base, Florida Air Mobility Command (AMC) West Fairchild Air Force Base, Washington Tinker Air Force Base, Oklahoma Travis Air Force Base, California BLB Group Barksdale Air Force Base, Louisiana Langley Air Force Base, Virginia Bolling Air Force Base, District of Columbia Buckley Air Force Base Buckley Air Force Base, Colorado Continental Group Edwards Air Force Base, California 1This sold. table does not include projects listed in DOD’s annual report to Congress that were Page 62 GAO-20-281 Military Housing Appendix II: List of Privatized Military Housing Projects as of September 30, 2017 Military department Project Installation/state Air Force Eglin Air Force Base, Florida Eielson Air Force Base, Alaska Hurlburt Field, Florida McConnell Air Force Base, Kansas Seymour-Johnson Air Force Base, North Carolina Dover Air Force Base Dover Air Force Base, Delaware Dyess Air Force Base Dyess Air Force Base, Texas Joint Base Elmendorf-Richardson (JBER I)– Elmendorf Air Force Base Joint Base Elmendorf-Richardson (JBER I)– Elmendorf Air Force Base, Alaska Joint Base Elmendorf-Richardson (JBER II)– Elmendorf Air Force Base Joint Base Elmendorf-Richardson (JBER II)– Elmendorf Air Force Base, Alaska Joint Base Elmendorf-Richardson (JBER III)– Richardson Air Force Base Joint Base Elmendorf-Richardson (JBER III)– Richardson Air Force Base, Alaska Falcon Group Hanscom Air Force Base, Massachusetts Little Rock Air Force Base, Arkansas Moody Air Force Base, Georgia Patrick Air Force Base, Florida Joint Base Pearl Harbor-Hickam–Hickam Air Force Joint Base Pearl Harbor-Hickam–Hickam Air Force Base Base, Hawaii Hill Air Force Base Hill Air Force Base, Utah Kirtland Air Force Base Kirtland Air Force Base, New Mexico Joint Base San Antonio-Lackland Air Force Base Joint Base San Antonio–Lackland Air Force Base, Texas Joint Base McGuire-Dix-Lakehurst–McGuire Air Force Base/Joint Base McGuire-Dix-Lakehurst– Fort Dix Joint Base McGuire-Dix-Lakehurst–McGuire Air Force Base, New Jersey Joint Base McGuire-Dix-Lakehurst–Fort Dix, New Jersey Nellis Air Force Base Nellis Air Force Base, Nevada Northern Group Cannon Air Force Base, New Mexico Cavalier Air Force Station, North Dakota Ellsworth Air Force Base, South Dakota Grand Forks Air Force Base, North Dakota Minot Air Force Base, North Dakota Mountain Home Air Force Base, Idaho Offutt Air Force Base Offutt Air Force Base, Nebraska Robins Air Force Base I Robins Air Force Base I, Georgia Robins Air Force Base II Robins Air Force Base II, Georgia Scott Air Force Base Scott Air Force Base, Illinois Southern Group Arnold Air Force Base, Tennessee Page 63 GAO-20-281 Military Housing Appendix II: List of Privatized Military Housing Projects as of September 30, 2017 Military department Project Installation/state Air Force Joint Base Charleston-Charleston Air Force Base, South Carolina Keesler Air Force Base, Mississippi Shaw Air Force Base, South Carolina Tri-Group Los Angeles Air Force Base, California Peterson Air Force Base, Colorado Schriever Air Force Base, Colorado Vandenberg Air Force Base Vandenberg Air Force Base, California Western Group Beale Air Force Base, California FE Warren Air Force Base, Wyoming Malmstrom Air Force Base, Montana Whiteman Air Force Base, Missouri Army Wright-Patterson Air Force Base Wright-Patterson Air Force Base, Ohio Aberdeen Proving Ground Aberdeen Proving Ground, Maryland Fort Belvoir Fort Belvoir, Virginia Fort Benning Fort Benning, Georgia Fort Bliss/White Sands Missile Range Fort Bliss, Texas White Sands Missile Range, New Mexico Fort Bragg Fort Bragg, North Carolina Fort Campbell Fort Campbell, Kentucky Carlisle Barracks/Picatinny Arsenal Carlisle Barracks, Pennsylvania Picatinny Arsenal, New Jersey Fort Carson Fort Carson, Colorado Fort Detrick/Walter Reed Army Medical Center Fort Detrick, Maryland Walter Reed Army Medical Center, District of Columbia Fort Drum Fort Drum, New York Joint Base Langley‐Eustis–Fort Eustis/Joint Expeditionary Base Little Creek‐Fort Story–Fort Story Joint Base Langley-Eustis–Fort Eustis, Virginia Joint Expeditionary Base Little Creek‐Fort Story–Fort Story, Virginia Fort Gordon Fort Gordon, Georgia Fort Hamilton Fort Hamilton, New York Fort Hood Fort Hood, Texas Fort Huachuca/Yuma Proving Ground Fort Huachuca, Arizona Yuma Proving Ground, Arizona Fort Irwin/Moffett Field/Parks Reserve Forces Training Area Fort Irwin, California Moffett Field, California Page 64 GAO-20-281 Military Housing Appendix II: List of Privatized Military Housing Projects as of September 30, 2017 Military department Project Installation/state Army Parks Reserve Forces Training Area, California Fort Jackson Fort Jackson, South Carolina Fort Knox Fort Knox, Kentucky Fort Leavenworth Fort Leavenworth, Kansas Fort Lee Fort Lee, Virginia Fort Leonard Wood Fort Leonard Wood, Missouri Fort Meade Fort Meade, Maryland Fort Polk Fort Polk, Louisiana Joint Base Lewis-McChord Joint Base Lewis-McChord–Fort Lewis, Washington Joint Base Lewis-McChord–McChord Air Force Base, Washington Presidio of Monterey/Naval Postgraduate School Presidio of Monterey, California Naval Post Graduate School, California Redstone Arsenal Redstone Arsenal, Alabama Fort Riley Fort Riley, Kansas Fort Rucker Fort Rucker, Alabama Joint Base San Antonio–Fort Sam Houston Joint Base San Antonio–Fort Sam Houston, Texas Fort Shafter/Schofield Barracks Fort Shafter, Hawaii Schofield Barracks, Hawaii Fort Sill Fort Sill, Oklahoma Fort Stewart/Hunter Army Airfield Fort Stewart, Georgia Hunter Army Airfield, Georgia Fort Wainwright/Fort Greely Fort Wainwright, Alaska Fort Greely, Alaska Navy/Marine Corps West Point United States Military Academy at West Point, New York Marine Corps Base Camp Pendleton I Marine Corps Base Camp Pendleton I, California Cherry Point/Camp Lejeune Overview (Atlantic Marines) Marine Corps Base Camp Lejeune, North Carolina Marine Corps Air Station Cherry Point, North Carolina Marine Corps Air Station New River, North Carolina Stewart Air National Guard Base, New York Westover Air Reserve Base, Massachusetts Marine Corps Air Station Beaufort, South Carolina Marine Corps Recruit Depot Parris Island, South Carolina Naval Hospital Beaufort, South Carolina Page 65 GAO-20-281 Military Housing Appendix II: List of Privatized Military Housing Projects as of September 30, 2017 Military department Project Installation/state Navy/Marine Corps Hawaii Regional Joint Base Pearl Harbor-Hickam–Naval Station Pearl Harbor, Hawaii Marine Corps Base Hawaii Kaneohe Bay, Hawaii Pacific Missile Range Facility Barking Sands, Hawaii Naval Air Station Kingsville II Naval Air Station Kingsville II, Texas Mid-Atlantic Regional Naval Sea Systems Norfolk Naval Shipyard, Virginia Joint Expeditionary Base Little Creek-Fort Story– Naval Amphibious Base Little Creek, Virginia Naval Support Activity Hampton Roads, Virginia Naval Air Station Oceania, Virginia Naval Station Norfolk, Virginia Weapons Station Yorktown, Virginia Naval Support Activity Annapolis–United States Naval Academy, Maryland Naval Support Activity South Potomac–Dahlgren, Virginia Naval Support Activity South Potomac-Indian HeadMaryland Naval Air Station Patuxent River, Maryland Navy Information Operations Command Sugar Grove, West Virginia Naval Support Activity Washington–Tingey House, District of Columbia Naval Support Activity Mechanicsburg, Pennsylvania Marine Corps Base Camp Lejeune, North Carolina Midwest Regional Naval Station Great Lakes, Illinois Naval Support Activity Crane, Indiana Naval Support Activity Mid-South, Tennessee Naval Air Station Joint Reserve Base New Orleans Naval Air Station Joint Reserve Base New Orleans, Louisiana Northeast Regional Joint Base McGuire-Dix-Lakehurst–Naval Air Engineering Station Lakehurst, New Jersey Submarine Base New London, Connecticut Naval Station Newport, Rhode Island Naval Shipyard Portsmouth, New Hampshire Naval Support Activity Saratoga Springs, New York Mitchel Complex Navy Recruiting District, New York Weapons Station Earle, New Jersey Northwest Regional Naval Base Kitsap, Washington Naval Air Station Whidbey Island, Washington Page 66 GAO-20-281 Military Housing Appendix II: List of Privatized Military Housing Projects as of September 30, 2017 Military department Project Installation/state Navy/Marine Corps Naval Station Everett, Washington PE/QU/YU (Camp Pendleton II) Marine Corps Base Camp Pendleton, California Marine Corps Base Quantico, Virginia Marine Corps Air Station Yuma, Arizona Marine Corps Air Ground Combat Center Twentynine Palms, California Marine Corps Mobilization Command Kansas City, Missouri Marine Corps Logistics Base Albany, Georgia San Diego Naval Complex Overview Naval Station San Diego, California Naval Base Coronado, California Naval Base Point Loma, California Marine Corps Air Station Miramar, California Naval Air Weapons Station China Lake, California Naval Air Station Lemoore, California Naval Base Ventura County, California Naval Air Facility El Centro, California Weapons Station Seal Beach, California Naval Air Station Fallon, Nevada Naval Support Activity Washington, District of Columbia Joint Base Anacostia-Bolling–Naval Support Facility Anacostia, District of Columbia Naval Support Activity Annapolis–Buchanan House, Maryland Naval Support Activity Bethesda, Maryland Naval Support Facility Thurmont–Camp David, Maryland South Texas Naval Air Station Corpus Christi, Texas Naval Station Ingleside, Texas Southeast Regional Naval Air Station Pensacola, Florida Naval Air Station Whiting Field, Florida Naval Support Activity Panama City, Florida Joint Base Charleston–Naval Weapons Station Charleston, South Carolina Naval Station Mayport, Florida Naval Air Station Jacksonville, Florida Submarine Base Kings Bay, Georgia Naval Air Station Key West, Florida Page 67 GAO-20-281 Military Housing Appendix II: List of Privatized Military Housing Projects as of September 30, 2017 Military department Project Installation/state Navy/Marine Corps Naval Air Station Joint Reserve Base Fort Worth, Texas Naval Air Station Meridian, Mississippi Naval Construction Battalion Center Gulfport, Mississippi Source: GAO presentation of Department of Defense information. GAO-20-281 Page 68 GAO-20-281 Military Housing Appendix III: Comments from the Department of Defense Appendix III: Comments from the Department of Defense Page 69 GAO-20-281 Military Housing Appendix Comments from the Department of Defense GAO REPORT DATED MARCH 2020 GAO-20-281 (GAO CODE 103153) HOUSING: DOD NEEDS TO STRENGTHEN OVERSIGHT AND CLARIFY ITS ROLE IN THE MANAGEMENT OF PRIVATIZED DEPARTMENT OF DEFENSE COMMENTS TO THE GAO RECOMMENDATIONS RECOMMENDATION 1: The Secretary of Defense should ensure that the Assistant Secretary of Defense for Sustainment, in collaboration with the military departments, provide updated guidance for oversight of privatized military housing, to include oversight objectives for each service to monitor the physical condition of privatized homes over the remaining duration of the ground leases. RESPONSE: Concur. The Assistant Secretary of Defense for Sustainment will issue updated guidance that includes oversight objectives for each military department to monitor the the physical condition of housing privatized under the Military Housing Privatization Initiative (MHPI) over the duration of their ground leases. RECOMMENDATION 2: The Secretary of the Army should take steps, in collaboration with the Army?s private housing partners, to review the indicators underlying the privatized housing project performance metrics to ensure they provide an accurate re?ection of the condition and qualtiy of the homes. RESPONSE: Concur. The Secretary of the Army will take steps, in collaboration with the Army?s MHPI project partners, to review the indicators underlying the MHPI housing project performance metrics to ensure they provide an accurate re?ection of the condition and quality of the homes. RECOMMENDATION 3: The Secretary of the Air Force should take steps, in collaboration with the Air Force?s private housing partners, to review the indicators underlying the privatized housing project performance metrics to ensure they provide an accurate re?ection of the condition and qulatiy of the homes. RESPONSE: Concur. The Secretary of the Air Force reviewed the key performance indicators the Air Force uses to monitor the performance of its privatized housing projects and found they are good indicators of some of the many facets of project performance, but that additional metrics are needed to ensure appropriate focus on resident satisfaction and the condition and quality of the homes (completed Oct 2019). The Services have adopted the October 28, 2019, common performance incentive fee framework that assesses resident satisfaction, maintenance management, project safety and ?nancial management. The Air Force is currently negotiating the revised framework with each project owner. To implement the revised incentive fee framework the Department of the Air Force is working with its project owners to deploy Satisfacts, a third-party survey company that collects resident feedback and Page 70 GAO-20-281 Military Housing Appendix Comments from the Department of Defense provides that unedited feedback to the Air Force and the project owners. Sixty of 63 project locations have implemented Satisfacts and we expect the remaining projects owners to implement the tool by December 2020. The Satisfacts survey program provides residents with an opportunity to provide feedback on the quality and completeness of the work performed in the homes. Additionally, implementation requires Military Housing Of?ce personnel to conduct inspections of all move-in, move-out, and change of occupancy maintenance (COM) events and all emergency, urgent, and life, health, and safety work orders as staf?ng at the installations can support the work load, The Air Force believes the pass rate ofthose inspections will provide valuable insight into the quality of the homes available to service members; we are incorporating those inputs into the revised incentive fee framework. This increase in resident feedback, inspections, and focus on staffing levels will give the Air Force better insight into the condition of the homes and quality of customer service. RECOMMENDATION 4: The Secretary of the Navy should take steps, in collaboration with the Navy and Marine Corps? private housing partners, to review the indicators underlying the privatized housing project performance metrics to ensure they provide an accurate reflection of the condition and quality of the homes. RESPONSE: Concur. The Navy and Marine Corps are updating the monitoring matrix by which housing officials evaluate the performance of privatized housing partners in order to better capture property manager performance with respect to the condition and quality of privatized homes. The Navy and Marine Corps have also developed a centralized electronic data warehouse that directly interfaces with privatized housing partner maintenance systems. An increased number of Navy and Marine Corps installation and regional housing officials will use the updated monitoring matrix and electronic data warehouse to provide more consistent oversight of privatized and government housing programs. This will be accomplished by reviewing trends in health and safety related service calls, repeat service calls, resident feedback on performance of property maintenance, sampling records for maintenance service calls, and change of occupancy maintenance completion. The Navy and Marine Corps also require privatized housing panners to provide recurring maintenance and customer satisfaction reports, which housing officials will review for trends and verify through more frequent site assessments. These efforts will result in improved advocacy for Sailors, Marines and their families. RECOMMENDATION 5: The Secretary of Defense should ensure that the Assistant Secretary of Defense for Sustainment, in collaboration with the military departments and private housing partners, establish minimum data requirements and consistent terminology and practices for work order collection for comparability across installations and projects and to track trends over time. RESPONSE: Partially Concur. The in collaboration with the military departments and MHPI housing partners, will establish, to the maximum extent practical, minimum data requirements and consistent terminology and practices for MHPI housing unit work order collection to aid in comparability across installations and projects, and for tracking trends over time. However, neither the nor the Military Departments can mandate changes to existing MHPI project ground leases or legal agreements, to include mandating that at Page 71 GAO-20-281 Military Housing Appendix Comments from the Department of Defense private entity an MHPI Project Partner Owner) change its own internal business practices or terminology for housing units that are privately owned and operated. MHPI projects are not contracts between Military Departments and MHPI project developers; cannot unilaterally make changes to the MHPI project ground leases and associated legal documents without MHPI Project partner landlord concurrence. RECOMMENDATION 6: The Secretary of Defense should ensure that the Assistant Secretary of Defense for Sustainment require the military departments to establish a process to validate data collected by the private housing partners to better ensure the reliability and validity of work order data and to allow for more effective us of these data for monitory and tracking purposes. RESPONSE: Concur. The will issue guidance directing the Military Departments to establish a process to validate data collected by their respective MHPI Project Partners Owners to better ensure the reliability and validity of MHPI Project work order data and to allow for more effective us of these data for monitory and tracking purposes. RECOMMENDATION 7: The Secretary of Defense should ensure the Under Secretary of Defense for Sustainment. in collaboration with the military departments. develops a process for collecting and calculating resident satisfaction data from the military departments to ensure that the data are compiled and calculated in a standardized and accurate way. RESPONSE: Partially Concur. Recognizing that there is no Under Secretary of Defense for Sustainment. the in collaboration with the military departments. will refine the process for collecting and calculating resident satisfaction data from the military departments to ensure that the data are compiled and calculated in a standardized and accurate way effective with the survey collection effort for Fiscal Year 202]. RECOMMENDATION 8: The Secretary ofDefense should ensure the Assistant Secretary of Defense for Sustainment provides additional explanation of the data collected and reported in future reports to Congress, such as explaining the limitations of available survey data, how resident satisfaction was calculated. and resons for any missing data, among other things. RESPONSE: Concur. The will provide additional explanation of the MHPI resident satisfaction data collected and reported in future annual MHPI reports to Congress, effective with the annual report covering fiscal year 2019, to include explaining the limitations of available survey data. how resident satisfaction was calculated. and resons for any missing data, among other things. RECOMMENDATION 9: The Secretary of the Army should develop and implement a plan to clearly and systematically communicate to residents the difference between the military housing office and the private partner. At a minimum, these plans should include the Army housing office?s roles, responsibilities, locations, and contact information and should ensure that all residents are aware that they can directly contact Army housing of?ce officials. RESPONSE: Concur. The Army will develop and implement a plan to clearly and systematically communicate to residents the difference between the military housing office and U.) Page 72 GAO-20-281 Military Housing Appendix Comments from the Department of Defense the MHPI project partner, and will ensure that the plan identi?es the Army housing of?ce?s roles, responsibilities, locations, and contact information and inform all residents that they can directly contact Army housing of?ce of?cials. RECOMMENDATION 10: The Secretary of the Air Force should develop and implement a plan to clearly and systematically communicate to residents the difference between the military housing of?ce and the private partner. At a minimum, these plans should include the Air Force housing of?ce?s roles, responsibilities, locations, and contact information and should ensure that all residents are aware that they can directly contact Air Force housing of?ce of?cials. RESPONSE: Concur. The Secretary of the Air Force added guidance to the Air Force Instruction (AF 1) 32-6000, Family Housing regarding collocation of the Military Housing Of?ce (MT-IO) with the project owneris staff. The new guidance will direct all Air Force Military Housing Of?ces to clearly distinguish themselves from project owner personnel through signage, appearance (uniform/nametags), and by clearly designating space for residents to meet with MHO personnel to discuss issues they are experiencing with the privatized project. The applicable Air Force Instruction, AF I 32-6000, is scheduled for publication in March 2020. The Air Force is developing a move?in checklist and new resident brie?ng that will emphasize the roles, responsibilities, locations, and contact information. It will also notify residents of their rights and responsibilities. The brie?ng will inform residents they can directly contact Air Force housing of?ce of?cials, including the installation commander, ifthey are not satis?ed with the housing to which they are assigned or with the project partner?s resolution of their concerns. We plan to roll out these brie?ngs in March 2020. Fifty four of 63 project locations either already have distinctive signage or are not collocated. Several ofthe locations without distinctive signage are installing the signage within the next 30 days. Only one of 63 project locations lacks a private counseling area; however, this location does have several rooms they can use for private counseling. RECOMMENDATION 11: The Secretary of the Navy should develop and implement a plan to clearly and systematically communicate to residents the difference between the military housing of?ce and the private partner. At a minimum, these plans should include the Navy housing of?ce?s roles, responsibilities, locations, and contact information and should ensure that all residents are aware that they can directly contact Navy housing of?ce of?cials. RESPONSE: Concur. DON has taken multiple steps to communicate to residents its role in PPV project oversight and the different role of the housing provider: 0 Distribution of notices to all residents ofissue resolution options that include contacting the housing service center or chain of command. 0 Briefings to new residents on the support offered by the Housing Of?ce. 0 Move in inspections and contact that reinforce the services offered by the Housing Office. 0 Updated resident handbooks that explain the military housing office's role and support for residents. Page 73 GAO-20-281 Military Housing Appendix Comments from the Department of Defense 0 Customer satisfaction surveys that provide opportunity for the DON to hear from residents. 0 (Navy Only) Housing Offices issuance of magnets to every household to provide a reminder reinforcing the support available. 0 Updating housing offices' social media and public websites with updated information to support residents or privatized housing. 0 Navy and Marine Corps Public affairs teams actively communicating with residents regarding the Housing Offices' roles and responsibilities and those of the private partners via social media platforms. Public Affairs also provides coverage of quarterly town hall meetings, resident satisfaction survey promotion, and marketing of housing programs. 0 in addition to hiring additional housing officials, the Marine Corps has also allocated military personnel to serve as Resident Advocates to provide the services and support previously mentioned until permanent civilian hires are completed. RECOMMENDATION 12: The Secretary of Defense should ensure that the Assistant Secretary of Defense for Sustainment, in collaboration with the military depratments, assess the risks of proposed initiatives aimed at improving the privatized military housing program on the financial viability of the projects. RESPONSE: Concur. The in collaboration with the Military Departments, will assess the risks of proposed initiatives aimed at improving the privatized military housing program on the financial viability of the projects. The Department of Defesne is committed to the long?term success of the MHPI projects and MHPI program, and will continue our oversight of the MHPI portfolio to ensure delivery of safe, quality, well-maintained housing for Service members and their families over the life of the proj ects. However, the Department is concerned about the new ?nancial requirements placed on the MHPI projects by the National Defense Authorization Act for Fiscal Year 2020 (Public Law 116-92) and will monitor their impacts. Page 74 GAO-20-281 Military Housing Appendix IV: GAO Contact and Staff Acknowledgments Appendix IV: GAO Contact and Staff Acknowledgments GAO Contact Elizabeth A. Field, Director, (202) 512-2775 or FieldE1@gao.gov. Staff Acknowledgments In addition to the contact above, the following are key contributors to this report: Kristy Williams (Assistant Director), Tida Barakat Reveley (Analyst in Charge), Austin Barvin, Ronnie Bergman, Vincent Buquicchio, William Carpluk, Juliee Conde-Medina, Mae Jones, Jordan Tibbetts, Kelly Rubin, Monica Savoy, and John Van Schaik. Page 75 GAO-20-281 Military Housing Related GAO Products Related GAO Products Military Housing: Preliminary Recommendations to Strengthen DOD's Oversight and Monitoring of Privatized Housing. GAO-20-471T. Washington, D.C.: March 3, 2020. Military Housing Privatization: Preliminary Observations on DOD’s Oversight of the Condition of Privatized Military Housing. GAO-20-280T. Washington, D.C: December 3, 2019. Defense Infrastructure: Additional Actions Could Enhance DOD’s Efforts to Identify, Evaluate, and Preserve Historic Properties. GAO-19-335. Washington, D.C.: June 19, 2019. Military Housing Privatization: DOD Should Take Steps to Improve Monitoring, Reporting, and Risk Assessment. GAO-18-218. Washington, D.C.: March 13, 2018. Defense Infrastructure: Army Has a Process to Manage Litigation Costs for the Military Housing Privatization Initiative. GAO-14-327. Washington, D.C.: April 3, 2014. Military Housing: Information on the Privatization of Unaccompanied Personnel Housing. GAO-14-313. Washington, D.C.: March 18, 2014. Military Housing: Enhancements Needed to Housing Allowance Process and Information Sharing among Services. GAO-11-462. Washington, D.C.: May 16, 2011. Military Housing Privatization: DOD Faces New Challenges Due to Significant Growth at Some Installations and Recent Turmoil in the Financial Markets. GAO-09-352. Washington, D.C.: May 15, 2009. Military Housing: Management Issues Require Attention as the Privatization Program Matures. GAO-06-438. Washington, D.C.: April 28, 2006. Military Housing: Further Improvement Needed in Requirements Determination and Program Review. GAO-04-556. Washington, D.C.: May 19, 2004. Military Housing: Better Reporting Needed on the Status of the Privatization Program and the Costs of Its Consultants. GAO-04-111. Washington, D.C.: October 9, 2003. Page 76 GAO-20-281 Military Housing Related GAO Products Military Housing: Opportunities That Should Be Explored to Improve Housing and Reduce Costs for Unmarried Junior Servicemembers. GAO-03-602. Washington, D.C.: June 10, 2003. Military Housing: Management Improvements Needed as the Pace of Privatization Quickens. GAO-02-624. Washington, D.C.: June 21, 2002. Military Housing: DOD Needs to Address Long-Standing Requirements Determination Problems. GAO-01-889. Washington, D.C.: August 3, 2001. Military Housing: Continued Concerns in Implementing the Privatization Initiative. GAO/NSIAD-00-71. Washington, D.C.: March 30, 2000. Military Housing: Privatization Off to a Slow Start and Continued Management Attention Needed. GAO/NSIAD-98-178. Washington, D.C.: July 17, 1998. (103153) Page 77 GAO-20-281 Military Housing GAO’s Mission The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony The fastest and easiest way to obtain copies of GAO documents at no cost is through our website. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. You can also subscribe to GAO’s email updates to receive notification of newly posted products. Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, https://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Subscribe to our RSS Feeds or Email Updates. Listen to our Podcasts. Visit GAO on the web at https://www.gao.gov. To Report Fraud, Waste, and Abuse in Federal Programs Contact FraudNet: Congressional Relations Orice Williams Brown, Managing Director, WilliamsO@gao.gov, (202) 512-4400, U.S. Government Accountability Office, 441 G Street NW, Room 7125, Washington, DC 20548 Public Affairs Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 Strategic Planning and External Liaison James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707 U.S. Government Accountability Office, 441 G Street NW, Room 7814, Washington, DC 20548 Website: https://www.gao.gov/fraudnet/fraudnet.htm Automated answering system: (800) 424-5454 or (202) 512-7700 Please Print on Recycled Paper.