United States Government Accountability Office Report to Congressional Committees November 2019 NAVY READINESS Actions Needed to Evaluate the Effectiveness of Changes to Surface Warfare Officer Training GAO-20-154 November 2019 NAVY READINESS Actions Needed to Evaluate the Effectiveness of Changes to Surface Warfare Officer Training Highlights of GAO-20-154, a report to congressional committees Why GAO Did This Study What GAO Found In 2017, the Navy had four mishaps at sea including two collisions that resulted in the loss of 17 sailors’ lives and hundreds of millions of dollars in damage to Navy ships. In the wake of those mishaps, the Navy identified deficiencies in SWO ship-driving training and related experience as contributing factors and has undertaken a number of efforts to improve these areas. Since 2017, the Navy has made numerous changes and plans additional changes to enhance Surface Warfare Officer (SWO) ship-driving training. The Navy plans for these changes to result in a threefold increase in the number of initial ship-driving training hours for SWOs by 2021, compared with the number of training hours prior to the 2017 collisions (see fig.). The Navy added classroom and simulator time to existing training courses to improve ship-driving skills and is developing two additional simulator-based ship-driving courses planned for 2021. These plans hinge on the completion of two new simulator-based training facilities, scheduled for completion in June 2021 and in January 2023. Senate Report 115-262, accompanying a bill for the Fiscal Year 2019 National Defense Authorization Act, contained a provision that GAO assess SWO training. This report (1) describes the changes the Navy has made to SWO ship-driving training since the 2017 collisions and (2) assesses the extent to which the Navy has taken actions to evaluate the effectiveness of changes made to SWO ship-driving training. GAO reviewed and analyzed changes made to Navy training and assessment practices and related investments; interviewed cognizant officials; and conducted discussions with SWOs aboard 12 ships. What GAO Recommends GAO is making four recommendations to the Navy to routinely evaluate SWO training, including that the Navy collect and evaluate fleet-wide feedback on the quality of training; routinely conduct ship-driving competency assessments; provide standard criteria for qualifying ship drivers; and develop a plan to analyze and use logbook information. The Navy concurred with GAO’s recommendations. View GAO-20-154. For more information, contact John Pendleton (202) 512-3489, pendletonj@gao.gov USS Fitzgerald Damaged after 2017 Collision and Navy’s Subsequent Threefold Increase in Actual and Planned Ship-Driving Training Hours for Surface Warfare Officers’ First 3 Years The Navy has relied on added skill checks conducted throughout a SWO’s career to ensure that each SWO has basic ship-driving skills, but has not put key processes and assessments in place to evaluate comprehensively the effectiveness of its changes to ship-driving training. Senior Navy officials stated that it could take 16 years or more to know if the planned changes to SWO training were effective in increasing Commanding Officer ship-driving proficiency across the fleet and stated that they intend to closely monitor the implementation of changes to the training. However, GAO found that in planning an approach for evaluating the changes, the Navy has not: • identified a method to solicit fleet-wide feedback on the quality of the increased ship-driving training received by SWOs; • planned to routinely conduct ship-driving competency “spot checks” that were instituted after the 2017 collisions despite Navy inspectors having found concerns with more than 80 percent of SWOs’ ship-driving skills; • provided standard criteria to ship Commanding Officers for qualifying SWOs to drive ships, contributing to significant variance in ship-driving experience and competency levels across the fleet; nor developed a specific plan to analyze and use information from logbooks in which SWOs are to document ship-driving and related experience. Without addressing these challenges, the Navy cannot assess in the near term if the significant investments made to expand and enhance SWO ship-driving training are effective; further adjustments are necessary; and Navy ships are being operated safely at sea. • ______________________________________ United States Government Accountability Office Contents Letter 1 Background The Navy Has Enhanced Ship-Driving Training Following the 2017 Collisions and Plans to Triple Training Hours by 2021 The Navy is Implementing Additional Skill Checks but Has Not Taken Other Actions Necessary to Evaluate the Effectiveness of Changes to SWO Ship-Driving Training Conclusions Recommendations for Executive Action Agency Comments and Our Evaluation Appendix I 4 10 15 27 28 29 Overview of the Nine Navy Surface Ship Classes That Surface Warfare Officers Serve Aboard 36 Status of Navy Review Recommendations Related to Surface Warfare Officer Training 39 Appendix III Scope and Methodology 43 Appendix IV Current and Planned Ship-Driving Skill Checks to Be Conducted during a Surface Warfare Officer’s Career 48 Appendix V Comments from the Department of Defense 50 Appendix VI GAO Contact and Staff Acknowledgments 56 Table 1: Navy’s 12 Recommendations Related to Surface Warfare Officer (SWO) Initial Ship-Driving Training and Explanation, as of August 2019, for Why They Are Considered Implemented 40 Appendix II Tables Page i GAO-20-154 Navy Readiness Table 2: Information on Current and Planned Surface Warfare Officer Ship-Driving Skill Checks, as of July 2019 48 Figures Figure 1: Overview of Significant Mishaps at Sea for Navy Surface Ships, January–August 2017 Figure 2: Career Progression and Associated Shipboard Duties for the Navy’s Surface Warfare Officers (SWOs) Figure 3: Comparison of Hours of Ship-Driving Training EarlyCareer Surface Warfare Officers Were Required to Complete in 2017 Prior to the Ship Collisions, Are Required to Complete in 2019, and Will Be Required to Complete in 2021 Figure 4: Littoral Combat Ship Training Facility in San Diego, California Figure 5: Officer of the Deck Competency Assessment Results, January–March 2018 Figure 6: Surface Warfare Mariner Skills Logbook Figure 7: Nine Navy Surface Ship Classes That Surface Warfare Officers Serve Aboard 5 7 11 14 20 26 36 Abbreviations DOD SWO Department of Defense Surface Warfare Officer This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page ii GAO-20-154 Navy Readiness Letter 441 G St. N.W. Washington, DC 20548 November 14, 2019 Congressional Committees In 2017, the Navy had four significant mishaps at sea, including two collisions that resulted in the loss of 17 sailors’ lives and hundreds of millions of dollars in damage to Navy ships. 1 The Navy completed two internal reviews to identify and correct the root causes of the mishaps, ultimately compiling 111 recommendations to improve surface fleet readiness. 2 In the reviews, the Navy identified numerous contributing factors to the mishaps, including deficiencies in Surface Warfare Officer (SWO) training and related experience. SWOs are Navy officers whose training and primary duties focus on the safe operation of Navy surface ships at sea, management of various shipboard systems, and the leadership of ships’ crews. 3 The Navy’s identified deficiencies regarding SWOs focused on “ship-driving training” and skills related to directing the ship crew in safely navigating and handling a ship on its intended course. 4 In response to the internal review recommendations the Navy has undertaken a number of efforts to improve SWO training. The John S. McCain National Defense Authorization Act for Fiscal Year 2019 and Senate Armed Services Committee report 115-262 to 1 A “mishap” is an unplanned event or series of events that results in damage to Department of Defense (DOD) property; occupational illness to DOD personnel; injury to on- or off-duty DOD military personnel; injury to on-duty DOD civilian personnel; or damage to public or private property, or injury or illness to non-DOD personnel, caused by DOD activities. 2 U.S. Navy, Comprehensive Review of Recent Surface Force Incidents (Oct. 26, 2017) and U.S. Navy, Strategic Readiness Review 2017 (Dec. 3, 2017). 3 SWOs serve on surface ships from nine different classes: aircraft carriers, cruisers, destroyers, littoral combat ships, mine countermeasures ships, amphibious assault, amphibious command, amphibious transport, and dock landing ships. See appendix I for additional information on Navy surface ships. 4 Throughout this report we refer to mariner skills such as navigation, seamanship, and ship handling as “ship driving” which refers to the skills required to direct the ship crew in safely and effectively navigating and handling a ship within a wide variety of maneuvering situations. According to the Navy, the core competencies associated with navigation, seamanship, and ship handling include, but are not limited to, theory and practice of navigation at sea, basic ship handling, steering and sailing rules for preventing collisions at sea, shipboard damage control, and understanding and calculating relative motion between maneuvering ships. Page 1 GAO-20-154 Navy Readiness accompany a bill for the National Defense Authorization Act for Fiscal Year 2019 contained provisions that we review SWO training and career paths. For this report, we (1) describe the changes the Navy has made or planned to make to SWO ship-driving training since the 2017 collisions and (2) assess the extent to which the Navy has taken actions to evaluate the effectiveness of those changes. We plan to issue a separate report on SWO career paths in the future. For objective one, we reviewed Navy documentation from Commander, Naval Surface Forces and Surface Warfare Officers School Command regarding the content, purpose, cost, and status of changes made to and planned for ship-driving training since the 2017 collisions. We focused our analysis on changes made to SWOs’ ship-driving training at the junior officer level as they constitute the majority of changes made to date to address the recommendations of the Navy’s two 2017 internal reviews to ensure safe and effective operations at sea. See appendix II for a listing of Comprehensive and Strategic Readiness Review recommendations specific to SWO ship-driving training and their implementation status as of August 2019. We analyzed planned investments from fiscal year 2018 through fiscal year 2025 for the construction of two ship-driving training facilities and the development of three ship-driving training courses, which includes the cost of purchasing new simulators, hiring new instructors, conducting military construction, and developing course curriculum. We discussed implementation plans—including the status of those efforts for addressing the 2017 internal reviews’ recommendations—with the Commander, Naval Surface Forces; officials from Surface Warfare Officers School Command; Office of the Chief of Naval Operations Surface Warfare Directorate; and officials from the Readiness and Reform Oversight Council. 5 For objective two, we reviewed Navy documentation and interviewed Navy officials on how they evaluate SWOs throughout their careers, gather and use feedback from SWOs, assess the effectiveness of SWO ship-driving training, and use available data to inform decisions regarding SWO training. Specifically, we reviewed the implementation of the 10 career milestone checks outlined in Naval Surface Forces Instruction 1412.5 Surface Warfare Officer Milestone Mariner Skills Assessments, Evaluations, and Competency Checks that are to be administered during 5 The Readiness and Reform Oversight Council is a group within the Office of the Chief of Naval Operations established to manage the implementation of the internal reviews’ recommendations. Page 2 GAO-20-154 Navy Readiness a SWO’s career; the Navy’s efforts to collect feedback from the surface fleet on the quality of SWO ship-driving training and the health of the SWO community; the Navy’s 2018 Officer of the Deck competency assessment results, criteria, and plans to continue evaluating SWO shipdriving competency; the extent to which the Navy had provided standardized criteria for ships’ Commanding Officers to use when evaluating SWOs for ship-driving qualification; and the format of SWO Mariner Skills Logbooks used to track SWO ship-driving experiences, collected and reviewed Navy policies regarding the logbooks. To do this we compared the Navy’s practices with relevant Navy reviews, instructions, and guidance, Standards for Internal Control in the Federal Government, and our prior work on assessing training efforts in the federal government. 6 We assessed the reliability of the results of the Navy’s 2018 Officer of the Deck competency assessments by examining them for missing values, comparing other sources that provide the same types of data to ensure consistency, and by interviewing knowledgeable agency officials regarding their accuracy and completeness. In addition, we reviewed the Navy’s internal controls for performing the assessments, such as grading criteria and use of independent inspectors to ensure quality and consistency in the information. We determined that the results of the Navy’s 2018 Officer of the Deck competency assessments were sufficiently reliable for the purposes of reporting on the number and percentage of the graded categories. We also visited a total of 12 surface ships within the Pacific and Atlantic fleets from January through April 2019, selected according to which ships and crews were available at each of the sites we visited. Aboard these ships we held 24 group discussions with SWOs at two levels—Division Officers and Department Heads—and 12 interviews with Executive and Commanding Officers. In these visits we met with approximately 225 SWOs and discussed the quality of SWO initial training and how the Navy had implemented and assessed changes to training. The SWOs with whom we held group discussions were from ships that Commander, Naval Surface Forces, had identified as available for discussions during our site visits, and do not represent a generalizable sample of SWOs. 7 6 GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: March 2014) and Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO-04-546G (Washington, D.C.: March 2004). 7 Due to the timing of our work, the interviews and group discussions did not include SWOs that experienced changes made or planned for SWO training beyond April 2019. Page 3 GAO-20-154 Navy Readiness We conducted an analysis of the discussion group responses to identify common themes and provide illustrative examples in our report. Our scope and methodology are discussed in greater detail in appendix III. We conducted this performance audit from November 2018 to November 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background 2017 Mishaps at Sea From January through August 2017, the Navy suffered four significant mishaps at sea that resulted in the death of 17 sailors and hundreds of millions of dollars in damage to Navy surface ships (see fig.1). Page 4 GAO-20-154 Navy Readiness Figure 1: Overview of Significant Mishaps at Sea for Navy Surface Ships, January– August 2017 More recently, the Navy experienced two incidents during which Navy surface ships collided. First, on February 5, 2019, a Ticonderoga-class guided missile cruiser—USS Leyte Gulf (CG 55)—collided with a Navy resupply ship—the USNS Robert E. Peary (T-AKE 5)—while conducting an underway replenishment operation off the coast of Florida. Second, on June 21, 2019, a Freedom-class Littoral Combat Ship—the USS Billings (LCS 15)—struck a merchant ship while leaving a pier in Montreal, Page 5 GAO-20-154 Navy Readiness Canada. 8 According to Navy officials, these recent incidents did not result in serious damage to the ships or injuries to the crews but they demonstrate the need for continued focus and attention on safe ship driving. Surface Warfare Officers As of March 2019, the Navy had approximately 8,400 SWOs—Navy officers whose training and primary duties focus on the operation of Navy ships at sea and the management of various shipboard systems. The Navy expects SWOs to progress over the course of their careers from Division Officers driving ships, to Department Heads participating in combat operations, to Executive Officers managing ship crews, and to Commanding Officers overseeing operations. 9 Figure 2 below outlines a SWO’s career progression and associated shipboard duties. 8 An allision is a collision between a vessel against a fixed or a stationary object, such as another ship, buoy, or pier. 9 After serving as a Commanding Officer, Surface Warfare Officers can advance to the Major Command level, where they may provide combat and operations leadership to larger forces beyond a single ship. The SWO career path and training continuum were created to chart a course to develop experienced and capable Commanding Officers who are specialists in four areas: (1) Seamanship, Navigation, and Ship Handling; (2) Combat Systems and Maritime Warfighting; (3) Engineering, Material Readiness, and Program Management/Administration; and (4) Command and Leadership. The Navy defines “expected competencies” for SWOs over their career in Commander, Naval Surface Forces Instruction 1412.4A, Surface Warfare Officer Requirements Document, (Oct. 11, 2018). Page 6 GAO-20-154 Navy Readiness Figure 2: Career Progression and Associated Shipboard Duties for the Navy’s Surface Warfare Officers (SWOs) Note: Ship personnel and operations are organized under the leadership of the Commanding and Executive Officers into departments, which are further split into divisions. Surface Warfare Officer Training and At-Sea Experience The Commander, Naval Surface Forces, in coordination with Office of the Chief of Naval Operations, Surface Warfare Officers School Command, and Navy Personnel Command, manages and provides ship-driving training to SWOs throughout their careers. Initially, the primary focus of a Division Officer is on leading sailors and developing ship-driving competency, ultimately working toward qualification as an Officer of the Page 7 GAO-20-154 Navy Readiness Deck and Surface Warfare Officer. 10 Therefore, a large part of initial SWO training focuses on leading a division and developing the ship-driving skills needed to qualify and perform as an Officer of the Deck. This training is provided during a SWO’s initial training—in the Basic Division Officer Course—offered in Norfolk, Virginia, and San Diego, California, and is a mix of classroom and simulator-based training. After completing the Basic Division Officer Course, SWO candidates begin their first at-sea assignment as Division Officers. Division Officers have three primary roles aboard a ship: 1. They support ship-driving operations. New Division Officers gain shipdriving experience in pursuit of the qualification to stand as Officers of the Deck. Once qualified as Officers of the Deck, they lead watch teams in driving ships. 2. They support ship department operations under the supervision of Department Heads, and are responsible for a portion of ship equipment and operations. 11 3. They lead a division of approximately 12 to 50 enlisted personnel within departments, and are responsible for the administrative and supervisory duties for divisions. 12 In addition to their Division Officer responsibilities, new Division Officers are expected to earn qualification as a SWO by completing required 10 Officer of the Deck qualification requirements include: being qualified as a conning officer, combat information center watch officer, SWO engineering, bridge launcher control panel operator, small boat officer, and helm/aft steering helm safety officer. In addition, SWOs must stand at least 11 under-instruction watches, and participate in nine special evolutions (e.g., underway replenishment, flight operations, sea and anchor duty). Commanding Officers observe and validate the officers’ performance. SWO qualification requirements include: (1) successful completion of the Basic Division Officer Course; (2) completion of all fundamentals, systems, and watch station personnel qualification standards; (3) demonstration of effective leadership skills and proficiency in performing Division Officer duties, to include management of personnel, spaces, programs, and equipment as well as significant experience as a Bridge, Combat, and Engineering watchstander; and (4) an oral qualification board to validate officers’ general professional knowledge of all aspects of surface warfare. 11 A ship is divided into departments, such as engineering, operations, and combat systems, for carrying out specific missions assigned to the ship. 12 Ships’ departments are split up into divisions to carry out specific activities in support of the departments’ missions. For example, the engineering department can include divisions such as the auxiliary, electrical, main propulsion, and repair divisions. Page 8 GAO-20-154 Navy Readiness education and meeting experience standards, as well as gaining watchstanding experience and demonstrating proficiency in the execution of their duties, according to Navy officials. These qualifications include Officer of the Deck and Combat Information Center Watch Officer, where an officer assists in observing and analyzing information of importance for combat, among others. Navy officials stated that after an officer completes their qualifications, their Commanding Officer reviews the officer’s ability and experiences, and can grant the candidate SWO qualification. Prior GAO Work In June 2010, we reviewed Navy policies for surface force training— including initial SWO ship-driving training—and found that the Navy had reduced and altered initial SWO training as an efficiency measure, but lacked performance measures and data necessary to evaluate the impact of changes to training programs. 13 We found that in 2003, the Navy had replaced its 6-month Division Officer course consisting of classroom and simulator training with software-based training where new Division Officers were instead expected to learn SWO skills from computer-based education software while onboard their first ship. The Navy said this change saved about $50 million annually, but we found that the Navy lacked outcome-based performance measures to evaluate the effects of these changes to training on officer performance. We recommended that the Navy develop metrics to measure the effects of training on SWO job performance, knowledge, skills, and abilities. The Navy concurred with this recommendation but did not implement the recommendation for the software training or for subsequent training programs. 14 13 GAO, Military Readiness: Navy Needs to Reassess Its Metrics and Assumptions for Ship Crewing Requirements and Training, GAO-10-592 (Washington, D.C.: June 9, 2010). 14 According to Navy officials, the Navy ended the computer software-based training and returned to initial classroom and simulator training prior to a SWO’s first at-sea assignment beginning in late 2010. Page 9 GAO-20-154 Navy Readiness The Navy Has Enhanced ShipDriving Training Following the 2017 Collisions and Plans to Triple Training Hours by 2021 The Navy has enhanced ship-driving training for SWOs at the early stages of their careers following the 2017 collisions at sea, and by 2021 plans to triple the number of ship-driving training hours when compared with the amount of training SWOs were required to receive prior to the collisions. The Navy’s plans to increase ship-driving proficiency hinge on the completion of two new simulator-based training facilities—the Mariner Skills Training Centers—which are planned to be completed in June 2021 (San Diego, California) and in January 2023 (Norfolk, Virginia). Overall, the Navy plans to invest more than $467 million to develop new shipdriving training courses, build simulator facilities, and deliver the training through fiscal year 2025. Prior to the 2017 ship collisions, SWOs were required to complete 174 hours of ship-driving training during their Division Officer assignment by attending the Basic and Advanced Division Officer training courses. Following the collisions, the Navy increased the amount of required shipdriving training in these two courses to 203 hours. In June 2019, the Navy added a 4-week ship-driving course—the Junior Officer of the Deck course—that focused exclusively on building ship-driving skills. This course added 158 hours of required classroom and simulator ship-driving training. In June 2021, the Navy plans to expand the curriculum of the Junior Officer of the Deck course and rename it the Officer of the Deck Phase I course, and add an additional 3-week Officer of the Deck Phase II course. These two courses will add an additional 185 hours of required ship-driving training for Division Officers in preparation for their first and second at-sea assignments. 15 Once these ship-driving training courses are in place, Division Officers will be required to complete a total of 535 hours of training—triple (a threefold increase in) the number of shipdriving training hours SWOs were required to complete prior to the 2017 collisions (see fig. 3). 15 Once the Officer of the Deck Phase II training course is in place, Surface Warfare Officer School Command plans to reduce the amount of training in the Advanced Division Officer Course by 11 hours. Page 10 GAO-20-154 Navy Readiness Figure 3: Comparison of Hours of Ship-Driving Training Early-Career Surface Warfare Officers Were Required to Complete in 2017 Prior to the Ship Collisions, Are Required to Complete in 2019, and Will Be Required to Complete in 2021 Notes: The Junior Officer of the Deck course will transition into the Officer of the Deck Phase I course in 2021. Surface Warfare Officer School Command plans to reduce the hours of training in the Advanced Division Officer Course by 11 hours once the Officer of the Deck Phase II course begins in 2021. Below are detailed descriptions of the changes completed and planned to enhance ship-driving training. Basic Division Officer Course. From November 2017 through January 2019, the Surface Warfare Officers School Command changed the Basic Division Officer Course—a 9-week course for new SWO candidates—by increasing the required hours of classroom instruction and simulator training by 12 percent, and broadening the course curriculum. Specifically, prior to the 2017 collisions, SWO candidates were required to spend 113 hours (81 hours of classroom instruction and 32 hours in simulators) in this course to develop their ship-driving skills. After January 2019, however, SWO candidates were required to spend 126 hours (89 hours of classroom instruction and 37 hours in simulators) to develop their ship-driving skills. Regarding added course content, the Surface Warfare Officers School Command added subject matter including additional training on the internationally accepted ship-driving standards that govern ship maneuvers; radar navigation; and the tools used to aid ship-driving. Advanced Division Officer Course. From November 2017 through January 2019, the Surface Warfare Officers School Command changed the Advanced Division Officer Course—a 5-week course for SWOs Page 11 GAO-20-154 Navy Readiness returning from their first at-sea assignment—to improve ship-driving skills by increasing the required hours of simulator training from 24 to 36 hours. Prior to the 2017 collisions, SWOs were required to spend 61 hours (37 hours in the classroom and 24 hours in simulators) refining their shipdriving skills in this course. As of January 2019, SWOs were required to spend 77 hours (41 hours in the classroom and 36 hours in simulators) developing and honing their ship-driving skills. Surface Warfare Officers School Command officials also added subjects to classroom time to build on the subject matter presented in the Basic Division Officer Course, including more complex ship-driving techniques and advanced radar navigation. Surface Warfare Officers School Command plans to reduce the hours of training in this course once the Officer of the Deck Phase II course comes online in 2021. Junior Officer of the Deck course. In June 2019, Surface Warfare Officers School Command provided this new 4-week course for the first time—the course having been developed after the 2017 collisions and focused predominately on building ship-driving skills. 16 The Junior Officer of the Deck course takes place after SWO candidates complete the Basic Division Officer Course and before they begin their first at-sea assignment. SWOs taking this course are required to complete 158 hours of classroom and simulator training designed to increase their ship-driving skills by exposing them to a variety of scenarios involving different maneuvers, and varying sea and weather conditions. The Navy plans to expand this course into a 6 week ship-driving training course (Officer of the Deck Phase I), scheduled to begin in June 2021. Officer of the Deck Phase I course. According to Commander, Naval Surface Forces documentation, the Junior Officer of the Deck course will expand into the Officer of the Deck Phase I course. Officer of the Deck Phase I is under development and will be 6 weeks long (an additional 2 weeks longer than Junior Officer of the Deck), and will take place after SWO candidates complete the Basic Division Officer Course and before they begin their first at-sea assignment. Officer of the Deck Phase I is intended to build on the Junior Officer of the Deck curriculum by increasing the required number of ship-driving training hours from 158 to 241, and expanding the course content to include instruction on more advanced radar navigation techniques. Surface Warfare Officers School 16 According to Navy officials, on July 17, 2019, Surface Warfare Officer School Command graduated the first 30 students from the Junior Officer of the Deck course in San Diego, California and 18 students from Newport, Rhode Island. Page 12 GAO-20-154 Navy Readiness Command and Commander, Naval Surface Forces officials expect the Officer of the Deck Phase I course to begin in June 2021. Officer of the Deck Phase II course. According to Navy documentation, the Officer of the Deck Phase II course that is under development will be 3 weeks long, and will take place after SWOs have completed their first at-sea assignment and before they attend the Advanced Division Officer Course. This course is intended to continue the development of shipdriving skills through an additional 102 hours of required classroom and simulator training. Surface Warfare Officers School Command and Commander, Naval Surface Forces officials stated that Officer of the Deck Phase II course could begin as early as June 2021. Mariner Skills Training Centers. According to Commander, Naval Surface Forces and Surface Forces documentation, Surface Warfare Officers School Command will provide the Officer of the Deck Phase I and Phase II courses at the Mariner Skills Training Centers—new simulator-based facilities expedited after the 2017 collisions. 17 Officials from the Office of the Chief of Naval Operations stated that these facilities—including upgraded simulators, the instructors, classrooms, and the curriculum development for the Officer of the Deck Phase I and Phase II courses—will cost approximately $467.5 million through fiscal year 2025. According to Navy officials, construction on the • San Diego, California Mariner Skills Training Center will begin in early fiscal year 2020 and will be complete by June 2021 and • Norfolk, Virginia Mariner Skills Training Center will begin in fiscal year 2021 and will be complete in January 2023. 18 The Mariner Skills Training Program is based upon the Littoral Combat Ship ship-driving training program, which according to the Navy, provides a balance of classroom, simulation, and shipboard experience. 19 17 Commander, Naval Surface Forces, Report to Congress—Review of Surface Warfare Officer Initial Training, (September 2018). 18 Navy officials stated that while the Norfolk, Virginia Mariner Skills Training Center is not expected to be complete until January 2023, they are planning on establishing temporary facilities in 2021 to conduct the training until the facilities are complete. 19 The Littoral Combat Ship is a Navy surface ship that utilizes rotational crewing, with multiple crews assigned to each ship. Most Littoral Combat Ship driving training is conducted off the ship in a classroom or simulator setting as operational demands do not allow sufficient time for training during operational periods. Page 13 GAO-20-154 Navy Readiness According to Navy officials, since Littoral Combat Ship SWOs serve in rotating crews and have less opportunity to train aboard their ships, the Navy developed the Littoral Combat Ship Training Facility to support SWOs’ training ashore (see fig. 4). The foundation of the Littoral Combat Ship ship-driving program is repetitive training in sophisticated simulators to build ship-driving proficiency. According to the Navy, the effectiveness of this training has been validated over the last 10 years by the superior ship-driving proficiency of Littoral Combat Ship officers during at-sea operations and assessment performance when compared with nonLittoral Combat Ship officers, in many cases. Figure 4: Littoral Combat Ship Training Facility in San Diego, California Page 14 GAO-20-154 Navy Readiness The Navy is Implementing Additional Skill Checks but Has Not Taken Other Actions Necessary to Evaluate the Effectiveness of Changes to SWO Ship-Driving Training The Navy has relied on a series of added skill checks throughout a SWO’s career to help validate that SWOs have necessary ship-driving and other skills, but has not developed key processes and assessments to evaluate the overall effectiveness of its existing and planned training programs. The Navy is Implementing Additional Skill Checks to Be Conducted throughout SWO Careers The Navy is implementing a series of ten skill checks on ship-driving and other mariner tasks at various career points—for example, before a SWO begins leading a ship department and before the SWO takes command of a ship. The Commander, Naval Surface Forces, issued an instruction in September 2018 detailing ten skill checks to be conducted over the course of a SWO’s career to periodically gauge SWOs’ ship-driving skills. 20 These checks, summarized in appendix IV, are to occur at standardized points in a SWO’s career, either during training or at the beginning or conclusion of certain at-sea assignments. Four of the ten checks were already in place at the time the instruction was issued in September 2018, with a preliminary version of a fifth check also in place. According to Navy documentation, three more of the ten checks had also been implemented as of August 2019, and Navy guidance states that the remaining checks are scheduled to be in place by 2021 or earlier. Navy officials stated they were making these checks more rigorous. For example, according to Navy officials, previously Department Heads were allowed to retake the Command Qualification Assessment ship-handling test as many times as they needed to pass the assessment. According to these officials, in 2018 Surface Warfare Officers School Command allowed only three chances to take the test, leading to five of the 256 20 Commander, Naval Surface Forces Instruction 1412.5, Surface Warfare Officer Milestone Mariner Skills Assessments, Evaluations, and Competency Checks (Sept. 21, 2018). Page 15 GAO-20-154 Navy Readiness Department Heads assessed in 2018 to be disqualified from advancing beyond the role of Department Head. Navy officials report that these skill checks are intended to enhance the development and sustainment of ship-driving proficiency across a SWO’s career and to ensure that the changes in training are resulting in competent SWOs at each level of their careers—essentially that SWOs have the skills required to perform their duties. Surface Warfare Officers School Command will administer checks during SWO training on shipdriving to better evaluate individual proficiency and target remediation for those whose performance presents significant concerns. Ship Commanding Officers will also observe and evaluate SWOs on a series of ship-driving scenarios before the completion of their first Division Officer assignment and later as a Department Head to certify that they are prepared for more advanced ship-driving training and responsibilities. The Navy Has Not Put Key Processes and Assessments in Place to Evaluate the Effectiveness of Changes to SWO Training While the planned skill checks are designed to help ensure that SWOs have the skills required to perform their duties, senior Navy officials stated that it could take 16 years or more to know if the planned changes to SWO training were effective in increasing Commanding Officer shipdriving proficiency across the fleet. These officials stated that they intend to closely monitor the implementation of changes to the training; however, we found a number of interrelated challenges that limit the Navy’s ability to determine in the near term if the significant investments it is making to expand and enhance SWO ship-driving training are effective. Specifically and described in detail below, in planning an approach for evaluating its efforts, the Navy has not (1) solicited fleet-wide feedback on the quality of the increased ship-driving training, (2) planned to routinely conduct shipdriving competency assessments, (3) provided standard criteria for qualifying Officer of the Deck candidates, and (4) determined how to analyze and use information from logbooks that SWOs are required to complete. The Navy’s Comprehensive Review of Recent Surface Force Incidents—one of the internal reviews completed after the 2017 mishaps—notes the importance of assessing and monitoring performance so that corrective actions can take place. 21 In addition, federal government internal control standards state that management should use quality information and monitoring activities to ensure the agency’s 21 U.S. Navy, Comprehensive Review of Recent Surface Force Incidents (Oct. 26, 2017). Page 16 GAO-20-154 Navy Readiness objectives are achieved. 22 Moreover, our prior work on assessing training efforts in the federal government states that an agency should evaluate the effectiveness of its training and development efforts, to include obtaining feedback, assessing competency, and analyzing relevant data. 23 The Navy Has Not Solicited Fleet-Wide Feedback on the Quality of Increased ShipDriving Training We found that while the Navy collects feedback from certain groups of SWOs, it did not have a formal fleet-wide process to solicit feedback from SWOs on the quality of the increased amount of ship-driving training or to gauge the health of the SWO community. In group discussions we held as part of our review, SWOs identified challenges that Division Officers experience in applying classroom and simulator training to their duties. 24 According to SWOs in 19 of 24 group discussions with Department Heads and Division Officers, Division Officers have challenges in applying the ship-driving training they receive, due to factors such as differences between training curriculum and actual duties, extended lengths of time elapsed between training and application, varying ship-driving opportunities during Division Officer assignments, and difficulty retaining the large volume of course material. 25 SWOs that participated in our discussion groups and interviews identified positive aspects of ship-driving training, as well as concerns about training material. • During five of 12 ship group discussions with Division Officers, those Division Officers that had taken the Basic Division Officer Course identified positive aspects of the training such as valuable practical exercises and simulator time. However, SWOs in all 12 Division Officer group discussions also identified challenges related to this 22 GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: September 2014). 23 GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO-04-546G (Washington, D.C.: March 2004). 24 We held 24 group discussions aboard 12 surface ships with SWOs at two levels— Division Officers and Department Heads—and 12 interviews with Executive and Commanding Officers. In these visits we met with approximately 225 SWOs and discussed the quality of SWO initial training and how the Navy had implemented and assessed changes to training. 25 Due to the timing of our work, the interviews and group discussions did not include SWOs that experienced changes made or planned for SWO training beyond April 2019. Page 17 GAO-20-154 Navy Readiness training, such as the information covered in training being too broad, and a lack of connection to actual duties on their ship. • More experienced Division Officers in four of 12 Division Officer group discussions identified challenges related to the Advanced Division Officer Course, such as insufficient time in ship-driving simulators, and too much time spent covering material that Division Officers were already expected to learn during their first at-sea assignment. • Commanding Officers and Executive Officers in seven of 12 interviews, and Department Heads in four of 12 group discussions likewise identified positive aspects of the Basic Division Officer Course, such as improved knowledge of ship operations for Divisions Officers that recently completed the course. However, Commanding Officers and Executive Officers in three of 12 interviews and Department Heads in seven of 12 group discussions identified challenges with the course, including areas where they had to compensate with on-the-job training for skills they felt should have been addressed in initial training, such as ship-driving proficiency in high-traffic environments. Our prior work on assessing training efforts in the federal government states that an agency should evaluate the effectiveness of its training and development efforts, to include obtaining and analyzing feedback. 26 However, the Navy does not currently have a formal fleet-wide method of soliciting feedback from SWOs to obtain input on the quality of their classroom, simulator, and at-sea training on Division Officer performance and evaluate trends in feedback, and instead uses more limited means to assess training. For example: • According to Navy officials, Surface Warfare Officers School Command conducts end-of-class surveys at the end of officer training, but no follow-up is conducted by the command after SWOs have assumed their ship duties or to obtain input from the trainees’ superior officers on the value of the training. • The Navy had a survey for Division Officers and Department Heads in the past, but this survey gave little helpful feedback on training and, 26 GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO-04-546G (Washington, D.C.: March 2004). Page 18 GAO-20-154 Navy Readiness according to Navy officials, the Navy discontinued the survey after 2015. 27 • Surface Warfare Officers School Command assembles a board of officers from the fleet each year to review areas of its training curriculum, but Navy officials stated that participants are invited based on their expertise. As a result, only those selected to serve on the board (not officers across the fleet) have the opportunity to provide feedback. 28 The Navy’s current means to assess training do not allow for the full range of junior and senior officers across the fleet to provide feedback on how well training prepares SWOs for their ship duties. Senior Navy officials acknowledged the value of conducting fleet-wide surveys of SWOs to obtain feedback on how to improve SWO training and gauge the health and morale of the SWO community. SWOs’ experiences in the fleet are diverse, therefore fleet-wide data is of particular value as centralized organizations like Naval Surface Forces, Surface Warfare Officers School Command, and the Office of the Chief of Naval Operations consider costly and consequential training investments. Without a method to regularly collect and analyze information from SWOs across the fleet, such as in a survey, regarding the quality of the increased classroom, simulator, and at-sea training on Division Officer performance, and evaluate trends in feedback received, Navy decision makers lack valuable information that could help them to assess the effects of training on SWO performance. 27 According to Navy Personnel Command officials, the command had conducted fleetwide annual surveys of Division Officers and Department Heads but discontinued these surveys after 2015. Officials from Navy Personnel Command stated that the survey results showed little change over time and they decided to discontinue the survey to conserve resources. These surveys covered a wide range of issues in a SWO’s career, such as professional aspirations and morale, with training comprising a minority of survey material. For example, none of the open-ended questions on the 2015 survey related to training, so the prompted opportunities to provide feedback on training in these surveys were limited to structured responses, providing limited value in identifying means to improve training. 28 According to an official from the office of Commander, Naval Surface Forces, the Navy also solicits feedback from groups of officers through Surface Warfare Officers School Command visits to officers stationed in fleet concentration areas, and through semiannual symposiums of ship Commanding Officers. Page 19 GAO-20-154 Navy Readiness The Navy Developed a ShipDriving Competency Assessment Conducted by Independent Inspectors, but Has Not Fully Planned to Routinely Conduct the Assessments Navy Surface Warfare Officer School Command training experts developed a ship-driving proficiency measurement system and used it in fiscal year 2018 to conduct ship-driving competency assessments. Specifically, from January through March 2018 Surface Warfare Officers School Command conducted “spot check” ship-driving competency assessments of 164 SWOs that had recently qualified as Officers of the Deck during their first at-sea assignment. Each assessment was conducted by three Navy inspectors that were independent of the assessed SWOs’ chain of command. The independent Navy inspectors found concerns in the ship-driving competency levels of more than 80 percent of these SWOs (see fig. 5). Specifically, Surface Warfare Officers School Command found that 29 SWOs (18 percent) had significant competency problems and 108 had some concerns (66 percent). According to Surface Warfare Officers School Command officials, those SWOs who experienced significant problems in their assessments likely should not have been qualified as Officer of the Deck at the time of the assessment because they violated fundamental ship-driving rules, among other issues. Navy guidance to the fleet emphasizes that these assessments performed by independent experts are valuable in supporting impartial results and providing quality information for analysis. 29 Figure 5: Officer of the Deck Competency Assessment Results, January–March 2018 29 Surface Warfare Officers School Command, “Fleet Officer of the Deck Competency Check Concept of Operations.” Page 20 GAO-20-154 Navy Readiness According to Navy documentation, the Navy also used the 2018 competency assessments to help validate its new Junior Officer of the Deck and Officer of the Deck training curriculum. Specifically, Surface Warfare Officers School Command used the same Officer of the Deck competency assessment criteria to assess six officers in May 2018 and 12 in July 2018 that completed a pilot version of the Junior Officer of the Deck course. Surface Warfare Officer School Command found that the students with no at-sea experience that had completed a pilot of the new Junior Officer of the Deck training course in some cases outperformed qualified Officers of the Deck that had over a year of at-sea experience. 30 According to Navy officials, the ability to compare ship-driving proficiency among populations and with earlier baselines using these competency assessments was valuable to the Navy in identifying the effects of changes to training, and could also be valuable in the future, as well. However, when we visited Surface Warfare Officers School Command in February 2019, officials told us they did not plan to conduct additional competency assessments until 2020. 31 In meetings with Surface Warfare Officers School Command and senior Navy leaders, we noted that delaying additional assessments could limit visibility over ship driving proficiency trends and that small sample sizes could affect the Navy’s ability to make comparisons over time. In response, the Navy accelerated and expanded additional competency assessments. According to Navy officials, in spring 2019, Surface Warfare Officers School Command began to assess a sample of Division Officers using the Officer of the Deck competency assessment at the beginning of each Advanced Division Officer Course to collect and analyze performance data and refine training curriculum. Further, as of July 2019, the Navy had assessed 38 SWOs from three courses and found that the proficiency 30 The competency assessment system collects demographic data from participants to help link trends in experience to performance and knowledge demonstrated in the event, allowing for quantitative analysis of competency assessment results and comparison among groups and over time. 31 According to Navy officials, the Navy plans to conduct Officer of the Deck competency assessments across all world-wide home ports in 2020, as was carried out from January to March 2018. Page 21 GAO-20-154 Navy Readiness level of the SWOs assessed had not improved from the proficiency levels seen in the 2018 assessments. 32 Senior Navy officials we met with as part of this review stated that they recognize the value in implementing periodic ship-driving competency assessments by independent inspectors to identify trends in ship-driving proficiency over time. However, we also found that the Navy has not planned to routinely conduct these assessments in the future. Specifically, in July 2019, Navy officials stated that they do not plan to complete these Officer of the Deck competency assessments beyond 2021 and plan to replace them with a different assessment at the end of the planned Officer of the Deck Phase II course. However, our analysis shows that mid-fiscal year 2024 is the first time Officer of the Deck Phase I course graduates will have completed their first at-sea assignment and be available to have their ship-driving training assessed, resulting in a multi-year gap in planned competency assessments. In order to measure the effectiveness of the full complement of Navy’s new and enhanced ship-driving training, the independent Navy inspectors will need to continue administering the Officer of the Deck competency assessments beyond 2021. In addition, an assessment performed at the end of training, such as the planned Officer of the Deck Phase II assessment, indicates the SWOs’ proficiency after additional training and may give a less accurate indication of prior at-sea proficiency. According to federal government internal control standards, management should use quality information and monitoring activities to ensure the entity’s objectives are achieved. 33 Moreover, our prior work on assessing training efforts in the federal government states that an agency should evaluate the effectiveness of its training and development efforts, to include assessing 32 Navy officials stated that the 38 SWOs that were assessed in 2019 had not benefitted from the new and expanded ship-driving training courses—since they attended the Basic Division Officer Course in 2017 before Surface Warfare Officers School Command had adjusted the training curriculum. According to Navy officials, the only change that these SWOs had experienced was an increased awareness of the importance of safe shipdriving. Navy officials stated that they believe that once SWOs receive the new and expanded ship-driving training they will demonstrate higher proficiency levels. However, this will not occur until at least mid-fiscal year 2022 when graduates of the recently implemented Junior Officer of the Deck course complete their first 30-month Division Officer at-sea assignment, attend their Advance Division Officer Course, and are assessed. 33 GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: September 2014). Page 22 GAO-20-154 Navy Readiness competency and analyzing relevant data. 34 Without routinely conducting Officer of the Deck competency assessments across the fleet using samples of sufficient size and selection methods, the Navy will be hindered in its ability to gauge fleet-wide ship driving proficiency trends and determine the effectiveness of the changes made to training, and the Navy may not know whether additional changes are needed. The Navy Lacks Standard Criteria for Informing the Qualification of Officer of the Deck Candidates We found that the Navy has not provided standard criteria to ship Commanding Officers on fleet-wide ship driving proficiency expectations to inform the qualification of Officer of the Deck candidates. Instead, the Navy has determined that ship Commanding Officers should use their individual judgment in granting this qualification based on a set of required officer experiences, which the Navy refers to as Personnel Qualification Standards. Following the 2017 collisions, Surface Warfare Officers School Command developed proficiency standards to measure and test Officer of the Deck ship-driving proficiency to implement the Officer of the Deck competency assessments described above. The proficiency standards require an Officer of the Deck to demonstrate knowledge of navigation systems, rules of the road, and effective bridge resource management and to demonstrate the ability to successfully navigate high-traffic environments. However, the varying at-sea experiences of officers and subjective nature of some requirements have led to different experiences for SWO candidates working to qualify as Officer of the Deck. SWOs must complete a standard series of requirements in ship-driving and other experience before they are eligible to qualify as Officer of the Deck, with Commanding Officers granting qualification after their assessment of the SWO’s performance and fitness. However, when we held group discussions with SWOs on ships in the fleet, SWOs in nine of 12 group discussions with Division Officers, eight of 12 group discussions with Department Heads, and three of 12 interviews with Commanding Officers and Executive Officers identified significant differences in opportunities, experiences, and assessments that Division Officers experience in earning their qualification as Officers of the Deck during their first Division Officer assignment. 35 For example: 34 GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO-04-546G (Washington, D.C.: March 2004). 35 Due to the timing of our work, the interviews and group discussions did not include SWOs that experienced changes made or planned for SWO training beyond April 2019. Page 23 GAO-20-154 Navy Readiness • In one group discussion, Division Officers reported being qualified as Officers of the Deck without ever having stood watch at sea, with the Commanding Officer granting qualifications based on their classroom and simulator experience alone. • In five of 12 group discussions with Division Officers, Division Officers stated that SWOs on ships in maintenance had few opportunities to stand watch on the bridge at sea to build proficiency in difficult shipdriving operations, but still received their qualifications. • SWOs in 17 of 24 group discussions stated that some Division Officers get more ship driving experience than others before earning their Officer of the Deck qualifications. For example, Division Officers assigned to ships with more time at sea or fewer Division Officers get more experience to practice ship driving than those on ships with little time at sea or that must divide ship-driving opportunities among numerous Division Officers. • Commanding Officers in three of 12 interviews reported that they had to temporarily place their Division Officers on other ships to gain qualifying experience, and had to rely on the judgment of the other ships’ Commanding Officers in determining their qualifications as Officers of the Deck. 36 According to Navy officials, the Navy has not provided Officer of the Deck assessment criteria based on the developed proficiency standards to ship Commanding Officers, out of deference to their judgment in interpreting an officer’s preparedness to drive their ship. Navy officials emphasized the importance of allowing ship Commanding Officers to make their own determination of an officer’s preparedness to drive a ship, due to their knowledge of the ship’s operating conditions. Navy officials also stated that they considered the Officer of the Deck assessment standards to be a resource for use by Surface Warfare Officer Schools Command in assessing training curriculum and had not considered using the standards in the fleet for other purposes. However, the Navy’s 2018 and 2019 Officer of the Deck competency assessments identified significant variance in the ship-driving competency levels of recently qualified Officers of the Deck. 36 According to Navy officials, the surface fleet is implementing new staffing practices for first at-sea Division Officer assignments under which the duration of planned ship maintenance periods are considered. Navy officials stated that the surface fleet intends to assign first at-sea Division Officers to ships that will be operational for 15 of their first 18 months. Page 24 GAO-20-154 Navy Readiness Since the Navy has developed fleet-wide standards for assessing Officer of the Deck proficiency, the Navy could use these to provide standard Officer of the Deck assessment criteria in guidance to ship Commanding Officers. Federal government internal control standards state that management should internally communicate the necessary quality information to achieve the entity’s objectives and ensure decisions are made based on consistent standards. 37 A SWO’s assigned ship, Commanding Officer, and operating conditions may change during a career, so a standard set of criteria would help Commanding Officers to determine what is expected of Officers of the Deck elsewhere in the fleet as they determine a junior officer’s qualification. Without providing standard Officer of the Deck assessment criteria and incorporating them into surface fleet guidance to Commanding Officers, the Navy risks creating uncertainty in Officer of the Deck qualification expectations— which can contribute to variations in ship-driving proficiency among SWOs that could jeopardize safe operations at sea. The Navy Has Not Determined How to Analyze and Use Surface Warfare Mariner Skills Logbook Data In September 2018, the Commander, Naval Surface Forces, U.S. Pacific Fleet and Commander, Naval Surface Forces Atlantic, began requiring SWOs to document their ship-driving and related experience in a handwritten logbook. The logbook—referred to as the Surface Warfare Mariner Skills Logbook (see fig. 6)—captures an officer’s experience gained during each watch aboard a ship, special evolution (e.g., underway replenishment, flight operations, and sea and anchor duty), and simulator training session. 38 37 GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: September 2014). 38 For each watch, special evolution, and simulator training session they participate in, SWOs are generally required to record in their logbooks the date, amount of time spent, location, complexity of scenario or situation, and any other notes they deem pertinent. Page 25 GAO-20-154 Navy Readiness Figure 6: Surface Warfare Mariner Skills Logbook During our ship discussion groups, SWOs at the Division Officer and Department Head levels reported that they had begun filling out their logbooks and having them reviewed as required, but some acknowledged that they are inconsistently filling them out or that they were not entering any information in them. 39 Specifically, SWOs in five of 24 discussion groups reported that logbooks are completed with inconsistent quality or not completed at all. Additionally, SWOs in four of 24 discussion groups reported that they are unaware of any plans to use the logbook information to identify any additional training needs and provide opportunities for SWOs to improve their ship-driving proficiency. Navy Personnel Command officials told us that, as of July 2019, they had received 174 summaries of Surface Warfare Mariner Skills Logbook data 39 Surface Warfare Officers School Command issues a logbook to SWOs when they begin the Basic Division Officer Course. Commander Naval Surface Forces Pacific and Commander Naval Surface Forces Atlantic Instruction 1412.9, Surface Warfare Mariner Skills Logbook Requirements was published on September 6, 2018 and our discussion groups were conducted from January through April 2019. Page 26 GAO-20-154 Navy Readiness from Commanding Officers. Navy officials stated that over time, as they gather these data, they intend to examine the link between ship-driving proficiency and SWO experience. However, officials did not have any specific, measurable plans to analyze and use these data or to assess the completeness of these data. Federal internal control standards state that management should obtain relevant data from reliable sources and process those data into quality information to aid decision-making. 40 Furthermore, Naval Surface Forces guidance states that the surface warfare community should analyze and use logbook data to link SWO experience with ship-driving proficiency. 41 Despite this guidance, the Navy does not yet have a plan that includes specific steps to analyze and use logbook information to link SWO experience with ship-driving proficiency. According to senior Navy officials, while the Surface Warfare Mariner Skills Logbook is still relatively new, developing a plan to use the information would be a logical next step. Without a plan for analyzing and using Surface Warfare Mariner Skills Logbook data, the Navy cannot determine the relationship between SWO experience and ship-driving proficiency or use these data to aid decision-making. Conclusions SWOs play a critical role in Navy surface fleet readiness, as they are responsible for safely driving ships at sea and successfully leading ships in Navy operations across the world. The Navy is making numerous changes and investments to enhance Surface Warfare Officer shipdriving training following the 2017 collisions at sea—with plans to triple initial training hours and spend nearly half a billion dollars to build simulator capacity to deliver this training. The Navy’s oversight of these efforts is centered on a series of added checks throughout SWOs’ careers to ensure that they have basic ship-driving and other skills. These checks are steps in the right direction but may not provide adequate assessment mechanisms in the near term and might lead to missed opportunities going forward. For example, the Navy is expanding its shipdriving training but is not planning to collect fleet-wide feedback on classroom, simulator, and at-sea training received. In addition, the Navy 40 GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: September 2014). 41 COMNAVSURFPAC/COMNAVSURFLANTINST 1412.9, Surface Warfare Mariner Skills Logbook Requirements (Sept. 6, 2018). Page 27 GAO-20-154 Navy Readiness developed standards for conducting spot checks on ship-driving competency but is planning to stop those checks in 2021, missing an opportunity for an outside assessment and to evaluate how well new and updated training is working. Moreover, ship commanders are expected to qualify SWOs on ship driving but have not been provided standard guidance for how to do this, which can contribute to wide variations in SWO competence. Finally, the Navy has developed detailed logbooks for SWOs to track their experiences but the Navy has not developed a specific plan to analyze and use the logbook data. Without actions to address these challenges, the Navy cannot fully assess in the near term if the significant investments it is making to expand and enhance SWO ship-driving training are effective; further adjustments are necessary; and, ultimately, Navy ships are being operated safely at sea. Recommendations for Executive Action We are making the following four recommendations to the Department of Navy: We recommend that the Secretary of the Navy ensure that the Commander, Naval Surface Forces, in coordination with Surface Warfare Officers School Command, develop a method to regularly collect feedback from SWOs across the fleet, such as in a survey, regarding the quality of their classroom, simulator, and at-sea training on Division Officer performance; and evaluates trends in the feedback received for the purpose of improving SWO training. (Recommendation 1) We recommend that the Secretary of the Navy ensure that the Commander, Naval Surface Forces, routinely conduct regular Officer of the Deck competency assessments using samples of sufficient size and using selection methods to gauge the level of fleet-wide ship-driving proficiency trends following the implementation of the planned shipdriving training programs. (Recommendation 2) We recommend that the Secretary of the Navy ensure that the Commander, Naval Surface Forces, in coordination with Surface Warfare Officers School Command, provide Commanding Officers with standard criteria to inform their evaluation of candidates for their Officer of the Deck qualification and incorporates these criteria into surface fleet guidance. (Recommendation 3) We recommend that the Secretary of the Navy ensure that the Commander, Naval Surface Forces, in coordination with Surface Warfare Officers School Command, develop a plan to analyze and use Mariner Page 28 GAO-20-154 Navy Readiness Skills Logbook information to inform decision-making. (Recommendation 4) Agency Comments and Our Evaluation We provided a draft of this report to DOD for review and comment. In written comments provided by the Navy through DOD (reprinted in their entirety in appendix V), the Navy concurred with all four of our recommendations and identified actions it plans to take to evaluate the effectiveness of changes to SWO training. The Navy also provided additional information and context in its comments and provided technical comments, which we incorporated as appropriate. The Navy concurred with our first recommendation that the Commander, Naval Surface Forces, in coordination with Surface Warfare Officers School Command, develop a method to regularly collect feedback from SWOs across the fleet, such as in a survey, regarding the quality of their classroom, simulator, and at-sea training on Division Officer performance; and evaluate trends in the feedback received for the purpose of improving SWO training. The Navy stated that it plans to explore additional means of garnering holistic SWO feedback regarding newly-implemented SWO training and assessments as well as gathering additional targeted feedback. However, the Navy stated that the use of performance data will remain the primary focus of surface force training improvement efforts. While using performance data is valuable, it will be important that the Navy follow through to develop a holistic means of collecting feedback, such as in a survey of SWOs across the fleet, on the effectiveness of Division Officer training on SWO performance to ensure a variety of perspectives are considered. In its comments, the Navy noted that the SWOs who participated in our ship visits and discussion groups had not experienced the changes made or planned to SWO training. We agree that our ship visits did not include officers who had experienced the expanded Division Officer training courses, as they were first introduced to the fleet in June 2019, after we had completed the majority of our work. While our discussion groups predate the implementation of new SWO training courses, the discussion groups we conducted with over 200 SWOs reinforced the our finding that the Navy needs to develop a method to regularly collect feedback from SWOs across the fleet. Also, the Navy plans to more than triple initial training, so routinely soliciting and analyzing feedback from SWOs on Division Officer training will be needed to determine the effectiveness of the Navy’s investments in these training programs and inform the Navy’s decisions on whether further adjustments are necessary. Page 29 GAO-20-154 Navy Readiness The Navy acknowledged that only officers participating in the Surface Warfare Officers School Command’s Board of Visitors provide direct feedback on the training curriculum. 42 The Navy noted, however, that all available Surface Warfare units are invited to participate in Surface Warfare Officer School Board of Visitors events and so could provide feedback then. In addition, the Navy noted that Surface Warfare Officer School Command also solicits feedback through visits to fleet concentration areas and through semiannual symposiums of ship Commanding Officers. 43 While such targeted means of collecting feedback may provide valuable information, officers may not be able to participate due to their deployment status, position on shore duty, timing of events during other personal responsibilities, or other factors. We believe that developing a method to regularly collect feedback from SWOs across the fleet would provide decision makers with valuable information that could help them assess the effects of training on SWO performance. The Navy concurred with our second recommendation that the Commander, Naval Surface Forces, routinely conduct regular Officer of the Deck competency assessments using samples of sufficient size and using selection methods to gauge the level of fleet-wide ship-driving proficiency trends following the implementation of the planned shipdriving training programs. The Navy stated that it plans to routinely collect and analyze standardized mariner skills performance data across an officer’s career path. However, the Navy stated it will use training checks, rather than the current Officer of the Deck competency assessment, to evaluate SWO performance after 2020. This presents two problems in meeting the intent of our recommendation. First, the Navy will need to ensure that the training checks are sufficiently rigorous to assess competency. Second, the Navy will not have valid data to compare the effects of training changes on competency if it changes its assessment approach. In 2018, the Navy used the Officer of the Deck competency assessment to establish a baseline of SWO ship-driving proficiency. We found that the 42 According to the Navy, the Surface Warfare Officers School Command’s Board of Visitors is a recurring forum designed to review training curricula, ensure training effectiveness, and maintain alignment with the vision of Navy Surface Warfare leadership. 43 Navy fleet concentration areas are the locations where U.S. Navy ships are homeported and include areas such as Norfolk, Virginia; San Diego, California; Pearl Harbor, Hawaii; and Yokosuka and Sasebo, Japan. Page 30 GAO-20-154 Navy Readiness 2018 competency assessments showed significant variation in shipdriving proficiency and the 2019 follow up assessments found that competency had not improved. In currently documented plans, the Officer of the Deck Phase II check after a Division Officer’s first assignment will occur at the end of the course. Even if the Navy changes the Officer of the Deck Phase II check to occur at the beginning of training as stated in its comments, performance data from this check cannot be directly compared with the results of the current competency assessment. Differences in assessment content or difficulty, remediation attempts, and the fact that the new check may have career implications for SWOs as a go/no-go assessment may affect proficiency measurements and pass rates. Due to these factors, we believe that a comparison between the current Officer of the Deck competency assessment and the planned Officer of the Deck Phase II check or another standard should not be considered as valid means for demonstrating changes in ship-driving proficiency over time. That is, adopting a new standard may affect the Navy’s ability to determine the impact of training on ship-driving proficiency compared with the 2018 baseline results. The Navy also stated in its comments that the SWO training and assessment continuum is designed to provide training and evaluation at all career milestone levels. The planned system of additional skills checks will provide the Navy with more insight into SWO proficiency levels over the course of an officer’s career and help the Navy to understand the effects of changes to training. As we stated in the report, we believe these checks are significant steps in the right direction but may not provide adequate assessment mechanisms in the near term. The more robust Officer of the Deck competency assessments are necessary to gauge the level of fleet-wide ship-driving proficiency trends following the implementation of the planned ship-driving training programs. Further, the Navy stated in its comments that while numerous means of assessing SWO mariner skills proficiency at various milestone levels are in place, the ultimate SWO career path goal is to develop the most proficient, experienced, and confident Commanding Officers, which occurs approximately 16 years into the SWO career path. While the quality of ship Commanding Officers is a vital component of Navy readiness and capability, the majority of SWOs do not remain in the Navy long enough to advance beyond the position of Division Officer, according to Navy documentation. Similarly few advance to the position of Commanding Officer during their career as a SWO. Since Division Officers constitute over one third of the SWO workforce and by design of the SWO career path do most of the ship-driving, it is of utmost Page 31 GAO-20-154 Navy Readiness importance to build and evaluate fundamental ship-driving skills for all Division Officers to support excellence in the ship-driving proficiency across the Navy. Finally, the Navy stated in its comments that our report language implies an absence of any Officer of the Deck assessments from 2021 through 2024. We acknowledge that the Navy will conduct assessments of Officers of the Deck during this time period in line with its planned system of ten checks over a SWO’s career. However, without maintaining the current Officer of the Deck competency assessments through at least 2024, the Navy will be unable to demonstrate any proficiency improvement, compared with the 2018 baseline, resulting from its new training programs. Further, the Navy stated in its comments that it is important to clarify that the SWOs who received competency checks in 2019 had not benefitted from the new and expanded ship-driving training courses. Our ship visits did not include officers who had experienced the expanded Division Officer training courses, as they were first introduced to the fleet in June 2019. Nonetheless, it is concerning that SWO competency had not improved in the 2 years since the 2017 collisions despite the fleet-wide attention to improving ship-driving skills. The Navy concurred with our third recommendation that the Commander, Naval Surface Forces, in coordination with Surface Warfare Officers School Command, provide Commanding Officers with standard criteria to inform their evaluation of candidates for their Officer of the Deck qualification and incorporate these criteria into surface fleet guidance. However, the Navy stated that such criteria are already in place. Specifically, the Navy noted that existing Personnel Qualification Standards provide the standard evaluation criteria for the Officer of the Deck qualification. We agree that the Personnel Qualification Standards are in place, but disagree that Qualification Standards provide standard evaluation criteria. Unless the Navy provides additional guidance for Commanding Officers to measure proficiency in addition to the list of required experiences present in the Personnel Qualification Standards, the actions the Navy identified as addressing our recommendation will not meet the intent of our recommendation. The Navy’s Officer of the Deck Personnel Qualification Standards provide a list of required experiences; however, the 2018 and 2019 competency assessments indicate that these existing criteria have not resulted in high levels of proficiency among Officers of the Deck. The Navy’s Personnel Qualification Standards do not require SWOs to demonstrate a standard level of proficiency, but rather that SWOs participate in a required number Page 32 GAO-20-154 Navy Readiness of ship-driving experiences at a level determined by his or her Commanding Officer. The absence of a common proficiency standard across the Navy may contribute to inconsistency in ship-driving skills among SWOs. Since the Officer of the Deck competency assessment provides a means to measure proficiency, communicating appropriate standards in line with those used in the current assessments as qualification criteria would help ensure a common understanding of proficiency expectations. In comments, the Navy stated that for junior officers whose ships experience maintenance periods, it is an historic surface force-wide practice for Commanding Officers to temporarily assign those officers to similar ships whose operational schedule better support qualification. This practice is understandable and may contribute to SWO career development, but can lead to significant differences in opportunities, experiences, and assessments that SWOs receive during their first Division Officer assignment. For example, as noted in our report, some Commanding Officers stated because of this temporary assignment, they had to rely on the judgment of the other ships’ Commanding Officers to determine their SWOs’ qualifications as Officers of the Deck. The Navy concurred with our fourth recommendation that Commander, Naval Surface Forces, in coordination with Surface Warfare Officer School Command, develop a plan to analyze and use Mariner Skills Logbook information to inform decision-making. The Navy noted that it would comprehensively evaluate performance data relative to Mariner Skills Logbook data in order to refine mariner skill milestone performance and proficiency criteria. If Navy efforts result in a plan that includes specific and measurable steps for analyzing and using Mariner Skills Logbook data, the efforts will meet the intent of our recommendation. In its comments the Navy stated that during the time we conducted our group discussions (i.e. January through April 2019), Mariner Skills Logbooks were still being introduced to the Fleet and recording practices were still being established. While at the time of our discussion groups the Mariner Skills Logbooks were relatively new, in September 2018, the Navy issued an instruction that established guidance for the implementation and use of the logbooks. In addition, all of the SWOs we met with as part of our review had already received their Mariner Skills Logbooks. Page 33 GAO-20-154 Navy Readiness We are sending copies of this report to congressional committees, the Secretary of Defense, the Secretary of the Navy, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-3489 or pendletonj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VI. John H. Pendleton Director Defense Capabilities and Management Page 34 GAO-20-154 Navy Readiness List of Committees The Honorable James M. Inhofe Chairman The Honorable Jack Reed Ranking Member Committee on Armed Services United States Senate The Honorable Richard C. Shelby Chairman The Honorable Dick Durbin Ranking Member Subcommittee on Defense Committee on Appropriations United States Senate The Honorable Adam Smith Chairman The Honorable Mac Thornberry Ranking Member Committee on Armed Serves House of Representatives The Honorable Pete Visclosky Chairman The Honorable Ken Calvert Ranking Member Subcommittee on Defense Committee on Appropriations House of Representatives Page 35 GAO-20-154 Navy Readiness Appendix I: Overview of the Nine Navy Surface Ship Classes That Surface Warfare Officers Serve Aboard Appendix I: Overview of the Nine Navy Surface Ship Classes That Surface Warfare Officers Serve Aboard Figure 7: Nine Navy Surface Ship Classes That Surface Warfare Officers Serve Aboard Page 36 GAO-20-154 Navy Readiness Appendix I: Overview of the Nine Navy Surface Ship Classes That Surface Warfare Officers Serve Aboard Page 37 GAO-20-154 Navy Readiness Appendix I: Overview of the Nine Navy Surface Ship Classes That Surface Warfare Officers Serve Aboard Page 38 GAO-20-154 Navy Readiness Appendix II: Status of Navy Review Recommendations Related to Surface Warfare Officer Training Appendix II: Status of Navy Review Recommendations Related to Surface Warfare Officer Training Following the four 2017 mishaps at sea, the Navy completed two internal reviews on surface fleet readiness, ultimately compiling 111 recommendations for improvement. 1 The Navy established a Readiness Reform and Oversight Council under the leadership of the Vice Chief of Naval Operations to oversee implementation of these recommendations. The Readiness Reform and Oversight Council reported in February 2019 that it considered 91 of these recommendations to be implemented. 2 We reviewed the recommendations, identified 12 recommendations related to Surface Warfare Officer (SWO) initial ship-driving training, and requested the implementation status of each of these recommendations from the Commander, Naval Surface Forces. 3 The Navy considers a recommendation to be “implemented” when there is a policy in place or action has been taken to address a recommendation. The Navy considers a recommendation to be “transitioned” when the Readiness Reform and Oversight Council no longer maintains regular oversight of a recommendation and has transitioned oversight to another Navy organization. As of August 2019, the Navy considered all 12 of the recommendations related to ship-driving training as implemented with the final recommendation estimated to transition by September 30, 2019. Table 1 lists the 12 recommendations related to SWO initial ship-driving training, and our summary of the Navy’s explanation for why they are considered to be implemented. 1 U.S. Navy, Comprehensive Review of Recent Surface Force Incidents (Oct. 26, 2017) and U.S. Navy, Strategic Readiness Review 2017 (Dec. 3, 2017). 2 Vice Chief of Naval Operations Memorandum, Readiness Reform Oversight Committee, (Feb. 25, 2019). 3 To determine the recommendations’ relevance to SWO training, two analysts independently reviewed the Navy’s recommendations and made individual determinations of each recommendation’s relevance to SWO training. The two analysts compared their determinations and made a final joint determination after discussing any differences in their individual determinations of relevance. Page 39 GAO-20-154 Navy Readiness Appendix II: Status of Navy Review Recommendations Related to Surface Warfare Officer Training Table 1: Navy’s 12 Recommendations Related to Surface Warfare Officer (SWO) Initial Ship-Driving Training and Explanation, as of August 2019, for Why They Are Considered Implemented Recommendation 1 Create an objective, standardized assessment program to periodically assess individual seamanship and navigation skills over the course of a SWO’s career. Status: Implemented—June 6, 2018 Transitioned: June 24, 2018 Comments: Commander, Naval Surface Forces maintains responsibility of SWO assessment, with support from Surface Warfare Officers School Command and Naval Education and Training Command to execute assessments. Commander, Naval Surface Forces Instruction 1412.5 Surface Warfare Officer Milestone Mariner Skills Assessments, Evaluations, and Competency Checks provides guidance on the revised SWO community training assessment continuum that includes 10 career milestone checks throughout the SWO career path. Recommendation 2 Improve seamanship and navigation individual skills training for SWO candidates, SWOs, Quartermasters and Operations Specialists. Status: Implemented—May 31, 2019 Transitioned: Estimated—September 30, 2019 Comments: As the responsible learning center with contractual control of training devices and curriculum development, Naval Education and Training Command maintains responsibility for improvements to training systems. These include completed improvements such as the addition of high density traffic and emergency simulator scenarios, and planned improvements, such as the systems planned for the Mariner Skills Training Centers to support Officer of the Deck training. Recommendation 3 Improve Operational Risk Management training and education at all SWO school milestone courses. Status: Implemented—March 15, 2018 Transitioned: June 24, 2018 Comments: As the responsible learning center with contractual control of training devices and curriculum development, Naval Education and Training Command maintains responsibility for improvements to Operational Risk Management training systems. Naval Education and Training Command is working to implement improvements to Operational Risk Management course systems requested by Surface Warfare Officers School Command. Recommendation 4 Provide additional fundamentals training for officers who qualified as a SWO without initial classroom training covering Automated Radar Plotting Aid, Electronic Chart Display and Information System, and Automatic Information System. Status: Implemented—June 6, 2018 Transitioned: June 24, 2018 Comments: Commander, Naval Surface Forces maintains responsibility with support from Naval Education and Training Command and Surface Warfare Officers School Command to execute training and assessments for SWOs. As part of the changes made to the SWO career path, additional fundamentals training and assessment is provided to SWOs at the Advanced Division Officers Course; the Department Head School; the Surface Commanders course; the Major Commanders course; the Bridge Resource Management workshops; the Mariner Skills Week events; and the additional required Navigation, Seamanship, and Ship-handling Training events. The above actions and SWO milestone assessments are sufficient to mitigate any shortfalls borne by SWOs who did not receive Basic Division Officer Course classroom training. Page 40 GAO-20-154 Navy Readiness Appendix II: Status of Navy Review Recommendations Related to Surface Warfare Officer Training Recommendation 5 Update the SWO Requirements Document to capture the metrics used to evaluate seamanship and navigation skills in surface warfare seamanship safety assessments. Status: Implemented—September 30, 2018 Transitioned: October 29, 2018 Comments: Commander, Naval Surface Forces maintains responsibility with support from Naval Education and Training Command and Surface Warfare Officers School Command to update the SWO Requirements Document based upon evolving requirements for SWOs. Navigation, seamanship, and ship-handling proficiency requirements have been codified within a revised October 2018 SWO Requirements Document—Commander, Naval Surface Forces Instruction 1412.4A. The SWO Requirements Document illustrates how the SWO community develops Commanding Officers with expertise in all of the following areas: navigation, seamanship, and shiphandling; maritime warfare; engineering and material management; program management; and leadership. Recommendation 6 Incorporate fatigue, crew endurance, and stress management into appropriate career milestone SWO training and enlisted leadership courses. Status: Implemented—July 15, 2018 Transitioned: January 31, 2019 Comments: Naval Education and Training Command maintains responsibility for incorporating fatigue, crew endurance, and stress management into career milestone and leadership classes. The command has worked to help other organizations add or develop this material. For example, Surface Warfare Officers School Command added 2 weeks of this training to the Surface Commanders course, and the Naval Postgraduate School generated training materials on circadian rhythm for training, among other examples. Recommendation 7 Evaluate use of Yard Patrol Craft in all officer accession programs. The study should include the feasibility of expanding Yard Patrol Craft use, and other training methods, so that every naval officer receives core competencies as articulated in the Officer Professional Core Competencies Manual. Status: Implemented—April 16, 2018 Transitioned: October 29, 2018 Comments: Commander, Naval Surface Forces maintains responsibility for this effort with support from Naval Education and Training Command and the U.S. Naval Academy. The Navy’s evaluation of the use of Yard Patrol Craft will include 80 Naval Reserve Officers Training Corps Midshipmen receiving Yard Patrol Craft training upon officer accession in fiscal year 2019. The evaluation will further include a comparative assessment in Junior Officer of the Deck and Officer of the Deck training of those officers who received Yard Patrol Craft training against those that did not. This evaluation will help the Navy to make determinations on the best investments for training resources. Recommendation 8 Update Personnel Qualification Standards for bridge and Combat Information Center watchstations including actions to address current navigation tools, surface search radars, ship control systems, and team performance related to navigation and contact management and avoidance. Status: Implemented—September 30, 2018 Transitioned: October 29, 2018 Comments: Naval Education and Training Command maintains responsibility with support from the Center for Surface Combat Systems and Commander, Naval Surface Forces, for updating Personnel Qualification Standards. The Personnel Qualification Standards for bridge and Combat Information Center watch stations was revised with emphasis on the noted technical skills deficiencies in navigation, radars, control systems, and team performance. All navigation watch stations, including the bridge and Combat Information Center, are combined into one Personnel Qualification Standards book called Ship Control and Navigation Personnel Qualification Standards—Naval Education and Training Command Manual 43492-2K. The SWO Mariner Skills Logbook data will be used as one of several tools for updating Personnel Qualification Standards criteria during subsequent revisions. Page 41 GAO-20-154 Navy Readiness Appendix II: Status of Navy Review Recommendations Related to Surface Warfare Officer Training Recommendation 9 Revise the Surface Force Readiness Manual to define the Officer of the Deck requalification process and circumstances under which watchstanders requalify on their current platform due to configuration changes. Status: Implemented—December 31, 2017 Transitioned: December 10, 2018 Comments: Commander, Naval Surface Forces maintains responsibility for updating the Surface Forces Training and Readiness Manual. The manual was updated in November 2018, and Chapter 3 Section 3 delineates the Personnel Qualification Standards requalification process and circumstances under which requalification is required, including configuration changes to current platform and personnel assigned in Temporary Duty capacity. These requirements were promulgated immediately via Surface Forces Training and Readiness Manual Advance Change Notices as part of initial Comprehensive Review efforts, and subsequently codified within the manual. Recommendation 10 Improve current seamanship and navigation team training and certifications to include assessment in high shipping density, emergency and in extremis environments. Status: Implemented—February 28, 2018 Transitioned: December 10, 2018 Comments: Commander, Naval Surface Forces maintains responsibility for seamanship and navigation team training and certifications. The Navigation Seamanship and Ship-handling Trainer and Bridge Resource Management courses have received curriculum improvements to address bridge and Combat Information Center training shortfalls with specific emphasis upon high shipping density and in-extremis maneuvering. In February 2018, Commander, Naval Surface Forces, U.S. Pacific Fleet/Commander, Naval Surface Forces Atlantic Instruction 3505.1B, Navigation Seamanship and Ship-handling Training updated training policies. Recommendation 11 Improve shore-based bridge trainers and add Combat Information Center functionality to team training facilities. Status: Implemented—December 10, 2018 Transitioned: July 31, 2019 Comments: Commander, Naval Surface Forces maintains responsibility for Navigation, Seamanship, and Ship-handling Training simulator systems. These systems are undergoing a three phase upgrade process. The first, to provide basic bridge and Combat Information Center integration was completed in July 2019. The second phase will include the construction of Mariner Skills Training Centers in Norfolk, VA, and San Diego, CA, with the full integration of the bridge and Combat Information Center in simulators. The third phase will include an upgrade to the fidelity of the simulators and further bridge and Combat Information Center training capability. The full project is expected to be complete by the end of 2022. Recommendation 12 Improve current seamanship and navigation team training and certifications to include assessment of Bridge-Combat Information Center team performance up to and including the Commanding Officer. Status: Implemented—June 13, 2018 Transitioned: January 31, 2019 Comments: Commander, Naval Surface Forces maintains responsibility for navigation and seamanship training and certification. Within the new Surface Forces Training and Readiness Manual, seamanship and navigation training, certification, and sustainment criteria have been updated and combined in order to codify the need for Bridge and Combat Information Center integration. The Navy has conducted 62 Bridge Resource Management Workshops as of August 2019 to provide additional training and mentoring regarding effective operations, Bridge Resource Management, and Bridge and Combat Information Center integration. Source: GAO analysis of U.S. Navy Information. I GAO-20-154 Page 42 GAO-20-154 Navy Readiness Appendix III: Scope and Methodology Appendix III: Scope and Methodology The John S. McCain National Defense Authorization Act for Fiscal Year 2019 and Senate Armed Services Committee report 115-262 to accompany a bill for the National Defense Authorization Act for Fiscal Year 2019 contained provisions that we review Surface Warfare Officer (SWO) training and career paths. This report (1) describes the changes the Navy has made or planned to SWO ship-driving training since the 2017 collisions and (2) assesses the extent to which the Navy has taken actions to evaluate the effectiveness of those changes. We plan to issue a separate report on SWO career paths in the future. For objective one, we reviewed Navy documentation from Commander, Naval Surface Forces, Surface Warfare Officers School Command, and the Office of the Chief of Naval Operations on the content, purpose, cost, and status of changes made and further changes planned to ship-driving training since the 2017 collisions. We focused our analysis on changes made to SWOs’ ship-driving training at the junior officer level as the Navy prioritizes ship-driving training and ship-driving experience for junior officers, and the Navy has identified actions it is taking to address recommendations from the Navy’s two 2017 internal reviews to ensure safe operations at sea through improvements to junior officer training. 1 We analyzed planned investments from fiscal year 2018 through fiscal year 2025 for the construction of two ship-driving training facilities and the development of three ship-driving training courses, which includes the cost of purchasing new simulators, hiring new instructors, military construction, and course curriculum development. We discussed implementation plans for the 2017 internal reviews’ recommendations with the Commander, Naval Surface Forces; officials from the Surface Warfare Officers School Command; and officials from the Readiness and Reform Oversight Council, a group within the Office of the Chief of Naval Operations established to monitor the implementation of the internal reviews’ recommendations. For objective two, we reviewed Navy documentation and interviewed Navy officials on how they evaluate SWOs throughout their careers, gather and use feedback from SWOs, assess the effectiveness of SWO ship-driving training, and use available data to inform decisions regarding SWO training. Specifically, we reviewed the 1 See appendix II for a listing of Comprehensive and Strategic Readiness Review recommendations specific to SWO ship-driving training and their implementation status as of August 2019. Page 43 GAO-20-154 Navy Readiness Appendix III: Scope and Methodology • implementation of the 10 career milestone checks outlined in Naval Surface Forces Instruction 1412.5 Surface Warfare Officer Milestone Mariner Skills Assessments, Evaluations, and Competency Checks that are to be administered during a SWO’s career; • Navy’s efforts to collect feedback from the surface fleet on the quality of SWO ship-driving training and the health of the SWO community; • Navy’s 2018 Officer of the Deck competency assessment results, criteria, and plans to continue evaluating SWO ship-driving competency; • extent to which the Navy had provided standardized criteria for ships’ Commanding Officers to use when evaluating SWO’s for ship-driving qualification; and • format of SWO Mariner Skills Logbooks used to track SWO shipdriving experiences, and Navy policies regarding the logbooks. To do this we compared the Navy’s practices with relevant Navy reviews, instructions, and guidance, Standards for Internal Control in the Federal Government, and our prior work on assessing training efforts in the federal government. 2 We assessed the reliability of the results of the Navy’s 2018 Officer of the Deck competency assessments by examining them for missing values, comparing other sources that provide the same types of data to ensure consistency, and interviewing knowledgeable agency officials regarding the assessments’ accuracy and completeness. In addition, we reviewed the Navy’s internal controls for performing the assessments, such as grading criteria and use of independent inspectors to ensure quality and consistency in the information. We determined that the results of the Navy’s 2018 Officer of the Deck competency assessments were sufficiently reliable for the purposes of reporting on the number and percentage of the graded categories. In addition to meeting with Navy offices, we visited 12 surface ships in the Pacific and Atlantic fleets from January through April 2019, selected according to which ships and crews were available at each of the sites we 2 GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: September 2014) and Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO-04-546G (Washington, D.C.: March 2004). We applied GAO’s strategic training guide to assess the Navy’s efforts to evaluate its training programs—to include evaluating the effectiveness of the Navy’s training and development efforts, obtaining feedback, assessing competency, and analyzing relevant data. Page 44 GAO-20-154 Navy Readiness Appendix III: Scope and Methodology visited. Aboard the ships we held group discussions and interviews with approximately 225 SWOs to discuss their views on the sufficiency and appropriateness of SWO training. Discussion group sizes ranged from two to 20 SWOs. In conducting these group discussions, we • held 24 group discussions, with two separate discussions for each of the 12 ships—one with Department Heads and one with Division Officers; • interviewed Commanding and Executive Officers aboard each of the 12 ships, where available; and • conducted each group discussion without the group’s supervisors or subordinates present. The ship crews we visited were those the Navy identified as available to hold group discussions with us during site visits, and the results of these group discussions are not generalizable to anyone outside these groups. Due to the timing of our work, the interviews and group discussions did not include SWOs that experienced changes made or planned for SWO training beyond April 2019. We asked each group a standard set of questions to obtain their views on the following topics: • the sufficiency and appropriateness of SWO training programs in preparing SWOs for their ship responsibilities, including ship driving; • the SWO career path, including the potential benefits and drawbacks of more specialized career paths; and • any opportunities to improve the SWO community. We conducted an analysis of the discussion group responses to identify common themes and provide illustrative examples in our report. Specifically, we reviewed the responses received during discussion groups, grouped the responses by themes, and counted how many discussion groups and interviews provided similar feedback to our questions. One GAO analyst conducted this analysis, coding the information and entering it into a record of summary, and a different GAO analyst checked the information for accuracy and agreement on themes. Any initial disagreements in the coding were discussed and reconciled by the analysts. The analysts then tallied the responses to determine the Page 45 GAO-20-154 Navy Readiness Appendix III: Scope and Methodology extent to which the certain themes were covered during our discussion groups and interviews. We interviewed officials, or where appropriate, obtained documentation at the organizations listed below: Office of the Chief of Naval Operations • Director of Surface Warfare (N96) • Surface Warfare (N96) Manpower and Training • Readiness Reform and Oversight Council Commander, Naval Surface Forces, U.S. Pacific Fleet • Littoral Combat Ship Training Facility • Navigation, Seamanship, and Ship-handling Training facility • USS Ardent (MCM 12) • USS Lake Champlain (CG 57) • USS New Orleans (LPD 18) • USS Paul Hamilton (DDG 60) • USS Tulsa (LCS 16) Commander, Naval Surface Forces, Atlantic • Navigation, Seamanship, and Ship-handling Training facility • USS Bataan (LHD 5) • USS Cole (DDG 67) • USS Mahan (DDG 72) • USS Mesa Verde (LPD 19) • USS Oak Hill (LSD 51) • USS San Antonio (LPD 17) • USS San Jacinto (CG 56) Page 46 GAO-20-154 Navy Readiness Appendix III: Scope and Methodology Surface Warfare Officer School Command • Basic Division Officer Course facilities—San Diego, California and Norfolk, Virginia Navy Personnel Command • Surface Warfare Officer (PERS-41) Congressional Research Service We conducted this performance audit from November 2018 to November 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Page 47 GAO-20-154 Navy Readiness Appendix IV: Current and Planned ShipDriving Skill Checks to Be Conducted during a Surface Warfare Officer’s Career Appendix IV: Current and Planned Ship-Driving Skill Checks to Be Conducted during a Surface Warfare Officer’s Career The Commander, Naval Surface Forces, issued an instruction in September 2018 listing ten ship-driving skill checks to be conducted periodically over the course of a Surface Warfare Officer’s (SWO) career. 1 Failure to pass some of the checks can result in required remediation or disqualification from career advancement. Table 2 lists the ten current and planned checks as of August 2019, as well as information on their timing and content as described in the instruction. Table 2: Information on Current and Planned Surface Warfare Officer Ship-Driving Skill Checks, as of August 2019 Check timing Check description Assessor Implementation status Upon completion of Junior Officer of the Deck or Officer of the Deck Phase I course Written exam with 50 questions on navigation, seamanship, shiphandling (ship-driving), and rules of the road (ship-driving rules), and a 45-minute light-to-medium traffic density simulator scenario Surface Warfare Officers School Command In place Prior to completion of first Division Officer assignment Observation by the ship’s Ship’s Commanding Officer Commanding Officer of the officer’s performance in a variety of ship handling, navigation, and traffic management scenarios, either live or in simulators Upon completion of Officer of Written exam with 50 questions on the Deck Phase II course navigation, seamanship, and shiphandling and rules-of-the-road exams and a 45-minute medium traffic density simulator scenario During Department Head course Written exam with 50 questions on Surface Warfare Officers navigation, seamanship, and shipSchool Command handling and rules-of-the-road exams and a 45-minute medium-tohigh traffic density simulator scenario During first Department Head Observation by ship’s Commanding assignment at sea Officer of officer’s performance in a variety of ship-handling, navigation, and traffic management scenarios, either live or in simulators During Command Qualification Assessment Surface Warfare Officers School Command Ship’s Commanding Officer Written exam on rules of the road Surface Warfare Officers and navigation, and a 45-minute high School Command traffic density simulator scenario assessed by post-command Commander To be implemented by 2021 upon the stand-up of the Officer of the Deck Phase II course Already in place as a competency check during the Advanced Division Officer Course, and will become a go/no go assessment in 2021. In place In place In place 1 U.S. Navy, Commander, Naval Surface Forces Instruction 1412.5, Surface Warfare Officer Milestone Mariner Skills Assessments, Evaluations, and Competency Checks, (Sept. 21, 2018). Page 48 GAO-20-154 Navy Readiness Appendix IV: Current and Planned Ship-Driving Skill Checks to Be Conducted during a Surface Warfare Officer’s Career During Surface Commander’s Course Written exam with 50 questions on navigation, seamanship, and shiphandling and rules-of-the-road exams and a 45-minute high traffic density simulator scenario Surface Warfare Officers School Command To be implemented in 2019 During Executive Officer’s assignment at sea, prior to attending the Prospective Commanding Officer’s course Evaluation by the Immediate Superior in Command of the sitting Executive Officer for readiness to advance to become a Commanding Officer Immediate Superior in Command In place During the Prospective Commanding Officer’s course Written exam with 50 questions on Surface Warfare Officers navigation, seamanship, and shipSchool Command handling and rules of the road exams and a 45-minute high traffic density simulator scenario In place During Major Commander’s Course Written exam with 50 questions on navigation, seamanship, and shiphandling and rules-of-the-road exams and a 45-minute high traffic density simulator scenario In place as a go / no go assessment Surface Warfare Officers School Command Source: GAO analysis of Navy documentation. I GAO-20-154 Page 49 GAO-20-154 Navy Readiness Appendix V: Comments from the Department of Defense Appendix V: Comments from the Department of Defense This report (GAO-20-154) had report number GAO-19576 at the time it was transmitted to DOD for comment. Page 50 GAO-20-154 Navy Readiness Appendix V: Comments from the Department of Defense Recommendation 1: The GAO recommends that the Secretary of the Navy ensure that the Commander, Naval Surface Forces, in coordination with Surface Warfare Officers School Command, develop a method to regularly collect feedback from SW05 across the fleet, such as in a survey, regarding the quality of their classroom, simulator, and at-sea training on Division Officer performance; and evaluates trends in the feedback received for the purpose of improving SWO training. COMNAVSURFOR Response: CNSF concurs with the recommendation. a. Since 2013, with the inception of'the Surface Warfare command quali?cation assessment (CQA) process, the Surface Community carried out Surface Warfare Officer training improvements based upon performance data as opposed to user experience (fleet surveys, etc.) for mariner skills, maritime warfare, and other major competencies. With the establishment of the Surface Warfare Officer Mariner Skills Assessments, Evaluations, and Competencies Checks, codified within COMNAVSURFINST 1412.5 of 21 September 2018, the Surface Community?initiated a more robust program for consistently collecting standardized mariner skills performance data across an officer?s career. This data provides the basis upon which to evaluate Surface Force health with respect to mariner skills, as well as to make training adjustments in support of performance improvements. As increasing quantities of SWOs complete the training and assessment continuum requirements associated with each milestone assignment (as defined by the revised SWO Career Path implemented in June 2018), data collection/analysis will steadily increase. At present, mechanisms, such as end of training critiques as well as periodic waterfront engagements are in place for obtaining student feedback regarding the quality of classroom, simulator, and at?sea training. The Surface Warfare community will explore additional means of garnering holistic SWO feedback regarding the correlation of newly implemented SWO training/assessments to Navigation, Seamanship ShiphandlingiNSS) to effective performance across all career milestones. Currently, the means by which the Surface Warfare community obtains this feedback include: personal engagements within each Fleet Concentration Area currently conducted by SWOS leadership in advance of the annual Surface Warfare Board of Visitors; and semi-annual Type Commander Training Symposiums. While student feedback regarding the quality of their classroom, simulator, and at?sea training experience will assist the Surface Force in identifying training shortfalls, the use of performance data will remain the primary focus of Surface FOrce training improvement efforts. b. GAO noted: "According to SWOs in 19 of 24 group discussionswith Department Heads and Division Officers, Division Officers have challenges in applying the ship-driving training they receive, due to factors such as differences between training curriculum and actual duties, extended of time elapsed between training and application, varying ship-driving opportunities during Division Officer assignments, and difficulty retaining the largevvolume of course material." 1) While this is true, it should also be noted that due to timing, the interviews and group discussions did not include SWOs that experienced changes made or planned to SWO training beyond February 2019 (See Report Footnote As such, the experiences borne and expressed by these officers formed the basis upon which many attributes of the revised SWO training/assessment continuum were predicated. c. GAO noted: ?Surface Warfare Officers School Command assembles a board of officers from the fleet each year to review areas of its training curriculum, but Navy officials stated that participants are Page 51 GAO-20-1 54 Navy Readiness Appendix V: Comments from the Department of Defense invited based on their expertise. As a result, only those selected to serve on the board (not officers across the fleet) have the opportunity to provide feedback.? 1) Although officers participating in the Board are the only ones providing direct feedback on the training curriculum, all available Surface Warfare units are invited to participate in SWOS Board of Visitors events. It is also important to note the Navy also solicits feedback through Surface Warfare Officers School Command visits to all Fleet Concentration Areas in advance of the annual SWOS Board of Visitors, and through semiannual symposiums of ship Commanding Officers (See Report Footnote Recommendation 2: The GAO recommends that the Secretary of the Navy ensure that the Commander, Naval Surface Forces, routinely conducts regular Officer of the Deck competency assessments using samples of sufficient size and using selection methods to gauge the level of fleet- wide ship-driving?proficiency trends following the implementation of the planned ship-driving training programs. COMNAVSURFOR Response: CNSF concurs with the recommendation. a. The Surface Force has a robust program to routinely collect and analyze standardized mariner skills performance data across an officer?s career path. As codified within COMNAVSURFORINST 1412.5 of 21 September 2018, Surface Warfare Officer Mariner Skills Assessments, Evaluations, and Competency Checks, which outlines ten (10) career milestone assessment points, four(4) of which are Go/No-Go for continued progression toward Surface Warfare Command-Afloat. As of October 2019, nine (9) of the ten assessments are in place. To date, 1,873 officers have been assessed in Junior Officer of the Deck, Fleet Officer of the Deck, Prospective Commanding Officer, Major Commander, and Command Quali?cation assessments and competency checks. Through this process, the Surface Force anticipates evaluating 1,200 officers annually in various milestone assessments. In addition, the Surface Warfare Officers School Command has planned to conduct another series of Fleet OOD Competency Checks in early CY 2020 using samples of sufficient size and appropriate selection methods to gauge. fleet?wide ship?driving proficiency trends following the implementation of the planned ship?driving training programs. By 2021, all ten assessments within the SWO training/assessment continuum will be in place. As such, these assessments will span all career milestone cohorts, and are expected to obviate the need for additional Fleet-wide OOD Competency assessments beyond those already scheduled for 2020. As an example, by 2021, a newly commissioned Ensign, after completing the Basic Division Officer Course (BDOC) and 000 Phase I (prior to reporting to his/her first ship) and a 30-month first Division Officer tour, will be required to complete a Go/No?Go assessment as part of the COD Phase ll course. if unsuccessful on that assessment, the offiCer will not be assigned to a second Division Officer tour, and will be referred to the Probationary Officer Continuation and Redesignation Board. Similar assessments precede all other career milestone assignments, and are already in place. b. GAO noted: ?The Navy is Implementing Additional Skill Checks but Has Not Taken Other Actions Necessary to Evaluate the Effectiveness of Changes to SWO Ship-Driving Training. The Navy has relied on a series of added skill checks throughout a SWO's career to help validate that SWOs have necessary ship-driving and other skills, but has not developed key processes and assessments to evaluate the overall effectiveness of its existing and planned training programs.? 1) The Surface Warfare Officer training/assessment continuum is specifically designed to provide training and evaluation at all career milestone levels. Data analysis and training/assessment Page 52 GAO-20-1 54 Navy Readiness Appendix V: Comments from the Department of Defense continuum modifications is being conducted in conjunction with the Milestone Assessments and Fleet OOD Competency Checks described above. For Division Officer training/assesSment, key measures of effectiveness willbe Fleet performance, qualification timelines and CO feedback. IAW Recommendation 1, solicitation/lncorporation of Division Officer feedback will be enhanced to further increase training/assessment effectiveness. 2) in addition to the Milestone Assessments and Fleet OOD Competency Checks described above, all of the following provide additional means by which to evaluate the overall effectiveness of existing and planned SWO training and assessment programs: a) SWO performance during Mobility?Navigation events b) Bridge Resource Management Workshops c) Navigation, Seamanship, and Shiphandling Trainer (NSST) events d) Navigation Check-rides 3) While numerous means of assessing SWO mariner skills proficiency at various milestone levels (and thus evaluating SWO training improvements) are in place, the ultimate SWO Career Path goal is to develop the most-proficient, experienced, and confident Commanding Officers, which occurs approximately 16 years into the SWO Career Path. c. GAO noted: ?However, our analysis shows that mid?fiscal year 2024 is the first time Officer of the Deck Phase course graduates will have completed their first at?sea tour and are available to have their ship-driving training assessed, resulting in a multi-year gap in planned competency assessments. In order to measure the effeCtiveness of the full complement of Navy's new and enhanced ship-driving training, the independent Navy inspectors will need to continue administering the Officer of the Deck competency assessments beyond 2021. In addition, an assessment performed at the end of training, such as the planned Officer of the Deck Phase II assessment, indicates the proficiency after additional training and may give a less accurate indication of prior at?sea proficiency.? This verbiage implies the absence of any OOD assessment for this officer between and By contrast, the officer is evaluated by the Commanding Officer throughout this period in support of COD qualification, proficiency development, and formulating a recommendation for OOD Phase Ii training. As described above, prior to reporting to his/her first ship, every Ensign completes the Basic Division Officer Course and Junior Officer of the Deck (as of Summer 2019)/ Officer of the Deck Phase 1 Course (as of Summer 2021). After reporting to the first ship, SWO career progression is defined by: OOD qualification (approximately 12 months or more); NSS pro?ciency and experience development over the remainder of 30?month assignment; Commanding Officer recommendation for attending OOD Phase ii (Commencing in 2021); and completion of a Go/No?Go assessment upon reporting to COD Phase ll training. A pending update to Commander, Naval Surface Forces Instruction 1412.5, Surface Warfare Officer Milestone Mariner Skills Assessments, Evaluations, and Competency Checks, (Sept. 21, 2018) codi?es this Go/No?Go assessment as occurring at the ?commencement? vice ?conclusion? of COD Phase II training. cl. GAO noted: ?According to Navy officials, in spring 2019, Surface Warfare Officers School Command began to assess a sample of Division Officers using the Officer of the Deck competency assessment at the beginning of each Advanced Division Officer Course to collect and analyze performance data and refine training curriculum. Further, as ofJuIy 2019, the Navy had assessed 38 Page 53 GAO-20-1 54 Navy Readiness Appendix V: Comments from the Department of Defense SW05 from three courses and found that the proficiency level ofthe SWOs assessed had not significantly improved from the proficiency levels seen in the 2018 assessments.? 1) It is important to clarify that the 38 SWOs that were assessed in 2019 had not benefitted from the new and expanded ship-driving training courses?since they attended the Basic Division Officer Course in 2017 before Surface Warfare Officers School Command had adjustedthe training curriculum. As such, the only change that these SWOs had experienced was an increased awareness of the importance of soUnd mariner skills. Once SWOs receive the new and mariner skills training/ assessments, their proficiency levels will increase (See Report Footnote Recommendation 3: The GAO recommends that theSecretary of the Navy ensure that the Commander, Naval Surface Forces, in coordination with Surface Warfare Officers School Command, provides Commanding Officers with standard criteria to inform their evaluation of candidates for their Officer of the Deck qualification and incorporates these criteria into surface fleet guidance. COMNAVSURFOR Response: CNSF concurs with this recommendation and believes such is in place. a. The Surface Force utilizes standard criteria to Commanding Officers to inform their evaluation of candidates for their Officer of the Deck qualification. NAVEDTRA 43492-2K of 25 October 2018, Ship?s Control and Navigation, and NAVEDTRA 43101-46 of 26 July 2018, Surface Warfare Officer Combat Information Watch Officer (CICWO) and Officer of the Deck (OOD) Underway Personnel Qualification Standards (PQS) provide, in detail, the standard eValuation criteria for OOD qualification. OOD PQS provides the basis for both the fundamentals each officer is expected to understand, and the tasks each officer is expected to complete. OOD PQS is routinely reviewed and tailored in order to ensure clarity, currency, and consistency with Fleet requirements. PQS, coupled with the Commanding Officer?s personal assessment of the officer?s knowledge and the effective application thereof, form the basis for OOD qualification. As such, Commanding Officers are equipped with all necessary tools to support OOD qualification. in addition, SWO candidates are required to successfully complete such standard milestone requirements as the Basic Division Of?cer and the JOOD OOD Phase courses as part of the qualificatiOn process. OOD qualification provides the opportunity to stand the watch. Experience, confidence, and increased proficiency are subsequently cultivated over time. As increasing quantities of SWOs complete milestone assessments and competency checks, the lessons learned via the associated performance data (and any corresponding training adjustments) will be distributed throughout the Surface Force. in this manner, Surface Force DOD performance metrics (and the enhanced achievement thereof) will be readily available to all SWOs. b. GAO noted: "Commanding Officers in three of 12 interviews reported that they had to temporarily place their Division Officers on other ships to gain qualifying experience, and had to rely on the judgment of the other ships' Commanding Officers in determining their qualifications as Officers of the Deck." 1) Forjunior officers whose ships experience planned/unplanned maintenance periods, it is a historic and Surface Force-wide practice for Commanding Officers to temporarily assign those officers to similar ships whose operational schedule better support qualification. Such facilitates the opportunity to cultivate professional skills and develop experience. This practice by Commanding Officers positively promotes junior officer professional development and prevents the stagnation of critical skills. Page 54 GAO-20-1 54 Navy Readiness Appendix V: Comments from the Department of Defense Recommendation 4: The GAO recommends that the Secretary of the Navy ensure that the Commander, Naval Surface Forces, in coordination with Surface Warfare Officers School Command, develops a plan to analyze and use Mariner Skills Logbook information to inform decision-making. COM NAVSU RFOR Response: CNSF concurs with the recommendation. a. The Surface Force has already implemented plans to collect mariner skills performance data across an officer?s career, as well as to collect statistical data (number of hours of bridge watch standing, number of evolutions, etc.) through the Mariner Skills Logbook. Efforts will be enacted to comprehensively evaluate performance data relative to Mariner Skills Logbook data in order to refine mariner skill milestone performance and proficiency criteria. The Surface Force will update its current Mariner Skills Logbook data collection requirements to reflect this requirement, as well as the potential long?term use of Mariner Skills Logbook data as one of many distribution tools. b. GAO noted]: "During our shipdiscussion groups, SWOs at the Division Of?cer and Department Head levels reported that they had begun filling out their logbooks and having them reviewed as required, but some acknowledged that they, are inconsistently filling them out or that they were not entering any information in them.? 1) The Surface Force published Commander Naval Surface Forces Paci?c and Commander Naval Surface Forces Atlantic instruction 1412.9 ?Surface Warfare Mariner Ski/ls Logbook Requirements? in September 2018, and began issuing logbooks in the fall of 2018. During the time GAO conducted its group discussions (Le. January through April 2019), logbooks were still being introduced to the Fleet and recording practices were still being established (See Report Footnote Page 55 GAO-20-1 54 Navy Readiness Appendix VI: GAO Contact and Staff Acknowledgments Appendix VI: GAO Contact and Staff Acknowledgments GAO Contact John H. Pendleton, (202) 512-3489 or pendletonj@gao.gov Staff Acknowledgments: In addition to the contact named above, Chris Watson (Assistant Director), Tobin McMurdie (Analyst-in-Charge), David Beardwood, Vincent Buquicchio, Mae Jones, Amie Lesser, Shahrzad Nikoo, Michael Silver, and Brandon Voss made key contributions to this report. (103123) Page 56 GAO-20-154 Navy Readiness GAO’s Mission The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO’s website (https://www.gao.gov). Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to https://www.gao.gov and select “E-mail Updates.” Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, https://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. Visit GAO on the web at https://www.gao.gov. To Report Fraud, Waste, and Abuse in Federal Programs Contact FraudNet: Congressional Relations Orice Williams Brown, Managing Director, WilliamsO@gao.gov, (202) 512-4400, U.S. Government Accountability Office, 441 G Street NW, Room 7125, Washington, DC 20548 Public Affairs Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 Strategic Planning and External Liaison James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707 U.S. Government Accountability Office, 441 G Street NW, Room 7814, Washington, DC 20548 Website: https://www.gao.gov/fraudnet/fraudnet.htm Automated answering system: (800) 424-5454 or (202) 512-7700 Please Print on Recycled Paper.