JOINT STRIKE FIGHTER ACQUISITION REFORM: WILL IT FLY? HEARING BEFORE THE SUBCOMMITTEE ON NATIONAL SECURITY, VETERANS AFFAIRS, AND INTERNATIONAL RELATIONS OF THE COMMITTEE ON GOVERNMENT REFORM HOUSE OF REPRESENTATIVES ONE HUNDRED SIXTH CONGRESS SECOND SESSION MAY 10, 2000 Serial No. 106–202 Printed for the use of the Committee on Government Reform ( Available via the World Wide Web: http://www.gpo.gov/congress/house http://www.house.gov/reform U.S. GOVERNMENT PRINTING OFFICE WASHINGTON 70–782 DTP : 2001 For sale by the Superintendent of Documents, U.S. Government Printing Office Internet: bookstore.gpo.gov Phone: (202) 512–1800 Fax: (202) 512–2250 Mail: Stop SSOP, Washington, DC 20402–0001 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00001 Fmt 5011 Sfmt 5011 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 COMMITTEE ON GOVERNMENT REFORM DAN BURTON, Indiana, Chairman BENJAMIN A. GILMAN, New York HENRY A. WAXMAN, California CONSTANCE A. MORELLA, Maryland TOM LANTOS, California CHRISTOPHER SHAYS, Connecticut ROBERT E. WISE, JR., West Virginia MAJOR R. OWENS, New York ILEANA ROS-LEHTINEN, Florida EDOLPHUS TOWNS, New York JOHN M. MCHUGH, New York STEPHEN HORN, California PAUL E. KANJORSKI, Pennsylvania JOHN L. MICA, Florida PATSY T. MINK, Hawaii THOMAS M. DAVIS, Virginia CAROLYN B. MALONEY, New York DAVID M. MCINTOSH, Indiana ELEANOR HOLMES NORTON, Washington, MARK E. SOUDER, Indiana DC JOE SCARBOROUGH, Florida CHAKA FATTAH, Pennsylvania STEVEN C. LATOURETTE, Ohio ELIJAH E. CUMMINGS, Maryland MARSHALL ‘‘MARK’’ SANFORD, South DENNIS J. KUCINICH, Ohio Carolina ROD R. BLAGOJEVICH, Illinois BOB BARR, Georgia DANNY K. DAVIS, Illinois DAN MILLER, Florida JOHN F. TIERNEY, Massachusetts ASA HUTCHINSON, Arkansas JIM TURNER, Texas LEE TERRY, Nebraska THOMAS H. ALLEN, Maine JUDY BIGGERT, Illinois HAROLD E. FORD, JR., Tennessee JANICE D. SCHAKOWSKY, Illinois GREG WALDEN, Oregon ——— DOUG OSE, California BERNARD SANDERS, Vermont PAUL RYAN, Wisconsin (Independent) HELEN CHENOWETH-HAGE, Idaho DAVID VITTER, Louisiana DAVID SUBCOMMITTEE ON KEVIN BINGER, Staff Director DANIEL R. MOLL, Deputy Staff Director A. KASS, Deputy Counsel and Parliamentarian LISA SMITH ARAFUNE, Chief Clerk PHIL SCHILIRO, Minority Staff Director NATIONAL SECURITY, VETERANS AFFAIRS, RELATIONS AND INTERNATIONAL CHRISTOPHER SHAYS, Connecticut, Chairman MARK E. SOUDER, Indiana ROD R. BLAGOJEVICH, Illinois ILEANA ROS-LEHTINEN, Florida TOM LANTOS, California JOHN M. MCHUGH, New York ROBERT E. WISE, JR., West Virginia JOHN L. MICA, Florida JOHN F. TIERNEY, Massachusetts DAVID M. MCINTOSH, Indiana THOMAS H. ALLEN, Maine MARSHALL ‘‘MARK’’ SANFORD, South EDOLPHUS TOWNS, New York Carolina BERNARD SANDERS, Vermont LEE TERRY, Nebraska (Independent) JUDY BIGGERT, Illinois JANICE D. SCHAKOWSKY, Illinois HELEN CHENOWETH-HAGE, Idaho EX OFFICIO DAN BURTON, Indiana HENRY A. WAXMAN, California LAWRENCE J. HALLORAN, Staff Director and Counsel J. VINCENT CHASE, Chief Investigator JASON CHUNG, Clerk DAVID RAPALLO, Minority Counsel (II) VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00002 Fmt 5904 Sfmt 5904 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 CONTENTS Page Hearing held on May 10, 2000 ............................................................................... Statement of: McNaugher, Dr. Thomas L., deputy director, Arroyo Center, Rand; Rodney Larkins, business development manager, 3M Corp.; and Dr. Wesley Harris, professor, Department of Aeronautics and Astronautics .............. Rodrigues, Louis J., Director, National Security and International Affairs Division, U.S. General Accounting Office ................................................... Soloway, Stan Z., Deputy Secretary of Defense, Acquisition Reform, Department of Defense; and Major General Raymond Huot, U.S. Air Force, Acquisition Programs, Department of Defense ............................... Letters, statements, etc., submitted for the record by: Blagojevich, Hon. Rod R., a Representative in Congress from the State of Illinios, prepared statement of ................................................................ Harris, Dr. Wesley, professor, Department of Aeronautics and Astronautics, prepared statement of ........................................................................... Huot, Major General Raymond, U.S. Air Force, Acquisition Programs, Department of Defense: Information concerning TRLs ................................................................... Prepared statement of ............................................................................... Larkins, Rodney, business development manager, 3M Corp., prepared statement of ................................................................................................... McNaugher, Dr. Thomas L., deputy director, Arroyo Center, Rand, prepared statement of ........................................................................................ Rodrigues, Louis J., Director, National Security and International Affairs Division, U.S. General Accounting Office, prepared statement of ............ Shays, Hon. Christopher, a Representative in Congress from the State of Connecticut, prepared statement of ........................................................ Soloway, Stan Z., Deputy Secretary of Defense, Acquisition Reform, Department of Defense: Information concerning TRL levels .......................................................... Prepared statement of ............................................................................... 1 100 5 50 37 120 96 72 112 104 10 3 98 56 (III) VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00003 Fmt 5904 Sfmt 5904 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00004 Fmt 5904 Sfmt 5904 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 JOINT STRIKE FIGHTER ACQUISITION REFORM: WILL IT FLY? WEDNESDAY, MAY 10, 2000 HOUSE OF REPRESENTATIVES, SUBCOMMITTEE ON NATIONAL SECURITY, VETERANS AFFAIRS, AND INTERNATIONAL RELATIONS, COMMITTEE ON GOVERNMENT REFORM, Washington, DC. The subcommittee met, pursuant to notice, at 10 a.m., in room 2247, Rayburn House Office Building, Hon. Christopher Shays (chairman of the subcommittee) presiding. Present: Representatives Shays, Terry, Biggert, and Blagojevich. Staff present: Lawrence J. Halloran, staff director and counsel; J. Vincent Chase, chief investigator; Robert Newman, professional staff member; Jason M. Chung, clerk; David Rapallo, minority counsel; and Earley Green, minority assistant clerk. Mr. SHAYS. I would like to call this hearing to order, to welcome our witnesses and our guests and again to invite anyone who wants to, to take off their coats. It’s a hot room and we have asked it to be cooled down, but feel free to take off your jackets. The military procurement holiday is about to end. Defense budgets being debated today on both sides of the Capitol reflect a bicameral and bipartisan consensus on the need to modernize the aging planes, ships, weapons and equipment used to win the cold war. Today we discuss the need to modernize the acquisition systems the Department of Defense [DOD], will use to procure postcold war weapons systems. Just as the weaponry of the last century won’t win the peace in the next, the acquisition practices of the past will not efficiently or affordably meet future defense needs. Fifteen-year development cycles enshrine old technologies now rendered obsolete in 15 months. Massive cost overruns and schedule slippages are fueled in part by the launch of engineering and design work before hoped-for technologies have been refined. Extraneous, often pervasive incentives, push program officials toward artificial deadlines and premature production commitments. Various iterations of acquisition reform at DOD have attempted to address these problems and reinvigorate a hidebound acquisition culture inside and outside the Pentagon. In launching the $200 billion Joint Strike Fighter [JSF], aircraft acquisition, DOD promised the prgram would be a model of reform driven by affordability and the technical knowledge base, not by the disingenuous optimism and defense budget politics that proved so costly in the past. (1) VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00005 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 2 At the subcommittee’s request, the General Accounting Office [GAO], analyzed the JSF acquisition strategy to determine if the promise of reform is being fulfilled in practice. Their report released today finds the Joint Strike Fighter program straying from commercial best practices and knowledge-driven benchmarks. As the date approaches to select a prime JSF contractor and begin engineering on the final system concepts, DOD appears ready to abandon quantitative measures of technological maturity and revert to the business as usual of concurrent technology development and product development. GAO recommends DOD focus on risk reduction efforts by maturing critical technologies prior entering the next phase of the JSF program, even if that means delaying contractor selection and contract awards beyond the planned March 2001 date. The program should be permitted to pursue the original low-risk acquisition strategy according to GAO, without the penalty of withdrawal of funding support. DOD disagrees, claiming critical technologies will be mature enough to proceed in final design and engineering next year. As the debate unfolds, the choice should not be between a fully funded Joint Strike Fighter and a commitment to acquisition reform. We can have both. If the program succumbs to cold war acquisition habits, costs will skyrocket, the development cycle will stretch over the horizon, and the next generation fighter needed by the Air Force, the Navy, and the Marines might never fly. We welcome the testimony of all our witnesses on this important subject. [The prepared statement of Hon. Christopher Shays follows:] VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00006 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 3 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00007 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 4 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00008 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 5 Mr. SHAYS. And if I could, I would like to welcome my colleague and ask if he has any comments to make. Mr. TERRY. Thank you, no. I don’t have any opening statement. I’ll submit one in a few days for the record. I do want to compliment you on holding this hearing. When we look through the issues concerning the future of the military and the confidence that people have in our military and as we make a renewed commitment as a Congress to our military, we have got to ensure that the protocol is in place, the system is in place to ensure that we use our money wisely, that we’re looking toward the future. And when I’m speaking at veterans organizations or just simply people that are interested in my home of Omaha, NE and home of Offutt Air Force Base, that’s what they want to know. That’s why this hearing is so important today. Since it is so important, let me not continue to use the time, let’s hear from our witnesses. Mr. SHAYS. Thank you Mr. Terry. Just some housekeeping, if we could do that now. I ask unanimous consent that all members of the subcommittee be permitted to place an opening statement in the record and that the record remain open for 3 days for that purpose. Without objection, so ordered. I ask further unanimous consent that all witnesses be permitted to include their written statements in the record. Without objection, so ordered. Our first panel, we have three, is Mr. Louis Rodrigues, Director, National Security and International Affairs Division, U.S. General Accounting Office [GAO]. In fact, I think that’s referred to more as GAO than the full title. Now Mr. Rodrigues, if you could stand up, we’ll swear you in, as we swear in all our witnesses. Raise your right hand, please. [Witness sworn.] Mr. SHAYS. Note for the record our witness has responded in the affirmative. What we’re going to do is we’re going to run the clock for 5 minutes then we will flip it again for another 5. And then if you could conclude within the 10-minute period. STATEMENT OF LOUIS J. RODRIGUES, DIRECTOR, NATIONAL SECURITY AND INTERNATIONAL AFFAIRS DIVISION, U.S. GENERAL ACCOUNTING OFFICE Mr. RODRIGUES. Thank you, Mr. Chairman. Mr. Chairman, Congressman Terry, I am pleased to be here today to discuss the application of best commercial practices to DOD weapons systems in general, the Joint Strike Fighter in particular. Before getting into details, I would like to emphasize the importance of the Joint Strike Fighter decision to reforming DOD’s weapons acquisition process. As you know, the Department is in the process of rewriting its directives governing systems acquisition, referred to as DOD 5000 series. At the Department’s request, we have been participating in this effort through input to its working group. The objective of the rewrite is to bring about better, cheaper, faster outcomes in weapons programs. It is acquisition reform. Our contributions or inputs to this effort are based on our reports for the Senate Armed Serv- VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00009 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 6 ices Committee on using best commercial practices to improve weapons program outcomes. The DOD draft rewrite embodies critical features documented in our work. Two of these features are critical to the upcoming Joint Strike Fighter decision. First, the technology development must be separated from product development. That is, before entering engineering manufacturing development, we must have a match between proven technologies and requirements. And second, metrics to accurately measure technology must be used. In the 5000 series rewrite, they are adopting a measurement system we used in our Joint Strike Fighter assessment, referred to as technology readiness levels. The commitment to this knowledgebased versus the current schedule- and funding-driven process is reflected in the testimony of the Deputy Under Secretary of Defense for Acquisition Reform on March 16th before the House Government Reform Subcommittee on Government Management, Information, and Technology, ‘‘In the new systems acquisition environment, key acquisition and long-term funding commitments will not be made until technology is mature.’’ I have a lot of respect for the people in DOD who are leading acquisition reform. Philosophically we are in agreement on the best practices and the changes that are needed in the DOD environment to make such practices work on weapons systems. We at GAO are extremely encouraged by the commitment of DOD acquisition leaders to improving the weapons acquisition outcomes through the use of a knowledge-based commercial business practices. At the same time, however, we are concerned that the written directives and oral commitments will have little impact if not reflected in key decisions. In that sense, the key acquisition decision of entering engineering and manufacturing development on the Joint Strike Fighter stands out as the flagship for weapons acquisition reform. To apply anything less than the standards in the directives will send a clear message that while the instructions and rhetoric are changing, it’s business as usual. I will now briefly describe what we have learned from our best practices work and how we have applied that to assessing the Joint Strike Fighter program. Our best practices work has shown that a knowledge-based process is essential to getting better cost schedule and performance outcomes. This means that the decisionmakers must have virtual certainty about critical facets of the product under the development when needed. This knowledge can be measured in three junctures that we refer to as knowledge points, as shown in the chart to my right. Knowledge point 1 is when a match is made between the customer’s requirements and the available technology. This occurs prior to entering product development. Knowledge point 2 is when the product’s design is determined to be capable of meeting performance requirements. This occurs about midway through the product development phase. And knowledge point 3 is when the product is determined to be producible within cost, schedule, and quality targets, which occurs prior to entering production. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00010 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 7 Today I’ll focus on only knowledge point 1 because it is the biggest contributor to a successful product development, achieving subsequent knowledge points depends on it, and it is the point where JSF should be as it enters EMD. As a technology is developed, it moves from a concept to a feasible invention to a component that must fit onto a product and function as expected. In between, there are increasing levels of demonstration that can be measured. In our review of best practices for including new technology and products, we applied a scale of technology readiness levels from 1 to 9, pioneered by NASA and adapted by the Air Force Research Laboratory. Without going into the details of each level, a level 4 equates to a laboratory demonstration of technology that is not in a usable form. Imagine an advanced radio technology that can be demonstrated with components that take up a table top. When initial hand-built versions of all the radio’s basic elements are hand wired and tested together in a laboratory, the radio reaches a readiness level of 5. A technology readiness level of 7 is the demonstration of the technology that approximates its final form and occurs in an environment outside the laboratory. That same radio at level 7 would be installed and demonstrated in a platform such as an existing fighter aircraft. The lower the level of the technology at the time it is included in product development, the higher the risk that it will cause problems in product development. According to the Air Force Research Laboratory, level 7 enables the technology to be included on a product development with acceptable risk. When we asked leading commercial firms to apply these standards to their own methods of assessing technology maturity, we found that most insisted on even higher levels of readiness before they allowed a new technology into product development. Regarding the Joint Strike Fighter, in conjunction with the program office and the two competing contractors, we determined the readiness levels of critical technologies. The table to my right shows the technology readiness levels of eight critical technology areas identified by the Joint Strike Fighter program office. Let me try to explain this a little bit. What we had them do is score the technologies at three points. The blue line reflects the readiness level for each of those technology areas where they were at program launch. The yellow line reflects where they were at the time they did the scoring. And the red extension reflects where they plan to be based on what they’re planning to do between now and the down select to the engineering and manufacturing development phase. So it’s the totality of the line that shows where they plan to be when they enter EMD. In terms of engineering and manufacturing development, which is reflected by the second diamond on the right, none of the critical technology areas are projected to be at readiness level 7, which the Air Force Research Laboratory considers acceptable for entry into engineering and manufacturing development. Should any of these technologies be delayed, or worse, not available for incorporation into the final JSF design, the impact on the program would be dramatic. For example, if one of the critical technologies needed to be VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00011 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 8 replaced with its planned backup, DOD could expect an increase of about 10 percent in unit costs. The backup technology would also significantly increase aircraft weight which can negatively impact aircraft performance. This technology is projected to be at a technology readiness level of 5 at the beginning of the engineering manufacturing development phase, substantially below the criteria of 7. As noted earlier, at the policy level, DOD officials have agreed that technology development should be kept separate from product development and that technology readiness levels are a valid way to assess technology maturity. However, in response to our report on the Joint Strike Fighter, DOD balked at the use of technology readiness levels and their implications for keeping technology development out of the fighter’s engineering and manufacturing development phase. One of the reasons DOD cited for its unwillingness to accept the technology readiness levels assessed was that the levels were based on integration in the Joint Strike Fighter aircraft. On the contrary, the technology readiness levels assessed by the program office and the contractors were based on a clear understanding that a level 7 could be reached by demonstrating the technology in a relevant environment. It was further made clear that a relevant environment would include demonstrating a technology in an existing aircraft like an F–16, not a Joint Strike Fighter. There is no misunderstanding. A second reason DOD disagreed with the readiness levels assessed was that its own risk mitigation plans and judgment were more meaningful and that they showed the technology risk to be acceptable. Risk mitigation plans and judgment are necessary to managing any major development effort. However, without an underpinning such as technology readiness levels that allows transparency into program decisions, these methods allow significant technical unknowns to be judged acceptable risks because a plan exists for resolving them in the future. Experience on previous programs has shown that such methods have rarely assessed technical unknowns as a high or unacceptable risk. Consequently, they fail to guide programs to meet promised outcomes. Technology readiness levels are based on demonstrations of how well technology has actually performed. Their strength lies in the fact that they characterize knowledge that exists rather than plans to gain knowledge in the future. In conclusion, we believe that separating technology development from product development can create conditions for a successful Joint Strike Fighter program. To proceed as planned, entering the engineering manufacturing development phase of the program with immature technologies, is to risk scheduled delays and cost growth. Instead, the program has an opportunity to mature technologies in a more risk-tolerant environment by making the right decisions now. In our report, we recommend that the Joint Strike Fighter program continue in its current program definition and risk reduction phase, delaying the decision to move into engineering and manufacturing development until technologies are demonstrated to acceptable levels. Taking the additional time to mature the technologies VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00012 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 9 will then allow the program manager to focus on design and manufacturing risks during engineering and manufacturing development. It also increases the possibility of completing product development in a more timely and predictable manner. Such a delay does not necessarily lengthen the total product development cycle. In fact, the knowledge gained from time spent developing technologies in the beginning can often shorten the time it takes to get the product to market. Similarly, a delay should not be misinterpreted as a lessening of support for the Joint Strike Fighter program. Rather, it would demonstrate decisionmakers’ willingness to make the up-front investment necessary to mature key technologies before committing the Joint Strike Fighter team to deliver a product. Such a commitment is more likely to put the program on a better footing to succeed than placing the burden on the engineering and manufacturing development phase. Mr. Chairman, that concludes my statement and I would be glad to answer any questions you or the other members may have. [The prepared statement of Mr. Rodrigues follows:] VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00013 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 10 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00014 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 11 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00015 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 12 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00016 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 13 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00017 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 14 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00018 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 15 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00019 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 16 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00020 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 17 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00021 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 18 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00022 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 19 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00023 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 20 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00024 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 21 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00025 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 22 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00026 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 23 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00027 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 24 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00028 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 25 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00029 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 26 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00030 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 27 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00031 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 28 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00032 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 29 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00033 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 30 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00034 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 31 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00035 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 32 Mr. SHAYS. Thank the gentleman very much. Thank you very much, Mr. Rodrigues. The bottom line is we’re scheduled to build the Air Force F–22 and the F/A–18, the F Super Hornet, and then we’re scheduled in the future to see this project go forward. You’re not suggesting in any way that we end this program and not do it; correct? Mr. RODRIGUES. Correct, Mr. Chairman. Mr. SHAYS. The bottom line to the debate we’re going to have today is you just want them to follow their game plan. Mr. RODRIGUES. We want them to—— Mr. SHAYS. By ‘‘they,’’ the DOD to follow the game plan. Mr. RODRIGUES. Yes, sir. Mr. Chairman, what we’re talking about is that the Joint Strike Fighter decision is such a major decision it is the signal of what happens in acquisition reform. It will underscore the implementation of the changes that are being made today in the regulations to guide this process. And if we make the same types of decisions that we have made in the past, that is, to allow unknowns to creep into product development, to allow technology development to be done concurrently with product development, we’ll continue to see very long development cycles like we have seen on the F–22, like we saw on the B–2, and like we have seen on programs historically. And that situation undermines our modernization effort. It leads to cost growth and schedule delays. It leads to the problems that we have seen with the programs in the past where we end up cutting the quantities in half and never do meet our modernization goals. In fact, it exacerbates the problems that we’re trying to resolve, one of which is aging of the fleet. It is a real issue. And to continue to move into programs where we can’t deliver on cost and schedule, that are fit into very fight funding wedges, puts us in a position where we end up in the future cutting the quantities because of cost problems and ending up without the modernization that we absolutely need. Mr. SHAYS. The bottom line, I just want to establish the point that GAO is not saying that we scrap this program. You want us to follow the game plan. And so what this committee has to understand is what you’re saying and what DOD is saying and where those disagreements occur. I’m going to expose my ignorance a little bit here. If we could go to the first chart, I find that’s the best way to learn. I sometimes make my staff nervous when I do this. Mr. RODRIGUES. What this depicts, Mr. Chairman is, in effect, the commercial model. Mr. SHAYS. What I need to do is just incorporate this with your nine product requirements. How does that—this isn’t the first three of the nine. Can you incorporate this to the—to TRLs? Mr. RODRIGUES. TRLs are a metric. They are a metric. They’re the metric that allows you to make the determination of the technology match to requirements. So that the first knowledge point in any program is assuring that you have a match between the technology and the requirement. Mr. SHAYS. Is point 3 here point 7 on TRLs? Mr. RODRIGUES. No. The 7 on the TRL is that that match has occurred in that technology-to-requirements match. It is the first VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00036 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 33 knowledge point. It is where we are approaching on the Joint Strike Fighter. Mr. SHAYS. And the debate, it seems to me, is whether DOD thinks your match point at 7 is the key point. They’re willing to move forward before they’ve reached 7. They dispute that they haven’t reached point 7. Mr. RODRIGUES. I’m sure that will become a confusing issue. Mr. SHAYS. I just want to know what you think. Mr. RODRIGUES. To me it’s absolutely clear. Applying technology readiness levels, which is the demonstration of these technologies, puts those technologies where they are on the second chart. Would you put that chart back up, please? And we’re accepting their projections of where they’re going to be. Once again as I said, the diamond to the right is where you should have to be. That is the level 7. That is the acceptable risk for entry into engineering and manufacturing development. Mr. SHAYS. What I’m asking you is do they dispute what you have up here and are they—believe that the acceptable risk is at point 6 and you believe the acceptable risk is at point 7? Where is the dispute as far as you would articulate it? Mr. RODRIGUES. They argue that there was a misunderstanding in the application by the contractors. We did not score these. The contractors applied the criteria. The DOD argument, as I understand it, is that there was a misunderstanding in the application. To get to a level 7 requires the demonstration of the technology or the hardware involved in this technology in the form fit and function. In other words, what it’s going to really look like when it has to go onto the Joint Strike Fighter in a relevant environment. And the Department’s position, as I understand it, is that there was a misunderstanding and that what people were thinking when they scored this was that we were saying it had to actually be on the Joint Strike Fighter. Well, that obviously cannot happen before the program enters engineering and manufacturing development because there is no Joint Strike Fighter. We worked very closely with the contractors and program office people who were at those meetings to explain exactly how to do this. I have the sheets that were provided to them. We spent days. And we made sure that they understood that we are talking about surrogates. There is physically no way to do the kind of demonstration they’re contending these people were thinking we were talking about absent a Joint Strike Fighter. What we were talking about was demonstrated on surrogates. Mr. SHAYS. To give some life to these technologies, you’ve listed under numbers because of proprietary issues, but—so I won’t debate which is which. But what we’re talking about is propulsion, we’re talking about flight systems, we’re talking about weapons. Each of these is a technology. We’re talking about structures and materials. In avionics we’re talking about radar and the mission systems and supportability and training and producibility. Those are the things that we’re talking about. Now when we started out, they were at level technology 1, whichever one it is, was at level 3. And you’re then saying that they have gotten to level 5 and expect to get in technology 1 to VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00037 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 34 level 6. And then make a commitment. They’re wanting to commit at level 6, correct? Mr. RODRIGUES. That’s what we’re saying. They’re not going to agree with that. Mr. SHAYS. But that’s what you think they’re saying. Mr. RODRIGUES. Yes. Mr. SHAYS. And they’re willing to commit on technology 8 at level 6. You’re not saying they’re willing to commit on technology 4 at level 5, are you; or are you? Mr. RODRIGUES. Right now our understanding is they plan to move ahead—— Mr. SHAYS. And every one of the levels I mentioned—— Mr. RODRIGUES. Unless they trade these off. But right now they’re in the program—these are critical path technologies. These things are essential. Not technology 1 necessarily—technology areas 2 through 7 are critical to meeting the affordability goal. And if affordability is the primary factor driving this program, which is my understanding of what everybody is signed up to do, an affordable next generation aircraft, then being able to launch or have a program without these becomes really problematic. The one example I used was if one of these areas was excluded or you had to go to its fallback, you would add almost 10 percent cost to the conventional variant which represents 1,700 plus of the aircraft they’re going to build. That is a significant cost difference. And that’s only one area. So these all deal with affordability. As long as affordability remains paramount, the question becomes if you don’t use these, what does it do to your cost projections? If you go do a block approach which—talking about using an iterative process going with the block one that doesn’t have everything and moving on—I think we have to understand what is the implications of that. What you’re doing then is launching into the development of a product and the production of a product that I’m not sure it even comes close to meeting cost goals, because I don’t know what the effect of deleting these from the first blocks are. And once you have that production line started, if you’re betting on technology in some subsequent block in order to get the affordability into the program that you need in the long run, I don’t think that we solved anything by going to a block approach. Mr. SHAYS. I would agree with that. Let me just go back to your first chart again on the best practices model. There are three knowledge points. And I appreciate the indulgence of the committee just to go a little further on this. Knowledge 1, this is basically the character of best practices act for any industry in this modern day and age. Mr. RODRIGUES. That’s the best commercial practice. Mr. SHAYS. The concept is matches made between the customer’s requirements and the available technology. So whatever technology is available, how can we meet the customer’s requirements? We try to match those two. Mr. RODRIGUES. Right. Mr. SHAYS. And then the second point is when the product’s design is determined to be capable of meeting performance requirements. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00038 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 35 Mr. RODRIGUES. Yes. And there’s a metric associated with that as well. The Joint Strike Fighter right now would be in the equivalent of what’s called the technology development phase. It is in risk reduction. There are concept demonstrations in the combined phase. So it’s really before that point of technology match. Mr. SHAYS. We’re not at point 2 yet. Mr. RODRIGUES. No, the product development is the equivalent of the engineering and manufacturing development phase. Knowledge point 2 is a standard knowledge point that occurs both in industry and in the Department. The Department just didn’t adhere to it. They actually have the standard and they have the metrics for measuring it. It’s something called CDR, critical design review, and the standard is that 90 percent of the drawings are released to manufacturing at that point. Now, unfortunately, the Department doesn’t adhere to that. The standard exists, they don’t apply it. In commercial industry, when we were doing the best commercial practice work, they exceed the 90 percent. Mr. SHAYS. I have two more basic questions. When we get to knowledge point 3, the product is determined to be producible within cost schedule and quality targets. Now, is the desire on the part of the military to move forward, in your judgment, with production before we have reached point 2, is that because they want the product to do more than right now the technology allows? Mr. RODRIGUES. Let me try to clarify something. This isn’t about moving forward at this point with production. This is about entry into product development. Mr. SHAYS. Right. Mr. RODRIGUES. And I don’t know—— Mr. SHAYS. But even the point of—the question still stands, and thank you for clarifying. The bottom line, though, is I’m just trying to understand the tension. In other words, they want this airplane to do more than the technology presently allows; correct? Mr. RODRIGUES. Using technology readiness levels, that’s an absolutely true statement. Mr. SHAYS. So the issue is if they had to accept the technology that existed today, they wouldn’t be able to have this plane do what they want it to do. So that’s the tension. But they want the project to keep moving forward. The message that I’m hearing from GAO is saying, relax, we’re at the cutting edge, we’re not talking about the cold war where we have to rush this to the marketplace. Mr. RODRIGUES. Absolutely. Mr. SHAYS. So we need to slow down, develop the technology, before we start to do the development. Mr. RODRIGUES. Yes. And the other thing they’re trying to get across in here is that when we make those decisions to accept these unproven technologies into a product development—and maybe I need to explain product development better. Product development is the engineering and manufacturing development phase in this context. In the engineering and manufacturing development phase, we should be focusing on developing and manufacturing the final product. In this case it’s the full-up plane with everything on it. What we do, or what we have done historically in the Department of Defense, is we go into that phase, which is difficult in and of itself if you’re using all proven technologies, and to try to inte- VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00039 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 36 grate a bunch of new technologies into a final product is still not an easy process. What we do is we allow immature technologies that are pacing items. These are defined, the ones we had up here, critical technology areas. Critical means they’re in the critical path to success. We allow that immature technology, unproven technology, to enter into the product development phase. What you end up with then are long development cycles because now you have to bring those technologies along. You have people having to focus on technology development when we should be focusing on engineering and manufacturing development of a product, not the subtechnologies that go into it. As I said, that’s a challenge by itself. And in industry they create a job that’s doable by a program manager. His job is to bring proven technologies together into a form that gives you the product that meets the customer’s requirements. We expect our program managers to manage technology development and product development concurrently. And technology development is invention. Invention cannot be scheduled. And we set up tight schedules where these things have to fit in those schedules. The money is all lined up. Money becomes the driver, schedule becomes the driver. And what we have actually accomplished tends to fall by the wayside. Mr. SHAYS. Let me recognize Mr. Blagojevich, the ranking member. I didn’t—if you have an opening statement you want to make I’m happy to have you do it, or we can get right into questions. But he’s your witness and you have him for as long as you need him. Mr. BLAGOJEVICH. Great. Thank you, Mr. Chairman I’ll just dispense with the opening statement and ask a couple of questions of Mr. Rodrigues. [The prepared statement of Hon. Rod R. Blagojevich follows:] VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00040 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 37 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00041 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 38 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00042 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 39 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00043 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 40 Mr. BLAGOJEVICH. Mr. Rodrigues, the chairman briefly mentioned that if we slow down the effort to get into the engineering and manufacturing phase—that’s essentially the gist of what your argument is—that does not necessarily slow down the entire program. And it’s conceivable, is it not, that if we slow down in the technological development phase, that in the long run the time saved early on to get it right could actually shorten the process? Can you speak to that. Mr. RODRIGUES. That’s exactly the fundamentals behind everything in the practices that we’re laying out from the work that we have done on best commercial practices: that putting the time in to get the technology match up front makes the product development cycle doable. Instead of having programs that extend for 10 and 12 years, that are absolutely unmanageable—if I could share with you—would you put up the chart on the 10 years? This is what happens to you as programs stretch and grow—and this is DOD data. Now, I would contend that it’s very optimistic, but what their data show is that on average you’d have an 11-year program cycle. In 11 years you go through 4 program managers, 5 program executive officers, 8 service acquisition executives, 8 defense acquisition executives, 5 chairmen of the Joint Chiefs of Staff, 7 Secretaries of Defense, 3 Presidents, and 11 cycles of coming up here to get money and going through the Department to get money. And we wonder why programs drag on and take forever? One of the problems we have is our cycle times becoming so long in the product development because we’re doing technology development and you should expect problems. We see them. I mean, this body approved entry into production on the F–22 with a single flight hour. And we expect that to stay on schedule? And we expect that to stay on cost? We knew virtually nothing about the capability of that plane in terms of demonstration. And what we have are a lot of hopes pinned on judgment about an ability to deliver those things. Judgment is not good enough in a best commercial practice. Demonstration is what counts. If you don’t demonstrate it, you end up with 12-year cycles and you end up with these problems and you end up with absolutely no accountability. We have to match cycle time to program managers’ tenures to be able to hold people accountable. Give them a job that’s doable, which is product development when we’re in product development. And the only way you can do that is separating technology development from product development. And the only way you can legitimately do that is through a metric that allows you to absolutely measure demonstration. Mr. BLAGOJEVICH. I know that chart—three Presidents, seven Secretaries of Defense, four program managers, the only ones that last 11 years are generally the Members of Congress. Mr. RODRIGUES. And, unfortunately, me. Mr. BLAGOJEVICH. And you. Can you talk about the importance of the continuity of keeping a program manager involved in a program like this, how it works in the commercial sector and how successful that is to have that continuity? VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00044 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 41 Mr. RODRIGUES. Clearly, as I look at these and try to figure out what can you control here, the only one you could really control and would really have a direct effect is the program manager. Program executive officers have a whole portfolio; some of the others are political appointees, you can’t control that. But the program manager tenure can be controlled. And if you don’t have that, what you have is a situation we’re in now. The Joint Strike Fighter is on its third program manager. It’s only been around for 31⁄2 years. It’s on its third. The third one is there now. His job is to deliver this program into engineering and manufacturing development as scheduled. The money is there. They’re ready to go. Everything is lined up. It’s in all their outyear budgets. We talk about wonderful plans for modernization and the need to be able to get a new plane. And I agree with that. We need to get new planes out there. My problem with the way we’re going to do this one is it won’t get there in time. It won’t get there in quantity and we won’t reach modernization. This isn’t what’s going to get you there. In terms of matching that program manager’s tenure—and it is what gives you accountability, we have to give these people doable jobs. Put them in there for the tenure of the program to deliver the product so we can get focused on the product. Our focus now is on the next increment of funding, the next milestone. How can people stay focused on delivery of a product that’s 12 years away? I can’t think of anybody. To tell you the truth I have been doing this, leading this work for over 12 years. I could not have focused 12 years ago on where I am today. There is no way. If you told me this is what I had to do, I would have told you you were crazy. It just doesn’t happen. You focus in shorter increments. People don’t think that way. Industry has gone from basically an 18-month to a 41⁄2 month cycle. The reason for that is to get that focus on product, give people doable jobs, get the stuff out there quickly because stuff turns over fast in terms of technologies. And rather than have these huge cycle times that lead to obsolescence—I mean, we’re spending hundreds of millions of dollars right now, on the F–22, to buy parts that are already obsolete. And we haven’t even built the plane. When you’re looking at 12- to 15-year cycle times, that’s what’s going to happen to you. We have to shorten that. I would submit as we did during a hearing before the Senate Arms Services Committee, Subcommittee on Readiness and Management—that in developing a model that would deal with these issues. One of the things that you have to do is you have to put a strict limit on the engineering and manufacturing development or the product development phase. And 5 years, I think Dr. Gansler has said 67 months at this point, is their goal. Five years is a doable thing. It allows you to match tenure. It brings accountability. It brings focus to the process. And it forces you to do trades on the front end. If you have a limit that you’re going to be held accountable to, then you will have to do the trades of the technology and costs on the front end that allows you to do a deliverable product in a 5-year cycle. Mr. BLAGOJEVICH. One final question, Mr. Rodrigues. We like the Joint Strike Fighter, you and I. You certainly—you have said you VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00045 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 42 do. I certainly think it’s a good idea. We’re for this. We want to see this get done sooner rather than later. It’s good for our military, good for our national defense. We agree to that. Right? Mr. RODRIGUES. Right. Mr. BLAGOJEVICH. What do you say to the argument that if we adopt what you’re recommending as opposed to what the Department of Defense is recommending, this can hamper the ability to keep the support here in the Congress to be able to fund this program? Mr. RODRIGUES. Unfortunately, I think that’s a real risk. I mean, it requires an understanding on the part of the Members. The fact of the matter is if we launch with immature technologies, you will not get this program on the schedule or on the cost. We will end up with the problems that we have had historically by going into product development with immature technology. You cannot schedule invention. It will create problems. And if they’re critical path items, they become the pacing item. And the cost of that is phenomenal. Let me give you some idea of this. Right now on the Joint Strike Fighter if you were to annualize numbers and take a look at it on a single contractor—because that’s where we’re going to go in EMD, down to one contractor—right now in the risk reduction phase, we’re spending the equivalent of about $265 million a year to do risk reduction. That includes the demo and all the other risk reduction activities on these critical technologies. To fail at $265 million a year on something is kind of OK. When we go into EMD, our costs for a single contractor immediately jumps to an annualized rate of $1.2 billion and within 2 years of that we’ll be at $4 billion a year. Now, you have a problem with a critical path technology at that point you are carrying a $4 billion program on your back. Not $265 million program, where you can tolerate some problems in invention. And you know the basic cost tradeoffs are for every dollar mistake I make in the risk reduction or technology development phase, if I correct that mistake in the engineering and manufacturing phase it’s going to cost me $10, and if I wait until the production phase to correct that same problem—which the Department is doing on the F–22s without having not completed the development phase—here that $1 mistake now costs you $100 to correct. So this has to do with bringing affordability and discipline to the process to get what we need, which is a modernized fleet. And I just don’t see where following the practices of the past are going to get us to where we need to be. Mr. BLAGOJEVICH. Thank you, Mr. Rodrigues. Mr. SHAYS. Thank you. Mr. Terry. Mr. TERRY. This is a very interesting discussion. Before I ask a couple of my questions I’ll say that I took a tour here a couple weeks ago of a company in my hometown that’s one of the leading high-tech support for our private sector industries across the Nation. I mean—and they made a comment during their presentation when I was being wowed by their technology. They said, ‘‘Our clients expect us to be cutting edge but not bleeding edge.’’ VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00046 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 43 And as I’ve listened to your testimony and thoughts about immature technology, that’s really what we’re talking about is getting ahead of ourselves in the technology and perhaps slowing down. I’ve got several questions but I’ll narrow it down to a couple here. And one is, just so I can put it in kind of a economical or simple terms, I see a catch–22 here where we in Congress expect out of our Department of Defense, and I think the people of this country expect if anyone is going to be on the bleeding edge or that’s acceptable, it would be our Department of Defense. They want that type of technology in there. Now, assuming that they have those type of political pressures, how do they develop a more acceptable best commercial practice time line? It seems to me that they never can. How do we do that? Mr. RODRIGUES. Right. Mr. TERRY. Taking in the desire to assure the public that we’re using best commercial practices, but then also accepting the role that we want them to be on the far beyond cutting edge to maybe the bleeding edge of technology that’s always moving. Mr. RODRIGUES. Right. I don’t see a disparity between these two things. What we’re talking about is how you go about getting a better product. It’s really about how do you get those technologies, those more advanced technologies in the field faster. Do you do it through a more iterative process or do you go for everything that’s out there. The latter takes us 20 years to get there and, by the time you get there, you can’t support it. It’s obsolete. You have got to buy out stuff because nobody builds this anymore. It really is about getting down to the basics of changing the culture and the incentives. But the other part of it is, you can be on the bleeding edge but be on it in technology development. Go chase those technologies. You need parameters around what you’re going to invest in, how you’re going to manage that technology base and how you’re going to graduate that technology to a product, and when is the right time to bring it into a product. I’m not saying don’t chase technologies. I’m saying don’t chase it as part of a product development. That has cost way too much money. Mr. TERRY. And hence, then, you’re doing your job in showing us or talking about the best commercial practices and how to institute that in the system. But then how do we justify, as Congressmen or the President or Department of Defense, you know, all right, we identify what we need in this plane, let’s manufacture it today in our Joint Strike Fighter force. Then the technology changes 2 or 4 years after production. That makes it obsolete and we end up spending more money in a production of the next generation, sooner than we probably should have. How do we justify that? I’m kind of asking you a political question. That’s not your realm here. Mr. RODRIGUES. I think the Department is moving to—and probably Mr. Soloway will talk about it—is the problem with the rapid changes in technology. It’s a reality. You can’t change that. People have to recognize that that’s the situation. And then what we really need to do is go to a more iterative process in how we build things. In other words, we don’t decide today on a firm design that we’re going to build for 20 years. I mean, that isn’t how the world works. In fact, that isn’t how we really build planes. If you look at VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00047 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 44 the F–16, we built A’s and B’s and block 10 and block 20, block 30, block 40, block 50. Now we’re at block 60. Now we’re getting ready to build block 60-I for somebody else. I mean, in reality that’s what you end up doing. Things change and you make those changes in the subsystems. But that basic design that you commit to has to last longer than just a few planes so you can get your money back out of it. So there are ways to do this. To me, it mostly rests with the Department. It’s what they come up here and sell. Unfortunately, we have a system now that incentivizes everybody to come up here and promise you the world and promise you all these wonderful things and lay out way early in a process an exact commitment to schedules and costs, build all that funding in. And you know when they’re doing that? They’re doing that before they even get into that technology development phase. They don’t have the slightest idea of whether those things are going to work or what they’re going to cost. But we lock into a commitment early on. And one of the things that needs to be done is to delay the commitment until you have the technology match. We shouldn’t allow people, to sell a product based on a technology that’s absolutely unproven. We’re raising everyone’s expectations to levels that are going to—— Mr. TERRY. And during your presentation when you were talking about that and answering the chairman’s questions, I wrote a note down: So what responsibility does Congress have? Because I think we encourage that. So maybe we should ask you to do a GAO report on what type of congressional or political reforms we need in house so we don’t encourage that type. I agree that in trying to abide businesslike procedures, we don’t encourage them from our side to do it either. Mr. RODRIGUES. We have done some pieces on this whole issue of what incentivizes the culture and the process at DOD, and Congress is a big player on that. Mr. TERRY. Before we run and vote, let me ask this one simple question of you. In your testimony you had mentioned and use kind of colloquial language here, but they said—they’re talking about adopting the best commercial practices, they’re talking the talk but they aren’t walking it yet. How do we ensure that they are implementing these type of best commercial practices at the highest level instead of just talking about it? Mr. RODRIGUES. I think you have to do it program by program. And the Joint Strike Fighter is the lead program and it is the one to do it on. And in the report I believe that you’re going to release here at the hearing, we have matters for congressional consideration. And those matters for consideration spell out what we think you should require the Secretary of Defense to lay out for you—and we need a metric that is understandable. Let me talk about technology readiness levels. They are actually very simple. I think that’s why people don’t like them because you’ll actually—you will understand them. And to simply say what they are, there are nine levels. What we’re talking about is progressing to the level of 7 for entry into product development or engineering and manufacturing development. The first three levels are basically paper studies. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00048 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 45 Now, I don’t know how many people want to launch a whole big aircraft program based on a paper study but I’m not too fond of it— paper study of a new technology. Not a good idea. The next two levels, 4 and 5 are basically laboratory hardware demonstrated in a laboratory environment. These are pretty easy things. I don’t know how many labs you’ve been out to, but I’ve been to laser labs and I was very disappointed. I thought I would see a laser. All I see is these parts spread all over tables. They’re hand wired. They can do the stuff you have to do in a lab. It’s only in that lab environment. It’s not going to be flying on a plane like we want it to, or space aircraft. And then to get to level 6 and 7, you actually have to get that big thing that’s spread out in a lab down to the form and the fit that it will have to be in in the intended product. You have to have actually taken it out into the environment you’re going to use it in. That doesn’t mean you put it on the plane it’s going to be on, because the plane doesn’t exist. That means you put it on a plane, if its something that has to fly, and you actually fly it. So now you have the form and fit that you’re going to need for the product, you put it into the environment—we’re not talking about operational testing. We’re saying understand the environment. That’s level 6 and 7. These things are understandable. They are things that you can measure fairly readily. They are not subject to engineering judgment or consensus, which is what we have tended to use or what we have used in the past that gets us into so much trouble. And they are things that you as a board of directors need to begin to understand and to focus on and to hold the Joint Strike Fighter to. Mr. TERRY. What is the timeframe for the product development timeframe? Mr. RODRIGUES. Right now they’re planning on going into—— Mr. TERRY. That you would recommend. Mr. RODRIGUES. Oh, I would recommend that we limit it to 5 years so we can bring accountability to the process. It would be hard with an 8 or 10-year cycle to say we’re going to put a program manager in there and we expect him to stay there for 10 years. I mean, life just doesn’t happen that way. Mr. TERRY. Thank you. Mr. SHAYS. I’m sorry, we’re going to have to keep you here. We’ll have just a few more questions afterwards. We have to vote. We stand in recess. We’ll try and be back in 10 minutes. [Recess.] Mr. SHAYS. Mr. Rodrigues, the hearing is reopened. I’m curious, that took me 15 minutes? So next time I’ll say 15. I’m sorry. I want to just establish some basic points and I’m not looking for long answers. One of the points I want to establish is to understand what I think I understand but to just have you describe it. It’s not an uncommon practice for a building to be built before it’s been fully designed. And the argument is you can build it faster and you can save yourself from increased costs, inflation rate and so on. I make the assumption that when you’re building a building and you haven’t designed it, you still have all the technology there. So that kind of argument isn’t compatible where you kind of build be- VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00049 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 46 fore you’ve totally designed. Here we’re trying to make sure that we know that technology exists, obviously, before production, but even before development; is that correct? Mr. RODRIGUES. Yes. Before product development, yes. Mr. SHAYS. And what I want to understand as well is that with the F–22, that basically is a new airplane, we’re not following this practice that we’re following with Joint Strike Fighter. The F–18 which we are—which the—E and F, and F is a modification, a larger F–18, but a modification of existing technology. It’s not a new plane, correct? Mr. RODRIGUES. There’s very little commonality between the F– 18C/D and the F–18E/F. The Department sold that as an ungraded F–18, but there’s nothing the same about it. Mr. SHAYS. So I didn’t get the answer I expected. But basically you consider it a pretty new plane. Mr. RODRIGUES. Yeah. Mr. SHAYS. We’re building the B–22 but we made a decision that we would not build the AFX which was to replace the A–12 Navy attack plane. We dropped the multi-role fighter. So we are now going to be developing the JSF and it is going to have to—the JSF is going to have to fulfill a lot of very different roles. It’s going to be used by the Air Force in conventional takeoff and landing. It’s going to be used by the Navy for shorter takeoffs on carriers, and my understanding is it has to be a tougher plane and it’s going to weigh a little more. And then we are going to be using it for the Marine Corps and the U.K. Royal Navy as a vertical takeoff, like the Harrier jet. And yet it’s all coming from one program. So there are a lot of technologies that are in play here. Your bottom line point is that we should not develop and produce this plane until we follow the game plan, which is to make sure the technology exists and that we use best commercial practices. The first question that I want to ask in regards to best commercial practices, what best practices would not be appropriate in the DOD acquisition process? Can you think of any? Mr. RODRIGUES. No I can’t think of any at all. Really what it then comes down to—and I think it was something that came out in an earlier question—it really comes down to how do you incentivize the process to put these things in place. What do we do to make it so that bringing knowledge to the table rather than judgment, to really be focused on setting a product development up for absolute success, how do you create the incentives to do that, to not oversell, to not overpromise, to not overcommit, but do what technology allows? Mr. SHAYS. How much cheaper is it to wait now to develop the technology than to begin to develop or produce this plane without the technology? Mr. RODRIGUES. As I said, the rules of thumb are if you run into problems when you’re in technology development, $1 problem there becomes a $10 problem when you enter into product development. And if you actually get all the way to manufacturing or production and you’re still doing technology and you’re having problems, that same dollar problem now becomes a $100 problem. So—— Mr. SHAYS. So a 10-to–1 ratio and then a 100-to–1 ratio. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00050 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 47 Mr. RODRIGUES. Order of magnitude increases, yes. And some estimates are much higher than that. Commercial industry, when you’re doing huge runs of production items where you have to do recalls, those numbers increase dramatically. Mr. SHAYS. Now, to the best of your knowledge—you’re speculating—what would motivate, in your judgment, the military, the DOD to move forward? And if—I always believe there are logical reasons why they want to move forward. Do you think it’s the potential that they think Congress might withdraw if they’re not heavily committed, and therefore they’d rather get the plane this way, even if it costs more and they get less planes? Because if it costs more they’re going to get less planes. Mr. RODRIGUES. Right. Mr. SHAYS. Or is there another reason that I’m not thinking? Mr. RODRIGUES. No, I think that is it. It is the incentives. The money is in place. It’s very difficult to get money in wedges and approval and get something going. I mean, picture a program manager that comes in now. His job at this point in time on something like the Joint Strike Fighter, is to get the program into engineering and manufacturing development on schedule. The money is all lined up. In fact, the program manager leaves as the down select is made, and the process is handed over to somebody else who is going to have to worry about this. But let’s imagine that he takes a look at it and he says, well, wait a second, you know, some of these technologies we really don’t know all that much about it, so what I want to do is propose a delay. Well, now he has to go through the process of the Department of Defense where there are all kinds of people competing for these limited dollars waiting to find a place where they can take it from. And the Department does it itself all the time. Then it comes up here to the Congress and there are a lot of people looking to fund other things and there’s only so much money to go around. That risk is absolutely real. Mr. SHAYS. It’s a real risk but it’s even more of a risk if the F– 22 and the F/A–18E and F are going to be much more costly than we anticipated because they’re not following these kind of practices. So in a sense their Joint Strike Fighter is going to be following a process that should save money in the long run or certainly not add to cost, competing with 2 other weapons systems that may gobble up costs. So I have sympathy for that if that’s it. Then I’ll just ask this last question. What is magical about a date, these time—I mean, is there—you haven’t developed any logic that says that a cold war enemy or a non-cold war enemy is going to be able to beat us and supercede us. So from that standpoint we don’t have a rush; correct? Mr. RODRIGUES. Right. The objective was to get an affordable family of next generation aircraft. The key was affordable. That’s not basically threat driven at that point. Mr. SHAYS. I’m going to just say it can be refuted by DOD. But the bottom line is one of the potential luxuries of the cold war ending—I consider the world a more dangerous place but for other reasons—but the cold war has ended, we have some ability to develop the technology or a lot of ability to develop the technology before we go into development and production. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00051 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 48 Mr. RODRIGUES. Absolutely. But going back, if you don’t have the technology match, the fact that there is some threat out there you need to deal with doesn’t justify the mismatch of technology because you’re going to run into problems when you try to build that thing. It is going to take you longer and longer and cost you more and more. Mr. SHAYS. So even if the cold war hadn’t ended, you would be arguing the same thing. Mr. RODRIGUES. I would be arguing for some constraint in what you do as you move forward. There are ways to deal with that. There are some things where we don’t have capability—and I can’t talk about those where there are real threats out there that we can’t deal with, particularly in electronics—where we would go ahead with a very limited scale of something to get some kind of capability, even if we know it won’t meet the whole thing. It gives us something in the short run to be able to deal with part of the problem. There are cases where you can make a case for why you would want to take those risks. I don’t see those in the Joint Strike Fighter. Mr. SHAYS. Thank you. Mrs. Biggert has joined us. I yield to her—not yield to her but give her the floor. Mrs. BIGGERT. Thank you, Mr. Chairman. I just have one question. I’m sorry I missed the testimony. But yesterday the Senate Armed Forces Committee deleted the engineering and preliminary manufacturing money but said that the Pentagon could get the $424 million back if the plane proved it was ready to take the next step. Do you think that this will be a possibility? Mr. RODRIGUES. That they’ll take the next step? Mrs. BIGGERT. Yes, that—— Mr. RODRIGUES. That they’ll be able to accomplish that, take the next step within this year? Not really; no, I don’t. Not based on where they are in technologies, provided those technologies remain untradable from meeting the affordability goal. As long as those technologies are there—I don’t believe you could get it done in a year. Could they get lucky and do it? I suppose. I don’t think they have the demonstrations laid out at this point that would get them to the appropriate technology level to have that assurance. I just don’t see how that would happen. Mrs. BIGGERT. Thank you. Thank you, Mr. Chairman. Mr. SHAYS. Thank you. Mr. Terry. Mr. TERRY. No. Mr. SHAYS. Is there any last—is there a question we should have asked that you would have liked us to ask or any point that you want to make? Mr. RODRIGUES. There is one point that I skipped, actually two things. One is when we talked about the differences as far as we’re concerned in the scoring of the technologies and the Department’s position on that. I do want to make you aware that we had a closed hearing and we went through those technologies. And could you put that back up again? I want to point one thing out. One of the basic arguments was that there was a misunderstanding that when we scored the technologies they thought we were saying in order for it to get to a level 7, the level needed to enter EMD, that it had to be on the VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00052 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 49 Joint Strike Fighter aircraft. As I said, that would be an impossibility, obviously not the standard. We spent a great deal of time working with the people who apply this at the contractor’s plants and the program office to get this clear understanding that we were talking about demonstrating something in close form and fit in the environment; not on the Joint Strike Fighter, on some kind of surrogate platform. But if you look at this, you can see a number of those technologies are projected only to be at readiness level 4; technology area 7, technology area 3, and technology area 2. As I said earlier, level 4 is not form and fit, it is laboratory hardware in a laboratory environment. So this argument about, oh, there was confusion about scoring this because we thought you meant you had to have it on a Joint Strike Fighter doesn’t even come into play in the scoring of those technologies. And I can tell you we made that absolutely clear. So there is no misunderstanding. The other thing is what can Congress do or what should you do? I think we should require them to be held to a level 7 of demonstration. And in those cases where they can or believe that they should move without that, they should have to provide a very discreet—first of all, that they score it properly and own up to the scoring. Let’s not play games. These are pretty clear things to score. There is no confusion. And once we score it, if the scoring comes out less than a level 7, the only way they should be able to move forward is make it clear to you where they are on those critical technologies and then explain to you why it is that we need to take on that additional risk of moving forward without having demonstrated the technologies that are going to be critical to the building of that final product, taking those technologies and having the very difficult task of trying to integrate a whole bunch of technologies into a final product. Mr. SHAYS. You wanted to make that point, but it does raise a question now. And the question it raises for me is, is this a package deal? Do all have to be there in order to go to that next step of development or do we isolate each one of these technologies as a separate issue before we move forward? Mr. RODRIGUES. As long as they’re on the critical path, the pacing item in any area, whatever it is—the long pole in the tent is what you have to worry about. Mr. SHAYS. Listen to the question I’m asking. Are they independent or do I have to take them all as a package? Mr. RODRIGUES. No, they’re independent. You do each one. Mr. SHAYS. Right. I anticipated that would be the answer but I just wanted to make sure. Are you at liberty to tell us which areas have the greatest challenge right now? Is this a proprietary issue or not? I can’t imagine it would be. Without identifying which is one and which is two and so on, can you tell us the area where we’re doing the best and where we’re potentially doing the worst? Mr. RODRIGUES. Clearly, the best—— Mr. SHAYS. Let me ask a question. Is that an uncomfortable question? VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00053 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 50 Mr. RODRIGUES. I can say clearly from this, the best are the ones that would be technology area 1 which I think you have a thing that tells you what that is. I can’t—I don’t want to say. Mr. SHAYS. Fair enough. Mr. RODRIGUES. But clearly one and eight. Mr. SHAYS. Someone will have to explain to me, and I should have clarified before but I’m not going to push the point, why it even matters, why we can’t have that dialog. But, fair enough. I guess because we haven’t made a choice on who gets what. Mr. RODRIGUES. Both contractors don’t work the same technologies. Mr. SHAYS. That’s fair. You’ve been wonderful. You’ve put the ball in play and you’ve given an opportunity for those that follow to answer your points. I don’t know if you or someone else can stay to hear the other presentations because we might seek to have your office respond. You preferably, but someone else. Mr. RODRIGUES. I’ll stay. Mr. SHAYS. We’ll go to our second panel. Is that OK? We’ll go to panel two. And I thank you, Mr. Rodrigues. We have Mr. Stan Soloway who is the Deputy Secretary of Defense Acquisition Reform, Department of Defense, and Major General Raymond Huot, U.S. Air Force Acquisition Program, Department of Defense. If you both would stand, we’ll swear you in. What I think I will do is just slide you over a little bit. Mr. Soloway, if you could just move your chair a little bit so we give Mr. Huot— General Huot, I’m sorry General. If you raise your right hands, please. [Witnesses sworn.] Mr. SHAYS. Thank you. Gentlemen, it’s great to have you here. I appreciate your spending the time with us. And I want to make sure that, Mr. Soloway, we have you first. Am I breaking protocol or am I keeping protocol? Mr. SOLOWAY. No, that’s correct. Mr. SHAYS. I’m in charge. Well, Mr. Soloway, we’ll start with you. Thank you for being here. STATEMENTS OF STAN Z. SOLOWAY, DEPUTY SECRETARY OF DEFENSE, ACQUISITION REFORM, DEPARTMENT OF DEFENSE; AND MAJOR GENERAL RAYMOND HUOT, U.S. AIR FORCE, ACQUISITION PROGRAMS, DEPARTMENT OF DEFENSE Mr. SOLOWAY. Thank you, Mr. Chairman and members of the committee. It’s a pleasure for me to be here today. If I could just divert for a moment from my prepared text and say that I do believe that there is extraordinary commonality between what the GAO is recommending and what we are looking at in terms of a strategy of how to deal with the issues before us, and hopefully we’ll have a robust discussion of that as we move forward. But I am pleased to be here to have this opportunity to discuss with you our continued progress with acquisition reform and particularly how it relates and applies to the Department’s Joint Strike Fighter program. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00054 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 51 As you know, acquisition reform has been a top priority for the Department for the last several years and encompasses a wide range of initiatives and has had many real successes. Let me list just a few. One excellent example is the Joint Direct Attack Munition [JDAM], which performed so flawlessly in Kosovo. Designated as an acquisition pilot program, JDAM was originally expected to cost in excess of $40,000 per unit, but through a combination of acquisition reforms and focused innovative program management we can now purchase JDAM for less than half that amount. Mr. SHAYS. That’s the unmanned plane. Mr. SOLOWAY. No, JDAM is essentially a guidance, is the best way to describe it. General HUOT. Probably the best way to say it, it’s a strap-on kit for a general purpose bomb. It gives it all weather capability. INS, Inertial Navigational System, with global positioning system guidance. Mr. SHAYS. Gotcha. I’ve seen it and I appreciate having it clarified. Mr. SOLOWAY. Then there is the precision location GPS receiver or PLGR. This receiver, purchased largely through the new commercial buying authorities contained in the Federal Acquisition Streamlining Act, replaces a previous field version built to extensive military specifications that weighed over 30 pounds, required two operators, had only one channel and cost us thousands of dollars per unit. PLGR, on the other hand, requires only one operator, has five channels, weighs just over 2 pounds and costs less than $1,000 per unit. The All Ordnance Destruct System [AODS], is a flight termination system used on the rocket system launch program vehicles at the Space and Missile System Center. The majority of these vehicles are used as targets in support of the ballistic missile defense programs. By utilizing the commercial buying authority known as FAR, Part 12, unit costs were reduced from the previous purchase price of $900,000 a kit down to $55,000 a kit, but more importantly, the new kits are also technically superior. Today, circuit cards for the avionics in the F–22 are being produced largely on a commercial line at TRW, thus saving the Department significant resources that would have had to be devoted to unique development and production facilities. Moreover, the reliability testing on those circuit cards have demonstrated excellent results and costs appear to be significantly lower than expected, 55 percent to 70 percent less than their military standardized counterpart. Acquisition reform is also central to the development of the Navy’s new Virginia Class Attack Submarine. Key to the success of the program has been the use of integrated product and process development teams, the use of open systems architecture, and the insertion of commercial off-the-shelf technologies. The Navy will benefit there with a cost avoidance of $30 million per ship but, more importantly, the Virginia Class will operate at a 32 percent lower total ownership cost than the comparable Seawolf. Indeed, much has changed and for the better. Given the complexity of our business practices and the entrenched cultures we have VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00055 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 52 inside and outside of the government, I believe we have made substantial progress. But clearly we must do more. The security environment we face is unpredictable and unstable. And our success in meeting the challenges of the battlefield of the near and more distant future will hinge in large part on our ability to access and integrate true cutting-edge technologies that provide us the dominance, speed, and scope of information that we need. One of the means for accomplishing these goals is through a restructuring of how we develop, manufacture, and maintain our weapon systems. It is no secret that cycle times for new weapon systems from concept to fielding remain unacceptably high and that such long cycle times too often result in the fielding of already obsolete technologies. Since some technology decisions must be made early in a program, it is clear that our history of taking 15 or more years to field new systems is not at all consonant with the torrid pace of technology change we see today. There are, of course, many reasons for these long cycle times. Key among them is often the very nature of the requirements set forth for any individual program. Traditionally, our requirements have been both inflexible and involved extraordinary technology challenges that can take many years to meet. This is beginning to change. Today both our operational and acquisition and technology communities recognize that to optimize support for our men and women in the field and to most responsibly steward the public’s tax dollars, we need to institute new requirements and acquisition strategies. Indeed, we are confident we can significantly reduce cycle times and costs and provide items for the warfighter faster, for a more flexible evolutionary approach. How we do this, how we develop, manufacture, and maintain systems is based on the Department’s 5000 Series documents, our bible for systems acquisition, which we are currently rewriting. The DOD 5000 rewrite will drive the Department further toward evolutionary acquisition and increase our focus on flexibility and requirements. Additionally, the new 5000 will require greater technology maturity prior to entering the manufacturing phase of a program. In the new systems acquisition environment, key acquisition decisions and long-term funding commitments may not be made until technology shows the required maturity and risks are better understood and mitigated than has traditionally been the case. And the JSF program is a forerunner of this new approach. Indeed, since its inception, the JSF program has been recognized for utilizing and actually pioneering many acquisition reform concepts and applying them to the actual business processes and contract vehicles being utilized, including but not limited to the critical precept of the new 5000 series. For instance, modeling and simulation has been proven in both industry and government to help reduce the time resources, and ultimately risk, associated with systems development. Representations of proposed systems, basically virtual prototypes, are embedded in realistic, synthetic environments to support the various phases of the acquisition process. The JSF program has made extensive use of M&S in the requirements development process and VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00056 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 53 is continuing the use of M&S with mission level virtual pilot-inthe-loop simulation to support a more thorough evaluation of required avionics capability. In keeping with the best proven practice in the commercial technology world, virtual manufacturing and virtual maintenance are also being pursued to facilitate planning and drive down the associated cost. The JSF is also incorporating an evolutionary acquisition strategy. In this process, the warfighter and the buyer work side by side to facilitate a better understanding of the requirements and decisions on tradeoffs between performance and cost. Specifying operational requirements in an incremental manner phased over time and matching them against the projected threat and available technologies have allowed the JSF program to exercise thoughtful judgment in balancing cost, schedule and performance. The use of evolutionary acquisition further mitigates risks by allowing technologies to be inserted as they mature. As I mentioned before, probably the most important change in the new 5000 rewrite is the emphasis on technology maturity before entering into system integration, or what is today known as engineering and manufacturing development. Of course, transition into EMD is a challenge in every program. And in the JSF program, as in others, it will be up to the design teams and the program office along with the Service Acquisition Executives and the Defense Acquisition Executives to determine the acceptable level of technology readiness prior to EMD decision. Among the many factors that can help us make overall technology readiness assessment are technology readiness levels [TRLs], which are used sometimes in the DOD. The minimum TRL rating of 1 begins at paper studies, as Mr. Rodrigues said, of a technology’s basic properties and rises to the maximum rating of 9 for a system in its final form, operating under mission conditions. However, there is no hard and fast rule in DOD and NASA or elsewhere in government as to specific threshold TRLs for any given decision. Additionally, it should be noted that there are more comprehensive methodologies that we do use which can and do provide more value as risk management as opposed to risk measurement tools. Indeed, the DOD 5000 rewrite does not prescribe a required technology readiness level but does recommend using TRLs as a tool to help measure the maturity level of the technology. What the 5000 does prescribe is that technologies be demonstrated in a relevant environment with a fallback plan at a higher maturity level. In other words, if a far-reaching, newer technology does not pass relevant testing, a lesser proven technology could be utilized as long as it enables the system to still achieve its critical performance requirements. Recently the GAO provided a draft report to the Department on the JSF that recommends extending the JSF development schedule to allow for further maturation of technologies. Their recommendation is based on their understanding that critical technologies will have inadequate levels of technology maturity based on TRLs at the time the EMD contract is to be awarded and the decision made in the spring of 2001. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00057 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 54 The GAO report clearly articulates a strategy for systems development that we embrace, as evidenced by the revisions to the DOD 5000 I mentioned earlier. Indeed, for the most part, we are in violent agreement. Where we differ is on the definition and applicability of TRLs in the decisionmaking process. GAO’s position does appear to be that achieving a TRL level of 7 should be required prior to entering EMD, again as Mr. Rodrigues pointed out. Our view, however, is that a level 7 requires the very kind of systems integration that takes place during EMD and that it is infeasible to produce full-scale testing of this type prior to that phase. Moreover, as noted earlier, we, like NASA and others, see TRLs as but one input to the decision process. In fact, as the NASA Director of Programs told us, ‘‘NASA does not formally use nor rigorously apply the definition of technology readiness level, TRL, to its systems development. We generally commit to development of operational systems when all critical technologies have achieved TRLs of 5 or 6.’’ In assessing a technology prior to EMD, we do seek to assess whether the individual technology has been proven or is close to being proven in appropriate developmental environments. We do not agree that it is necessary or even feasible to demonstrate the full integration of the technology prior to EMD. Indeed, the GAO report does seem to require that all technology must be flown on an actual or prototype JSF platform in order to demonstrate adequate maturity for EMD. However, much of the JSF avionics and software, for example, can and should be demonstrated on the ground at a lower cost. Technologies that must be flown for adequate demonstration will in fact be or have been flying on the concept demonstrator aircraft, commercial aircraft, the F–16, the F– 22, the F/A–18E/F and the Eurofighter. So, in short, although the technology will not be demonstrated on the actual JSF or a JSF prototype per se, it will be tested in a relevant environment. And the success of those demonstrations is critical to our confidence in our ability to successfully integrate the technologies during EMD. And we are confident we will have all critical technologies at an adequate and appropriate technology readiness level by the time an EMD decision must be made next year. I emphasize the decision is not to be made for another year. GAO has also expressed concern that if we make a premature production decision we could be locked into manufacturing processes based on an expected technology capability, thus creating the risk that if the integration fails we will face exorbitant cost and time in redevelopment. We agree there is risk. There is always risk in the integration of complex systems. And how the JSF program mitigates those risks is the key. As I think you will hear from Major General Huot, the risk mitigation initiatives associated with JSF have probably been the most comprehensive and aggressive of any DOD program ever. Further, as I noted before, the rapid maturation of modeling and simulation capabilities has enabled the development and testing of a wide range of critical technologies that might not otherwise have been possible and has played a key role in our ability to effectively assess the risks on the JSF program. Thus, any decision to proceed VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00058 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 55 into EMD and production of the JSF will carefully assess all risks against the fallback alternatives. Finally, let me be very clear. The strategy of revised acquisition process that will be prescribed in the DOD 5000 rewrite and which is largely being implemented on the JSF does represent a very real departure from our traditional approach to systems development. In the past, we would indeed consider technology development as part of the engineering and manufacturing phase. Now we have technology development before we make a commitment to—proceed into system development and demonstration, and eventually production. Mr. Chairman, acquisition reform has been made possible by a strong partnership between the Congress and the executive branch, and reflects our mutual commitment to ensuring that the government operates far more efficiently and effectively than has historically been the case. I appreciate having had this opportunity to be here today, because as we continue on the successful path we have forged, we need the continued support and commitment of the Congress. We also appreciate the continued support and encouragement the GAO has provided as we continue to change the worlds’s largest buying institution. Our disagreement on the specific role of TRLs notwithstanding, we are in close agreement on where the DOD’s acquisition process must go. We also believe it is important to let programs like JSF that are demonstrating the acquisition and management reforms of recent years to have the flexibility to manage their programs and make decisions based on weighing the risks against costs, schedule, and technology maturity. That concludes my statement. I would be happy to answer any of your questions now and certainly stand ready to provide any additional information the committee believes would be helpful in fostering a clearer understanding of this or other DOD programs. Mr. SHAYS. Thank you Mr. Soloway. That was a very helpful statement. We did allow you to go over the 10 minutes because I think it’s important for you and DOD to really state your case on the record. Then we can examine your statement. [The prepared statement of Mr. Soloway follows:] VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00059 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 56 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00060 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 57 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00061 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 58 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00062 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 59 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00063 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 60 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00064 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 61 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00065 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 62 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00066 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 63 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00067 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 64 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00068 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 65 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00069 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 66 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00070 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 67 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00071 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 68 Mr. SHAYS. I don’t think I’ll be as generous with the third panel. We’ll stick with the 10 minutes. General, now you only have 2 minutes. Mr. SOLOWAY. That’s the way it usually works between OSD and the services. Mr. SHAYS. General, you have your time. And we welcome your testimony. General HUOT. Thank you, Mr. Chairman and members of the committee. I appreciate the opportunity to talk about the Joint Strike Fighter program and the significant accomplishments it has demonstrated in the area of acquisition reform. By way of background, for those of you on the committee who I haven’t had the opportunity to meet, I have been in the operational end of the warfighting business for most of my career. I flew the F–105 Thunderchief in combat in Southeast Asia during the Vietnam conflict. Since then, I have flown several other attack aircraft, the A–7, A–10, F–18, and almost every model of the F–16. I was a wing commander during the Persian Gulf war. And since then, in my headquarters experience, I’ve been the Deputy Director of Operational Requirements and almost 2 years in this acquisition position as the mission area director responsible for fighters, bombers and munitions, and the JSF program falls into that category. Now, the JSF program has been a leader in the implementation of acquisition reform since its inception as the Joint Advanced Strike Technology Program or the JAST program. You may recall that we combined Air Force and Navy requirements after we cancelled the multi-role fighter program and the AX, helped them meet joint service requirements. Now, the need to produce an affordable aircraft drove several key decisions early in the program, including a single engine, a single seat, and a common family of airplanes for all three services. Specific advanced technologies were selected and then prioritized based on their contribution to not only warfighting benefits but also in terms of life cycle costs. We established aggressive cost goals for average unit recurring flyaway costs and for engineering and manufacturing development. And cost goals are being established even now for life cycle costs. These cost goals continue to serve as baseline independent variables for requirements and technology affordability trades. And I’ll talk about that regard in a minute because this is a new process that we have gone through. It is a robust and highly successful cost and operational performance trade process has been implemented by the Joint Strike Fighter on a continuous basis in order to help achieve those cost goals. Participation by the three major stakeholders in this process— our warfighters, industry, and the government has been key to the success of this cost and operational performance trade process. The warfighters, represented by both operators and maintainers, continue to provide a clear and unambiguous view on fighting concepts and needs as well as expected threats in combat conditions. And industry provides us with detailed weapons system concept and cost information and the program office adds an understanding of cost schedule performance supportability and resource constraints. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00072 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 69 More importantly, government engineers and analysts provide assessments on the cost and performance of contractor concepts. The result is that every requirement on this airplane has to earn its way onto the airplane based on cost effectiveness or cost-benefit analysis. Now, in contrast to some programs which are not initiated until a formal validated requirements document existed, the Joint Strike Fighter program was established so that specific weapons system requirements would not be frozen until the leveraging cost and operational performance trades had been performed and key technologies and concepts had been matured. This just-in-time approach to requirements avoided premature commitment to requirements that would be excessively costly to meet, fail to take advantage of available technology or, conversely, depend upon immature technology. In other words, it allowed time to work down the cost of the weapons system and insures that the requirements are consistent with the available technology. I would like to spend just a few minutes discussing the current phase of the JSF Program Concept Demonstration Phase we’re in. This phase began in November 1996 and it’s scheduled to be complete in the spring of next year. The major activities during this phase are propulsion system development, requirements analysis and definition, the technology maturation programs and, of course, building and flying two concept demonstrator aircraft per contractor. In regards to the concept demonstrator aircraft, I want to emphasize a couple points to the committee. These aircraft are concept demonstrator aircraft. They are not and were never intended to be prototypes. These demonstrator aircraft are required in the concept demonstration phase to accomplish three very specific objectives: One, to demonstrate a high degree of commonality across all three common service variants; to demonstrate short takeoff, the vertical landing, hover and transition to and from forward flight; and then demonstrate satisfactory low-speed carrier approach flying and handling qualities. In fact, there are other items to be demonstrated by the contractors on these airplanes but these are for competitive advantage purposes. But the hard requirements are the three things I listed above. We are totally convinced the JSF program office and our contractor teams have done a great job in ensuring that the concept development phase of this program will demonstrate a low level of technology risk for critical enabling technology and processes prior to entering EMD in the early part of next year on the current schedule. This has been accomplished through a very rigorous and disciplined methodology for risk assessment, risk management, and risk reduction. Now, Pratt & Whitney is developing engines for both the JSF contractors’ demonstrator aircraft based on their highly successful development program for the F–22. The result here is a high degree of commonality not only among the JSF contractors but also between the Joint Strike Fighter and the F–22. The F–119 engine core is essentially the same for both the F–22 and the Joint Strike Fighter. This commonality lowers JSF risk, VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00073 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 70 development time, life cycle cost, and accelerates that propulsion system maturity so we can provide a safer airplane for our warfighters. Under the current schedule, in fact, we expect to have over 500,000 hours of operational F–22 engine time before we have that engine at initial operational capability in the JSF. This is more than double what we had on the F–100 engine in the F–15 when we transitioned to the single engine F–16. I would like to just talk a little bit about the last two concept demonstrator—concept development goals—technology maturation and concept demonstrator aircraft, a bit further. The JSF program identified numerous technology maturation efforts to ensure low risk into engineering and manufacturing development. Now these technology maturation efforts aim to fulfill two key recommendations of the 1985–1986 Packard Commission: to apply advanced technology to reduce costs, not just to increase performance, and to demonstrate advanced technologies prior to the start of the EMD. In the Single Acquisition Management Plan [SAMP], that was signed by Dr. Kaminski in November 1996 it clearly states that the goal of ‘‘tech mat’’ program is to evolve the most promising leadingedge technologies to a low level of risk prior to integration into the JSF EMD program. It’s important to emphasize that integration and its corresponding risk is, and always was, to be addressed in EMD. Exit criteria from the concept development phase states the Joint Strike Fighter will demonstrate to a low level of technical risk those critical enabling technologies, processes, and system characteristics. Integration on the JSF program is the focus of EMD. Now the Joint Strike Fighter Program Office, in conjunction with each competing contractor, identified critical technologies processes and system characteristics required for the program tailored to each contractor design. Robust risk management processes were established by each competing contractor and validated by the program office. The government did not specify to the contractors which techniques must be used to track risk. The contractors selected what they felt were the best methodologies to accomplish that task. Now the government has been provided real-time access to those systems, actually on a computer data base, for oversight and review during the entire phase of the program. Both contractors utilize what is known as ‘‘waterfall charts’’ using a Willoughby template. This is a common and accepted methodology in industry and government. In fact, it’s taught at our Defense Systems Management College where risks have been identified, baselined, and tracked to document the very specific events required to reduce the risk of these critical technologies, processes, and system characteristics to a low level of risk prior to EMD initiation. Implementation of that risk management strategy has not changed since the program entered the concept development phase in 1996. And, most significantly, all of the critical technologies have achieved or are on track to achieve a low level of risk prior to the start of EMD. I want to assure the committee that DOD is convinced that our JSF weapons system contractors are appropriately reducing the VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00074 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 71 risk of these technologies through an affordable mix of flight and ground demonstrations, component demonstrations, and modeling and simulation and analysis. Based on a request from the Subcommittee on Military Procurement of the House Armed Services Committee, both our JSF prime contractors and Pratt & Whitney recently reaffirmed in detailed written responses, which were also shared with the GAO, that sufficient testing and demonstration is in place for the Joint Strike Fighter program to enter EMD at low risk. In summary, clearly since the program’s inception almost 6 years ago, the program office has been following a rigorous risk reduction plan. The risk reduction plan is on track to reduce the risk of each technology to low, by entry into EMD, and leave the integration of those technologies to the EMD phase where it belongs. This risk reduction effort has been an important part in the program’s overall goals to implement acquisition reform. This Joint Strike Fighter is vital to the modernization of all of our services’ air forces and many of our closest allies. Any significant delay to the program would result in increased costs and also have serious impacts on our force structure and readiness. This was highlighted in detail in a recent May 2 Deputy Secretary of Defense letter to our services’ most senior leadership. That letter was addressed to each of the service Secretaries, to the Chief of Staff of the Air Force, the Commandant of the Marine Corps, and the Chief of Naval Operations. The JSF was chartered to do business differently to demonstrate leadership in acquisition reform, and it has done this. Having embraced these concepts, it was rewarding to those who have worked so hard on this program to be presented the DOD David Packard Excellence in Acquisition Award in March 1997. All of this has been accomplished under the twin goals of developing an affordable weapons system that can meet the warfighters’ needs well into the 21st century while helping to reform the acquisition process. So, Mr. Chairman, when you ask, ‘‘Joint Strike Fighter, acquisition reform: Will it fly?’’, my answer to you is that JSF has already demonstrated acquisition reforms as called for by Congress, and it will continue to write the book that future programs will follow. Thank you. Mr. SHAYS. Thank you. [The prepared statement of General Huot follows:] VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00075 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 72 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00076 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 73 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00077 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 74 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00078 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 75 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00079 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 76 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00080 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 77 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00081 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 78 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00082 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 79 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00083 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 80 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00084 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 81 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00085 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 82 Mr. SHAYS. It’s great to have you both here and your statements were helpful. And I would like to ask a number of questions, stating first that I think we on both sides of the table agree that JSF is a necessary program. We’re going to see this plane built and all its variations. And so it’s just really a question of how to proceed. And—but I do think there are some substantive differences between GAO and your position, DOD’s position. And I’d like to investigate that a little bit. But I would also like to acknowledge, you know, where we may agree, so then we can just really focus on where we need to focus on. The bottom line is that the new attack fighter, the AFX for the Navy and the multi-role fighter, MRF, for the Air Force, is dropped and we don’t see any hope or need to resurrect that. That’s pretty much off the table. So it makes JSF even more important. I think we both can agree—so you nodded your heads. I take that as a yes. You’re going to have to speak. Actually it’s very hard for her to take a nodding of a head. Since I didn’t address it to one, I understand. So I’ll start with you, Mr. Soloway. And General Huot, if you disagree, you certainly would step in. So I assume if Mr. Soloway says—gives the answer, you are in agreement with him and vice versa. Is that fair enough? Mr. SOLOWAY. OK. Mr. SHAYS. So the AFX and MRF are off the table and we agree that JSF, Joint Strike Fighter, is our next plane, in addition to the F–22 and the F/A–18E and F. Would you agree with GAO that the F/A–18E and F is a significantly different plane, almost a brand new plane, or would you contend that it is a variation on a plane that we already have? I’ll ask both of you. Mr. SOLOWAY. Actually, the general has the expertise. General HUOT. Mr. Chairman, I really don’t feel qualified to provide a specific answer there. I don’t know that much about the specific differences between those two aircraft. Mr. SHAYS. I’ve always known the nonmilitary side to be willing to give an answer on this. So, Mr. Soloway. Mr. SOLOWAY. But I’ve already passed to the general. Mr. SHAYS. So that’s going to be—it’s going to stand basically, that comment, that it is a significantly different plane by the GAO, if it’s not refuted. Mr. SOLOWAY. That would be my understanding, but we can certainly get you more detail on what the differences are. General HUOT. It may be appropriate, Mr. Chairman, to go to the Navy and answer that question. Mr. SHAYS. OK. Fair enough. We would agree that the Joint— JSF is intended to do three basic tasks: for the Air Force, a conventional airplane, tactical airplane; for the Navy, slow-speed structure that has to be a little more durable to take the harder landings and the shorter landings; and for the Marines, a vertical takeoff. And those are, General—I make an assumption—three very different tasks. General HUOT. Yes, Mr. Chairman. Probably the best way to clarify that, for the Air Force the Joint Strike Fighter will replace F–16s and A–10’s. For the Marine Corps they will replace the AV– 8s and some of their F–18s. And for the Navy they will complement their F–18E/F force. And in the three variants of the airplane we VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00086 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 83 achieve—we plan to achieve a high degree of commonality, and right now that appears to be somewhere between 70 and 80 percent commonality between each of the three variants. There is the conventional takeoff-and-landing variant for the Air Force. There is the carrier-suited variant for the Navy, and the STOVL variant, the short-takeoff-and-vertical-landing variant, that will satisfy the needs not only for our Marine Corps, but also for some of our foreign participants, notably the United Kingdom. Mr. SHAYS. Things like the avionics would be similar in all three? General HUOT. Right now the avionics are planned to be common in all three variants. Mr. SHAYS. So some will be the same? General HUOT. Significant savings there in terms of commonality. Mr. SHAYS. But admittedly we tried this in the past where we tried to have one plane meet the needs of more than one branch— Navy, Marines, Air Force—and we haven’t always had—it hasn’t been all that successful, correct? I know we have—— General HUOT. I would say that the F–4 is probably an example that you might refer to where it didn’t meet everyone’s requirements. I would point to the fact in this program and as a warfighter I would say this has been a great process. This Cost and Operational Performance Trades process that I talked about in my oral testimony, this is where we had what we call an ops advisory group, an OAG, a group of warfighters that got together and worked with the contractor and the SPO, the systems program office; and we worked this requirements process to evolve the requirements for the services over a 5-year period. And in fact we finally got their final Operational Requirements Document approved by the Joint Requirements Oversight Council, that’s a group chaired by the vice chairman of the Joint Chiefs and with membership of the vice chiefs of each of the services, approved on March 13th. And as I said, what we did in there is, we went carefully through to make sure that each of the services gets the requirements that they need to do their individual job. So, you see, in each of these variants there will be some unique requirements to meet unique service requirements, but at the same time achieve the greatest degree of commonality that we can. And we think—— Mr. SHAYS. But the bottom line is—and I accept what you’re saying, but the F–4 was an example of a plane that didn’t do quite enough for any of the branches. It’s not to say we shouldn’t do it again to try to make it work, but this is a significant undertaking. Because it is not just for the Navy and it’s not just for the Air Force, it’s not just for the Marines; it’s for all three with a variation of basically between 20 and 30 percent from one to the other. And I would like to know if you agree or disagree with GAO when they said that if the technology development is $1, if they haven’t developed the technology by production development—product development, excuse me—that it’s $10, and that if they’ve gotten to the point of production, that $1 becomes $100. General HUOT. Mr. Chairman I have never heard those numbers before. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00087 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 84 Mr. SHAYS. So you would want us to substantiate those numbers? General HUOT. I would have no way of commenting on those. Mr. SHAYS. Fine. If you would speak just a little louder. But the bottom line is you have no way. Fair enough. Intuitively, to me, that is not an unrealistic number. In other words I wouldn’t be surprised if that were the case, but I accept for the time being that it hasn’t been documented. But it is now part of the record as a statement that we need to address. Mr. SOLOWAY. I think that we can—we will take the question for the record also, but I am not aware of any program that we have ever done that has had a 100 fold increase in cost from—— Mr. SHAYS. But it may just be that aspect. Mr. SOLOWAY. Or even that aspect, I’m just not aware of it. Mr. SHAYS. Let me clarify. Not that the whole program goes up, but that particular failure of technology has resulted in a 10 to 1 in product development and a 100 to 1 in production. So on the table—and your point is you’re not aware of that kind of—but intuitively we would agree that costs would go up significantly, and we have a heck of a lot of past history to document that that’s the case. Now, I see a difference between the logic when you’re building a school building your technology is there, but you haven’t designed it. So you say, let’s do it all at once, and in many cases it’s proved to be cheaper and you get the product sooner. You don’t wait to design everything; some things happen in process. But this is a bit different, correct? I can’t use that same analogy, and the DOD would be—wouldn’t use that same analogy. Mr. SOLOWAY. What we would say, sir, on that question is that we are in fact demonstrating the critical-path technologies prior to going into EMD. The difference really comes down to how one defines technology readiness levels as a means of measuring where you are in that process. We are not entering into EMD where we have critical-path technologies that either don’t exist or we haven’t demonstrated in what we call a relevant environment. But the difference really comes down to—and if you look, for instance, at the way in which the technology readiness levels are defined, even in the GAO report where at level 6 they talk in terms of system or subsystem models or prototypes demonstration in a relevant environment, we will have done that, I believe, in every technology, every critical-path technology prior to EMD. Level 7 speaks to a system prototype. Mr. SHAYS. Could someone put up that chart that GAO had. It was figure 4 in their report. That’s the technologies 1 through 8. Mr. SOLOWAY. My only point was going to be—— Mr. SHAYS. I know, but you were making reference to it, so let’s just leave it. Thank you very much. Mr. SOLOWAY. When you talk in terms of technology readiness level 7, which Mr. Rodrigues said they believe should be the threshold test before you go into engineering and manufacturing development, our view on that is that definition of TRL 7 speaks to a system prototype demonstration in an operational environment. That suggests to us that you have had to do a great deal VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00088 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 85 about both systems, engineering and systems integration, because you’re dealing here with a system of systems. Mr. SHAYS. What does it mean, though, when they say in a ‘‘test bed aircraft.’’ Mr. SOLOWAY. A test bed aircraft can be any aircraft you’re flying to test a given—it might not be one aircraft which—I said in my testimony, we have the CDA, we’re using commercial aircraft, the F–16s, F–18s, F–22, Eurofighter; various different technologies that need to fly will fly in an aircraft—— Mr. SHAYS. But it isn’t a prototype then? Mr. SOLOWAY. No, it’s not a prototype, though; that’s correct. That’s the key difference. Mr. SHAYS. The question that my counsel is asking is, who said it was? I mean, are you putting up a strong—— Mr. SOLOWAY. I’m sorry? Mr. SHAYS. That it was a prototype? Mr. SOLOWAY. It says under the definition of ‘‘technology readiness levels’’ in the GAO report of last July the definition of a level 7 is a system prototype demonstration, and I think if—without trying to get—— Mr. SHAYS. Then it says, examples include testing the prototype. Mr. SOLOWAY. Testing the prototype. Mr. SHAYS. So we have eight basic technologies and they don’t technically have to be tested on a prototype plane; they can be tested on another plane, correct? They could—so—— Mr. SOLOWAY. Well, you get into various discussions. For instance, on some of the—we can’t get into specific—— Mr. SHAYS. I think this is a key point. Mr. SOLOWAY. It is a very key point, I agree. Mr. SHAYS. And it would be wrong to suggest that somehow you have to have a prototype. The question is, can you test it in other ways to know it works before we move forward with development? Mr. SOLOWAY. Sir, actually—and I’m going to give you how I read their chart, and then General Huot may have a different sort of interpretation. I think there’s actually a great deal of grey area about the TRLs. They’re not quite as clear as has been defined, and I think that’s one of the reasons NASA and others have said you have to have some flexibility. Let me use the language out of the report that says, examples include testing the prototype in a test bed aircraft. So you have to have a prototype of the system. Now, does that mean in full form, fit and function where, for instance, the radar array has to be the full size, full scale that you’re going to use; or if technology allows you to, would you be testing a variant of the radar where the basic technology exists on a—for instance, an F–18 or some other aircraft. Others would be in the avionics. As I noted and I believe General Huot has talked to, the avionics don’t necessarily need to be flown on a test bed aircraft. They can be tested in an environment which is tremendously—has great integrity with regard to the operational requirements. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00089 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 86 So it’s not an automatic, in our view, threshold that says you have to test everything in a prototype which involves, essentially full scale, for each individual technology? General HUOT. Mr. Chairman, could I make just—— Mr. SHAYS. Let me just ask this point. The laboratory goes from one to three, correct, and then it gets out of the laboratory? General HUOT. Those definitions become fuzzy in the sense that in order to demonstrate some technologies on a low level, you don’t necessarily have to fly them at all. You may be able to do that in an environment that never involves putting that technology in an airplane. And so what the program did—let me talk about what the program did a little bit. The program did not use technology readiness levels, primarily because they didn’t include a risk management process. What they did do is, they used these waterfall charts, or Willoughby templates where they identified all of the key technologies that needed to be reduced to low risk before going into EMD. Now, these—this process identifies the technology and then lays out over a schedule the critical events that have to happen to reduce that specific technology to a low level of readiness. In addition to that, it provides off ramps, if you will, where if that technology is not going to work, they would revert to an alternate technology. This was chosen by the contractors as a way of not only assessing risk, which the TRL process does, but also managing risk and being sure that you get to that low level of risk when you’re ready to go into EMD. And that’s the process that they used and that’s the process that the program office monitored each of the contractors on a—— Mr. SHAYS. Where you both agree, and then we’ll define it: You both agree you shouldn’t go into EMD before those critical technology—before you come to low-level risk. General HUOT. Exactly. Mr. SHAYS. So the debate now is whether what you described as low-level risk is low-level risk. I mean, that’s going to be—because you buy into the point, though, that you don’t move forward if you’re at high-level risk. And high-level risk is where you don’t have the technology. Mr. SOLOWAY. There’s always—as I said in my testimony, when you’re dealing particularly in this kind of complexity of systems, the risk of integration, which is what you do in EMD, is always there. I mean, to say that we can whittle that risk down to a very low level in this level of complexity is not realistic. What we are looking to do, just to make sure we’re clear on the terminology, is to arrive at a very low level of risk relative to the individual technologies. Mr. SHAYS. Let me just tell you what makes me nervous. You have described—you basically have described the technical readiness levels as tools rather than requirements. And then you’re doing one thing more; as I hear it, you’re also in a sense redefining low-level risk. And I thought at least we could get to the point where we agree you don’t move forward unless you’re at low-level risk. You’re trying to qualify low-level risk in a way that makes me uncomfortable. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00090 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 87 Mr. SOLOWAY. I just don’t want to leave on the table a suggestion that when you go to the integration and EMD that your risks are automatically low. What you have done is substantially reduce your risk by proving the individual technologies. But you still have a high-risk element because this is a complex undertaking. Mr. SHAYS. Well, I’m just going to tell you, that leaves me a little uneasy because that allows you to define it in a way that allows to you move forward, no matter, as I hear it. Mr. SOLOWAY. In the past, we would not necessarily in all of our systems development bring individual technologies to the level of maturity that the JSF is doing, that we will now be requiring in the new 5000 rewrite; that’s the individual technology levels. To get further than that into and integrate the system is what you do in engineering and manufacturing development. That is always— there is always risk there, as I said in my testimony. We have reduced that risk by proving out the individual technologies. That’s the big step forward that we have taken over previous practice. Mr. SHAYS. I feel like I’m dealing with a moving target here. I know you’re trying to be helpful and you’re trying to be responsive. What I want to say, and then we’ll just have our disagreements— what I want to say to you is, you view technical readiness levels as a tool. I view them as something more than a tool. I view them as more a requirement, and you view them more as a tool. I view low-level risk as—obviously, when you’re dealing with military hardware, there are risks, but I think that low-level and high-level are pretty clear. And then there are going to be different levels of low risk. Now, where I may have some question is, if you’ve leveled—if so much of one technology, say the structures and materials, you’ve got almost everything licked, but one thing, and that one thing is still there, but you don’t want to hold up the rest of the project because everything else is ready, I would think that you may still have something that needs to be resolved; and you wouldn’t want to wait another year for the whole project if you want to start to integrate these technologies together. So I may be a little off with that and I’ll, you know, ask GAO to help me out on this later as well, and other witnesses. All I’m saying to you is, I’m uncomfortable by your referring to this as a tool and your redefining low-level risk. That’s where I’m uncomfortable. Let me do this: I’m going to come back. I’m going to let counsel and minority staff ask questions as well, because in the process of their asking questions it helps to define what I want to continue to ask. Mr. Halloran. Mr. HALLORAN. You want him to go first? Mr. RAPALLO. Sure. Mr. SHAYS. This is David Rapallo. Mr. RAPALLO. Could you just describe what effects you mentioned briefly in your statements—what effects a delay would have, a delay of say 6 months? General HUOT. I wrote down some specific notes to talk to that. I think probably the best source of information there now—there VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00091 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 88 was an attachment to Mr. DeLeon’s letter to our senior service leadership that talked about the impacts of delays. Obviously, if you delay the program any period of time, you are going to incur some costs of—increased costs in the program itself. But more importantly, in this particular case, you’re looking at a delay in development that could roll into a delay in production that could be more than 6 months. And if you will go back and look at the history of programs, some delays in development of a program have led to just that. If you were to incur any kind of a significant delay here, we would have some significant force structure impacts and some impacts on our readiness. The DEPSECDEF letter attachment looked at a scenario that went out as much as 3 years and said, if you delayed the program procurement phase by 3 years, that we would end up with about a three fighter wing equivalent force structure shortfall that we’d have to deal with in a couple of different ways. You could buy gap-filler aircraft, F–16s, more F–16s, or you could do service life extensions to those aircraft if that were possible. Now, we have done service life extensions to F–16s, so in general, we think that’s probably a ‘‘doable’’ do. How much you would have to do would have to be determined. In the case of the Marines, though, it appears that they would not have that kind of an option where there really isn’t felt to be, in my view or my understanding, a viable way to carry the AV– 8 further into the future. So they would end up with a force structure deficit in order to meet their military commitments worldwide. And without getting into the details of what that would mean, in terms of their ability to do their mission, that attachment in the DEPSECDEF letter talked about some of those impacts to our CINCs worldwide. So we think in the long run there are some very significant impacts if you would slip this in terms of force structure and readiness impacts. Mr. SOLOWAY. I might add one other to that in terms of cost. The longer we delay a decision—and I don’t want to suggest that the decision—Mr. Chairman, you asked the question, one of the members asked the question about what is sacrosanct about a certain timeframe for a decision. We set out a timeframe in which we thought we could make a decision, and we still believe that within a year we’re going to be able to make an EMD decision. But it is driven by our requirement of having the technology maturity of where we think it needs to be to make that decision, not by any arbitrary line in the sand. But presuming for a moment that we have what we believe are adequate levels of technology maturity, you have to realize that we would be delaying that decision for a longer period of time. Carrying both contractors, for instance, which is going to continue costs and so forth, would have an impact on our international partners who are participating in the program and the potential of either pullout or other reopeners, if you will, in terms of the international relationship. So there are a number of sort of follow-on impacts that could transpire as well. Mr. SHAYS. I want Mr. Rapallo to continue to develop this line, but I just wanted to ask you the question again: You say ‘‘adequate VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00092 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 89 levels of technology and maturity.’’ That’s just a different term than ‘‘low level,’’ and ‘‘adequate’’ leaves me really—— Mr. SOLOWAY. I apologize if I seem to be obfuscating, because I really wasn’t, sir. When you made the comment that we would be at low risk, we will have low risk relative to the individual technologies, yes, that is a requirement. And that is a requirement. What is not a requirement necessarily is that TRLs be the measure of that risk. We are recommending TRLs, by the way, in the new 5000 just as a common language, but there are, as General Huot said, other ways in which one assesses, measures and manages to risk mitigation and risk management other than TRLs. They themselves are not the only. Mr. SHAYS. I’m going to go back. You’ve made the point that the Harrier—I call it the Harrier jet; is that the British term for it? But the vertical takeoff jet is deteriorating significantly. It needs to be replaced soon. The question mark is, do you speed up everything because of that? And one of the questions—I have total acceptance that if you’re in production, and then we in Congress do what we sometimes do: We say, instead of 30 planes, it’s 20 planes; we add costs, I understand that. I understand how we slow things in production, but frankly sometimes the technology isn’t there or is being revised. That’s one of the costs of slowing it down. But I have a harder time understanding how slowing down the decision before you go into development and production adds significantly to costs. And I’ll just have you come to that question afterwards, but you had mentioned it and I just want to—I would love you to continue. Mr. RAPALLO. That would be—my only followup is, GAO has sort of stated or described a process where if you delay the EMD process, you can develop the technologies and ultimately save money and save time. Do you disagree with that? Mr. SOLOWAY. No, I don’t disagree with that, but on this given program, I think what we’re saying is, we would not go into EMD if we did not have the technologies developed, so that the delay in time is not necessarily going to help. We believe the technologies will be at a level that allows us to make a responsible decision to move to EMD. Now, whether we agree that, based on a TRL chart, if one were to use that measure, one would need to be at TRL 7 or 6, that’s— that is actually the crux of the debate, and what does it require to achieve a TRL 7 under the construct. But clearly if the technologies are not there, you cannot—particularly the critical-path technologies that we’re talking about, then you wouldn’t make that decision. So in this case, all we’re saying is, to say today that we ought to delay the program, when we believe the technologies are either there or close to there and will be there in the timeframe, there would be no reason to delay. Otherwise, if the technologies aren’t there, it’s correct, you would not want to go into EMD; you would want to take advantage of further development time. Mr. RAPALLO. Thank you. Mr. SHAYS. If you want to come back, we will. Mr. Halloran. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00093 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 90 Mr. HALLORAN. Well, I guess the followup that occurs then is, you believe that technology—to be ready, how will you know? If not with some more quantitative or objective measure than TRLs, how will you know that, say, the vertical-takeoff-and-landing technology—for example, speaking hypothetically, how will you know whether it’s ready or not? How will you measure that? Mr. SOLOWAY. First of all, as General Huot said, that there are things some people call the Willoughby templates, and so there are other methodologies that are used. And you may, I think, hear from other witnesses in terms of probability, as well as objective measures and so forth. But the other reality is, and I don’t mean to be glib when I say this, it’s either going to be working or it’s not. It’s either—the capability to do what you are talking about has either been demonstrated or not demonstrated on a concept aircraft or what have you. The radar or whatever technology you’re talking about will have or will not have been demonstrated on another plane. The avionics will or will not have been demonstrated in a suitable laboratory environment. Actually, when the chairman mentioned laboratories at levels 1 to 3, there is a category in level 6 that talks about a high-fidelity laboratory, different than sort of the low grade. So we will actually have demonstrated the individual technologies in a variety of different ways most relevant to what we have to prove and see. General HUOT. I think the short answer there is that we will continue this rigorous and disciplined approach that we have had using these waterfalls to track each one of these technologies and assure that it’s a low level of risk. In fact, in the input that we got from Boeing and Lockheed and Pratt & Whitney in the inputs requested by House Armed Services, the Subcommittee on Military Procurement, they went through, in great detail, each of the technologies describing the process that they were using and reaffirming that we would be at low level of risk prior to entry into EMD. That is really the tool that we have been using. And it is a risk management tool that tracks, is event based, and it also again allows you to make decisions if you need to go to an alternate technology to be at low risk prior to EMD. Mr. HALLORAN. Stay with that for a second and describe the waterfall again. What I heard and I think what the chairman heard is that high risk plus risk mitigation plan equals low risk. General HUOT. No, not a plan. You have to execute the plan and then you monitor the plan to assure you’ve gotten there. So the—— Mr. HALLORAN. That’s a quantitative way? General HUOT. In a quantitative way. The program office goes through, and as the contractor completes each of the events that they’ve agreed are to track this from high risk to medium risk to low risk, they check them off. And so you end up essentially with a checklist that says you are at low risk in that technology area—you have done all the things that you said you needed to do to get there. Mr. SOLOWAY. Let me just add one other thing, if I could, on that point. Once you’ve done that, you also have, and General Huot mentioned a couple of times, fallbacks. I think it’s important to understand that as you go through this process, let’s say for instance VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00094 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 91 that we have a technology that was demonstrated on an F–16 and that we thought this is ready to go. For some reason, as we go forward, it doesn’t work. What do we have to have done prior to EMD to address that potentiality? That is where the fallback is. You have to have an assessment of what lower-level capabilities you can access without affecting the critical performance parameters of the plan. Mr. HALLORAN. And cost? Mr. SOLOWAY. Certainly. Mr. HALLORAN. Because I was going there next anyway. I’m glad you raised that. Because GAO made the point that in a couple of these critical areas the fallback was quite expensive and did affect performance or the requirements anyway, for example, and yet you testified that the 5000 series calls for a fallback that it would be at a higher—— Mr. SOLOWAY. The fallback itself, if it’s measured, a TRL would be higher, but it might not be as high technology. Mr. HALLORAN. But it involves a tradeoff of some kind. Mr. SOLOWAY. There’s a tradeoff certainly, and you have to assess all that in this process. And you have critical performance parameters that you have to meet and you can’t go below that. Mr. HALLORAN. Let me change the subject slightly and talk about the definition of threshold between technology maturation and integration. It struck me that in rescoring the original contractors, TRL scoring, the Department kind of expanded the integration to get itself out from under some low scores here and to push things up to 6s and 7s, to where you could define your risks more clearly, or at least as lower. So talk about that threshold some more. I mean, integration, there’s an example in some testimony that comes in in a minute of a DOD system that looked fine in pieces, but when they came together they interfered with each other and didn’t work. And that’s clearly an integration issue, and it’s a separate set of risks, as you testify. But is getting to demonstration of form, fit and function in a proper environment, is that integration or is that maturization, maturity? Mr. SOLOWAY. I would suggest that the way—if you’re taking that comment and weighing it against the way TRLs are laid out and what levels you hit, because it’s sort of the context of the question—that you have to do a fair amount of integration to do that. So my answer would be, yes, we think that—let me go back to your initial question, because what you really were asking is, did we try to jimmy numbers to get out from under a bad score. That was the first question. And my answer is, and I can say this as one who was not involved in that initial process, but have gone back over the last month or two and have met with all of the players, have had extensive discussions, services and so forth. I genuinely believe that there was, despite what Mr. Rodrigues said, a lack of understanding of how these were being applied, because what was being done was assessing levels relative to the risk of systems integration. This is for the JSF. And so when you do that, you are—your relative risk is higher. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00095 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 92 But when you’re weighing it against a question of, have you proven out the individual technology, which is what we believe and the 5000 really speaks to—and the new 5000 needs to be matured in a way we have not traditionally done before, before you go to EMD—when you do that, that is—the result is the rescored numbers. Mr. HALLORAN. What was the source of the misunderstanding? As you say, the 5000 draft had been out a while. It had reached— it had received some resistance. Mr. SOLOWAY. This was not relative to the 5000 rewrite. This was relative to the GAO—— Mr. HALLORAN. But the issue and the proposed role of TRLs, as you state in your testimony, at some point push would come to shove and a low TRL would prevent a technology from moving into EMD; that was pretty well known. What was the source of the misunderstanding? General HUOT. How to grade TRLs? Our program office had never used TRLs. When they graded them to come up with these ratings that you see on the chart, they rated those, including the risk of integration on the Joint Strike Fighter. Of course, that’s going to give you a much higher risk; that’s why the lower ratings. They did not rate the individual technology area by itself without consideration of the risk of integration. When they went back and did that, in fact, that was spelled out in Dr. Schneiter’s letter that responded to the GAO report, he made it very clear that the Department position was that the Department does not agree with the conclusion, which is based on misinterpretation of a process for determining the readiness of technologies for incorporation in major systems; and went on to say later that the GAO ground rules for scoring technology rating levels included the risk of integrating the technology onto the JSF platform. The JSF program office used those ground rules to arrive at the ratings contained in the draft report, and upon review and discussion with other users of technology readiness levels, that program office determined that only the maturity of the technology, not its integration, should be rated to determine the readiness under EMD. One finds that technology risk is expected to be an acceptable level of EMD start. Mr. HALLORAN. Let’s go back to my original question. Define ‘‘integration’’—‘‘integration’’ meaning form, fit and function down to the size it will have to perform next to other systems, or ‘‘integration’’ meaning actually working with other systems on an airplane? If you define it that broadly, then of course you’ll, I think, diminish the use effectiveness of TRLs as a threshold judgment to get into EMD. General HUOT. Let me say just a couple things here. First of all, it was clear that some of these things have to fly on aircraft in order to get to the low level of risk. In fact, if you would bear with me just a moment if you look at those eight technology areas without addressing the eight technologies specifically—for example, technology No. 4 is flying on an F–16, technology No. 1 is flying on a concept demonstrator aircraft—in fact, a lot of these technology areas will fly on aircraft. But when the program office made VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00096 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 93 their initial assessment using those, the TRL scoring, they included the risk of integration on the JSF platform to score those. Mr. HALLORAN. Let me finally, Mr. Soloway, in your testimony, in talking about the 5000 series, and you say—which is largely being implemented in JSF, speaking more broadly about acquisition reform, now, what—in what way is it not largely—gives you some room out—in what way is it not—what challenges does the Department face in more fully applying—was this referring to this program and others that follow it? Mr. SOLOWAY. You’ve got two separate questions. Let me start with the JSF specific and then move to the broader question if I could. On the JSF, when you—we complete the 5000 rewrite, which is now out for comment, we expect to have it finalized in the next 30 to 60 days; it’s been a long process; it’s a fairly complex set of documents—there will be actually a new systems acquisition model, if you will, with different kinds of entry and exit criteria based on much of the kind of model that GAO talked about this morning. And this is precisely where we are going in the Department, and the kinds of criteria that you have to meet to go into various phases of the process, and they may have different names, and it’s really a different model. JSF has, in effect, mirrored that model, but not precisely because the model didn’t even exist until the last several months, so it’s not a precise marry-up in that way. But I think JSF has very much reflected what we are seeking to do in the 5000. On the broader question, what DOD or the Congress needs to do, I think there are—the list is quite long in terms of the challenges we face. Part of it is the sort of entrenched cultures that we both have in terms of how we view programs and systems development, and some of the issues that you raised with Mr. Rodrigues earlier today. And that’s both an internal, DOD problems with the Congress, and the external world. Part of it has to do with—and I’ll give you one example. I think one of the real challenges with the new 5000 rewrite— and this is not relative to JSF, this is a broader question—is if you are in this front-end process where you want to, and have to, demonstrate technologies, you want to also—and thus not make your commitment to EMD and, really, program commitment—until you’ve gotten to that point, you will also hopefully, as we go down this path, be looking at more options than you might otherwise have been looking at. And what that leads you to is having to avoid, or wanting to avoid, a program commitment either in the building or from the political environment before you’re ready to say, I’ve got a set of technologies that can get me this capability, and that’s when I’m ready to make that commitment. You know, traditionally we’d say, we’re going to do a new jet or we’re going to do a new ship, and then we sort of work it through. But the ship never left the table or the plane. It was fairly rare once you got that far down the path. What we really want to do is both for ourselves inject a great deal more discipline. As I said, we have tremendous agreement, and GAO has been a partner at the table as we have built this model, but we also have to have discipline in the process to realize VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00097 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 94 that we may want to be looking at different kinds of capabilities and different kinds of mission options before you make those kinds of commitments. And that’s traditionally been very difficult, I think, for all of us. So there are a number of challenges that we all face in moving this forward. And I think that they’re very—I think they’re challenges that we can overcome. As I said, I do believe that JSF is really the forerunner coming out and applying many of these principles. The General was talking about the process they went through with the various stakeholders, the different services. A really flexible requirements process driven by what the customer really needed and what we believed as we went through the process technology was going to be able to provide in a reasonable period of time, so that we didn’t end up waiting 18 years or 20 years, as has been the case in the past. Mr. HALLORAN. Finally, let me ask a followup where Mr. Rapallo began. Would you agree that all of the impacts you cited that might flow from a delay at the demonstration evaluation phase can potentially be more serious, more costly and lengthier if they’re incurred in the next or subsequent phases of the program, that, as is described— we may disagree on the $1 to $100, but there is an almost inevitable escalating effect in delaying problems in this process. Mr. SOLOWAY. That would certainly be the conceivable output— outcome. However, I think our view at this point is that the delay—to arbitrarily decide today to delay, to really do, in essence—and I don’t want to get into nitpicking, but in essence what we’re talking about here when you get to level 7 is the early phases of EMD. That’s what we do in engineering. We focus on the manufacturing part and the production part of it. We don’t focus on the ‘‘E’’, which is the engineering. That’s a critical next step in bringing these technologies together. We believe, and we could be proven wrong next year and the decision would not thus be made, but we believe the technologies will be at that proven level that enable you now to move into that engineering of EMD phase. So given that, the delay to us would be costly, unnecessarily costly, and would be just unnecessary on the face of it. But in a hypothetical world having nothing to do with JSF, certainly if you have tremendous technology issues and you go into EMD and lock yourself in, you could be asking for an escalated problem. I wouldn’t disagree with that; I just don’t think that applies to the JSF. Mr. HALLORAN. Thank you. Mr. SHAYS. Let me say, then, I would like David to come back with answering a question related to what you were saying up here. I’ve heard you both say, you know, this is something, you know, we have never done before. You’re very proud of the program, the fact that you’re moving toward a best commercial practices. But what I feel like, as I hear you, is that it’s going to be so significant to DOD that you’ve moved in this area if you do it halfway, a three-quarters way, this is monumental. And I might agree with that, that it would be monumental, but I have to hear an argument VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00098 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 95 that says you can’t go all the way with the program. And so for you to say you’re doing three-quarters of it—and you haven’t used those numbers, but that’s the feeling I get—wouldn’t satisfy me unless you say we can’t do best commercial practices here, here and here because, and then let us evaluate it. So I am going to stay on the table that really there is no argument against using best commercial practices to the nth degree, unless, and I just kind of want to say that to you. Mr. SOLOWAY. If we gave that impression, I think both of us would regret it. How do you define doing it all the way versus the three-quarters? Mr. SHAYS. Well, when you start to talk about it as a tool and not a requirement. And maybe in the private sector it’s a tool and not a requirement, and then that’s fine. Yes? General HUOT. I would just say that—I would emphasize that we use a different tool. And again the reason we did it is because we wanted something that would allow us to manage risk, not just assess it. And our intent is not to go three-quarters of the way; we intend to reduce every critical technology that we identified to a low level of risk before we go into EMD—very important point. Mr. SOLOWAY. And the requirement that is there is the technology maturation requirement. That is a requirement; that is not a negotiable. What is negotiable is the TRL becomes like the yardstick, so it’s versus using a yardstick or a ruler or another method of measuring what the outcome is going to be. But the key here, the outcome we all seek is technology maturation. And that is not a negotiable. Mr. SHAYS. Does the DOD 5000 rewrite follow this practice, best commercial practices? Mr. SOLOWAY. Yes. Mr. SHAYS. Because when you say it, you say in your statement on page 10, ‘‘Finally, let me be very clear. The strategy I have articulated for the revised acquisition process that will be prescribed in the DOD 5000 rewrite, and which is largely being implemented on the JSF, represents a real departure from our traditional approach to system development.’’ That’s the basis for my making that comment. I buy into the fact that it represents a real departure from our traditional approach but it’s still the word ‘‘largely,’’ you know. So just so you have a sense of why I get the feeling I get. Mr. Rapallo, you had a question. Mr. RAPALLO. I just had one followup on Mr. Halloran’s questions about the TRL levels. You said—I’m just trying to understand. The information that was provided to GAO was from the contractors, their evaluation, or from the program office? General HUOT. You know, I wasn’t involved in that process, but I know the program office is involved with the contractors. Mr. SHAYS. I didn’t hear the question. I would like to ask you to ask it again. Mr. RAPALLO. It was just a question of where the information came from, the TRL levels, from the contractor or the joint program office, or probably some combination of both. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00099 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 96 Mr. SHAYS. So what is the answer? [The information referred to follows:] The input for the TRLs reflects a consolidated answer from the JSF Program Office, incorporating input from the Boeing Company and Lockheed Martin Corporation. General HUOT. I believe it is, Mr. Chairman—I wasn’t involved in the exact process. I know the program office was involved, but I think the contractors actually did the work either with the program office or the oversight of the program office. Mr. RAPALLO. And your position was that the GAO asked for information based on TRL levels, asking also, including the risk of integration? General HUOT. Yes. Mr. RAPALLO. That information was provided? General HUOT. Yes. Mr. RAPALLO. And that would be incorrect, you’re saying? Basically, I’m trying to figure out if these numbers are too low because of that. Mr. SOLOWAY. That’s the crux issue here, that if you measure individual technologies relative to the risk of being able to integrate, or to integration, you come up with one answer. If you measure them as individual technologies relative to their technology maturity, you come up with the second. And that’s what the rescoring was. So in our view, the rescoring actually does raise the numbers, not to level 7, but they certainly raise it. Mr. RAPALLO. Are these the numbers that represent the rescoring or the initial? Mr. SOLOWAY. These numbers are the initial. Mr. RAPALLO. So the rescore numbers would be higher than this? The question is what are they I guess. Mr. SOLOWAY. I think all of them would be at five and six. Mr. SHAYS. I would like you to tell us what would they be at. Mr. SOLOWAY. I don’t know if I have the specifics on each of them. Why don’t we take that for the record and give you—I’m sorry. OK. These are our estimated ratings. Mr. SHAYS. Based on not having integrated? Mr. SOLOWAY. Right. These are what was referred to as ‘‘the rescored numbers.’’ Mr. SHAYS. Without the risk of integration? Mr. SOLOWAY. Right, they were not scored. We have—and I’ll just take them in order—7, 6, 6, 7, 6, 7, 6, 7. Mr. SHAYS. OK. Mr. SOLOWAY. And I have the rationale, if you just—I mean, I can tell you what the technology—but in order of these numbers, here again, as—I think General Huot referred to this a moment ago. One, flying on the concept demonstrator aircraft, which is we believe consistent, technologies already flown in aircraft helicopters or spacecraft, another one, such systems are similar or being used in similar commercial environments, including the Boeing 777, Federal Express Caterpillar, etc.; another one flying on the F–16; another technology flown on the JSF flying test bed; another one, the same, flying test bed; another one, flown on the JSF flying test VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00100 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 97 bed; and another one, flying on the CDA as well as the F–22, the F/A–18 and the Eurofighter. So in each of these cases, if you look at are we using the technology in a relevant environment, per se, they are all being demonstrated where—in that means. Now, that is the rescored numbers. Mr. SHAYS. But that’s based on your assessment, correct? Mr. SOLOWAY. It’s a DOD assessment. Mr. SHAYS. Right. And so in technology 7 you have a leap of two levels just by not having to integrate. And technology 1 you leaped up one, in technology 2 you went up two, technology 3 you went up two. In technology 4 you went up two. In technology 6 you went up two. In technology 7 you went up two. That’s a big jump. General HUOT. Again, because you’re trying to mature the technology without considering the risk of the integration on the JSF platform. Mr. SHAYS. So we can basically accept GAO’s assessment that they got—if we have a footnote that says it’s not integration, based on—excuse me, based on that, and we could accept your numbers based—excuse me, the other way around. They’re saying it’s not integrated; therefore, that’s the score. And you’re saying, if you don’t consider—I’m going to say it right here. Mr. SOLOWAY. I think I understand where you’re going. Mr. SHAYS. For my own sense of self-esteem, I need to say this again. The bottom line is, GAO gave you a lower score because of a failure to integrate. And you’re saying integration shouldn’t have been a factor in your score, and your score is higher without integration. Mr. SOLOWAY. You’re asking if we can accept this as—under that other definition, if you will, and I think I would have to say I don’t believe we can because there are other issues associated with it. Mr. SHAYS. You want a little conference here? Mr. SOLOWAY. We might want to take it for the record. But the table itself is confusing, and I’m not sure that we would precisely accept every level that was there. Mr. SHAYS. I welcome sometimes a little bit of qualification by two people in the same table. If that’s all right, General, do you want to just say what your sense is? General HUOT. I think once again that you go back to where these things came from. They were provided to the GAO, but those scores were based on considering the risk of integration on the JSF. When those scores were rescored again and provided to the GAO, they were without integration of JSF, the integration on the JSF, that’s when they go up to the appropriate level. Mr. SHAYS. But I just want to know if you accept their score when there’s—by—and look at the score that you have provided us, would they accept your score, in your judgment? I mean, in your talks with each other, was this really the big debate whether it was integration or not and you could pretty much agree on your numbers? OK. The answer is hard to figure out what they were saying when they did this, right? Mr. SOLOWAY. Let us take that for the record, sir. [The information referred to follows:] VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00101 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 98 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00102 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 99 Mr. SHAYS. Fair enough. You all have been very responsive. And is there a question you would have liked me to ask that you prepared all night for that you want to—I’m just going to say my staff gave me so much to read I stayed up all night. The statements have been very interesting on the third panel, as well as your presentation has been very, very helpful and, I think, candid. Mr. SOLOWAY. I’d just add one point, it’s not a question, and that is to go back to what we said earlier. The debate here is not over whether or not we do need to demonstrate and whether or not we’re committed to demonstrating technology maturity before we move into EMD. And that is a very important step in a critical discipline that we are instituting in our process with the JSF and in future programs. And I believe it’s very important, as we do that, to recognize what it takes and to recognize the ‘‘E’’ engineering part of EMD, which is the next step. So I would just reiterate that point again. Mr. SHAYS. I don’t know if I’ll be chairman next March but it will be very interesting. One of the things we do is, we put every statement on the record; then we have to live with it. I’m just—the problem is, when it comes to this kind of program, the same people aren’t always the same people that have to answer for the decisions that were made 2 years ago—I mean on the congressional side as well as your side. Mr. SOLOWAY. Sir, I just point out I won’t be here next March; that’s a guarantee. I think you saw the news this morning where both the House and Senate Armed Services Committees have included language requiring us to be able to demonstrate to the Congress that we have achieved technology maturity before we move into EMD. And we can live with that and have no problem with coming back to demonstrate that or document that for the Congress next year. Mr. SHAYS. That means low-level risk. Mr. SOLOWAY. Thank you. Mr. SHAYS. General, any comment you’d like to make? General HUOT. No, I think that covered it. Mr. SHAYS. We’ll end on that note. Very nice to have you both here. Thank you. We’ll get to our third panel. I appreciate the patience of our third panel. I have three panelists, if you would remain standing so I can swear you in: Dr. Thomas McNaugher, deputy director, Arroyo Center, RAND; Mr. Rodney Larkins, business development manager of 3M Corp.; Dr. Wesley Harris, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology. If the three of you would—we’ll make sure we get you there. If you would raise you right hands, please. Thank you. [Witnesses sworn.] Mr. SHAYS. Note for the record that all three have responded in the affirmative, and we’ll just go down the row. Doctor, we’ll start with you first. And actually we have two doctors here. I’m sorry. Dr. McNaugher. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00103 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 100 STATEMENTS OF DR. THOMAS L. McNAUGHER, DEPUTY DIRECTOR, ARROYO CENTER, RAND; RODNEY LARKINS, BUSINESS DEVELOPMENT MANAGER, 3M CORP.; AND DR. WESLEY HARRIS, PROFESSOR, DEPARTMENT OF AERONAUTICS AND ASTRONAUTICS Mr. MCNAUGHER. Thank you, Mr. Chairman. It is a pleasure indeed and an honor to be here. At the risk—— Mr. SHAYS. I will ask you to speak a little louder or move the mic a little closer. Can you pull it a little closer? Mr. MCNAUGHER. Somewhere between last night and this morning I have acquired a cold, so I am about an octave lower. Mr. SHAYS. I have absolutely no sympathy for you. Mr. MCNAUGHER. Let me start by saying I don’t know much about the JSF program per se. I was asked to speak here because I think I know a great deal about the weapons acquisition process, at least as it functioned during the cold war. I wrote a book about that. It came out in 1989 from the Brookings Institution. And in that book, called ‘‘New Weapons, Old Politics,’’ I examined what might be called some of the perverse patterns of cold war weapons acquisition, one of which was the tendency to rush weapons through development and into production. In particular, I focused less on the move from what’s called the ‘‘demonstration-validation phase’’ and more on the move out of what is now called EMD and into production, so-called ‘‘production concurrency.’’ So it’s not exactly the same issue that you’re dealing with today although I think you’ll find a lot of what I have to say about the incentive structure and the way we measure—— Mr. SHAYS. Could you just suspend for 1 second. I’m sorry. Thank you. Mr. MCNAUGHER [continuing]. Production concurrency or the tendency to substantially overlap the latter stages of development with the early stages of a move to high-rate production was a fairly common practice during the cold war. It was justified on two grounds. Technically it was justified on what might be called the ‘‘theory of declining uncertainty.’’ back in the ‘‘dem-val’’ phase. In a somewhat linear fashion, we reduce those uncertainties during development until, by the end or nearing the end of EMD, we should be able to move into production. On the other hand, on strategic grounds we were confronting a numerically superior enemy, so we justified the rush to production as a way of getting the jump on the Soviet Union, getting ahead. The faster you could get these new technologies out, the farther ahead of the Soviet Union you were, everything being equal. Now, when I started my book, I more or less agreed with those premises. When I finally started writing, I had come to disagree, especially with the first one, the theory of declining uncertainty. I would substitute what I call the ‘‘J-curve theory of program uncertainty.’’ True, at the beginning of a development project you have enormous risks, and you do, through ‘‘dem-val’’ and in the early part of EMD, neck those down to reasonable levels. But I encountered in almost all programs a rather sharp upturn in risks as the program actually went into production. Three reasons for that: One was systems integration. And I like the fact that you’re talking about systems integration this early in VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00104 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 101 the program. During the cold war systems integration often occurred very late in the development program, sometimes after production systems were already out in the field. And what you find, looking back at the history, is that even simple components stuck together in a new way produced new problems that you didn’t know were there, these problems often force a design iteration. The second problem that drove risks up was the move to production tooling itself. Again, even with simple systems, somehow the move to a production model from a preproduction prototype introduced risks, some of them great, some of them small, which almost always led to a subtle interaction between the producing engineers and the developing engineers. And finally, when you got this thing out into the field where real soldiers and sailors and pilots could use it, they invariably discovered new ways to use it that it wasn’t designed to do, and it started to break, and then you would have to go back and redesign it and fix those. This is, in a sense, a tribute to the ingenuity of developers and also users in the military, but it could be pretty embarrassing sometimes, the kind of breakage you could get. So all three of those then would appear very late in the process. To the extent that production concurrency ignored those risks, and to use the jargon of today’s hearings, to the extent that, in a sense, it wasn’t ‘‘knowledge-based,’’ it didn’t take account of these late-arising risks, we had chosen a fairly expensive way of developing weapons during the cold war. If you’d geared up for production, capitalized, and hired the labor, every delay, every change, cost a lot of money. If you already had systems in the field you were faced with this vexing question, do we retrofit the fixes to the fielded systems, which was always expensive, or do we just let those go? This led to what I would call the ABC approach to development. The A model of a lot of airplanes, and other things, too, often had serious design anomalies in it; the B model got it roughly right; and the C model is where you really wanted to be all along, but it took maybe even 100 or 200 production models to get there. So it was expensive. And when you did that, the average effectiveness of our overall force actually was lower than it would have been had you waited and gotten all of the production run, or most of it, to the C level. Now, we accepted those costs ostensibly in a desire to confront the Soviet Union. And you would think that now with the cold war over, as you say, Congressman Shays, we could relax and we could do this better. Let me make clear that the way you do it better is not by having a sharp divide between development and production. The point of my research was that the early stages of production are part of the development process. Again, to turn to the jargon of today, knowledge points 2 and 3 tend to fudge together and you don’t want to have a sharp gap. Rather, you had to treat the early stages of production in a way that with respect to late-arising uncertainties, perhaps starting very low, with a low rate of production, getting the systems out into the field, flying or shooting or driving the dickens out of them for awhile, and taking all of that information back and imparting it to your design before you really went to high-rate production. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00105 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 102 One of my colleagues at RAND years ago remodeled some of the Air Force aircraft programs of the 1960’s, 1970’s and 1980’s on that basis and concluded that you could probably get a more effective fleet overall for a little less money if you did it that way. Now, you added a little bit to the development cycle, which everybody thought was too long, but with the cold war over, that shouldn’t be much of a concern. So we might be in a position to move now to a more relaxed approach. I’m skeptical of our ability to do this, however, because I think that production concurrency was as much a political as a military strategy; that is, it was rooted in the politics that we see here today—in the politics of the acquisition process. Surely in the development of a new system, the move into late development and early production has to be the most vulnerable stage. You now have prototypes of the system, you fly them, drive them; the data almost always is going to contradict the optimism of early assessments. Sometimes you get tragic accidents, a helicopter crashes or a plane crashes. Sometimes you get funny accidents, you know—the gun that’s supposed to shoot a helicopter shoots the fan on a nearby latrine, as was the case with the Armys DIVAD. If this were purely a technical environment, everybody would have a good laugh, and they’d go back and redesign that system and fix it. But in the very charged political environment that can come to surround expensive programs, you know, it’s hard to desensitize that evidence. Knowledge and information become very dangerous. And delay—I’m thinking of your earlier discussion—you cannot convince a program manager that a 6-month delay isn’t the beginning of a mortal wound. How do you handle that? Well, in a sense you stack the deck. You raise the costs of slowing down. And you may even have production versions out there before you actually get that information. What I’m describing, in a sense, is the dilemma of weapon acquisition in this country as a political as well as a technical process. From a technical point of view, you really want to have a great deal of flexibility late in the process. From a political point of view, flexibility can be downright dangerous. So we tend to structure a certain amount of inflexibility in there. This is not a dilemma we have ever resolved very well in our history. If you step back from the cold war and look at the 200 years in which this Nation has bought weapons, you would argue that the problem wasn’t slowing the acquisition process down, it was getting it to produce anything at all. We would test endlessly and then not buy. Our technology generally lagged the Europeans, and as a rule, we had trouble getting things through the juggernaut of the political process. That ended with World War II and with the cold war. So while talking about political strategies may sound like I’m being critical, we ought to understand that for 50 years we produced and systematically modernized the best force posture in the world.? So it’s not clear to me that we can slow this thing down. And this is why I refuse to pass judgment on the patterns of acquisition I studied in the cold war. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00106 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 103 We are now in a unique period in the Nation’s history. We are not confronting a great Soviet threat, as you’ve said, although the world remains dangerous; moreover, we remain engaged, and there are threats out there. So we’re sort of at the Goldilocks situation, you know, it’s not no threat, not a big threat, but sort of right in between. And the question is, can we come to the Goldilocks solution for weapons acquisition? That is a process that is relaxed enough to take account of late-arising uncertainties, but not a process that goes to sleep and doesn’t produce anything. And that returns me really to the first paragraph of your statement, Mr. Chairman, which talks about the emergence of a consensus about how we’re going to handle this. This hearing focuses on the JSF specifically, but the JSF, as you recognize, is the beginning of a wave of new modernizations and recapitalizations after a 10-year procurement holiday. So this hearing is also part of the way in which we construct this consensus, and it remains to be seen how that will work out. Thank you. Mr. SHAYS. Thank you very much. Your book was the reason why you’re here and your testimony justifies your presence. [The prepared statement of Mr. McNaugher follows:] VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00107 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 104 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00108 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 105 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00109 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 106 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00110 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 107 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00111 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 108 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00112 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 109 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00113 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 110 Mr. SHAYS. Mr. Larkins. Mr. LARKINS. Good morning, or I guess I should say good afternoon. Mr. SHAYS. Good afternoon. You’ve been wonderful to be so patient. But I learn as much from the third panel, and sometimes more. So from my standpoint, I’m very happy you are here. Mr. LARKINS. Thank you. As business development manager for government research programs at 3M, I’m here to discuss 3M’s ‘‘best practices’’ associated with our new product commercialization process and the timely integration of breakthrough technology into ‘‘change the basis of competition’’ products. 3M is a diversified manufacturing company with sales of more than $15.5 billion. We manufacture a broad range of products directed at six distinct commercial markets. 3M has grown by pioneering innovative technologies and creating new products with these technologies, thereby creating new markets and revolutionizing existing ones. 3M has a written policy objective which states that 30 percent of all product sales for a given year come from products that were introduced in the 4 preceding years. Thirty-four percent of 3M’s total sales in 1999 came from products in this category. It is these new products which help sustain the profitable growth of our corporation. To maintain this rapid pace of technology development and new product introduction, 3M has evolved a well-defined technology development and commercialization process. A basic tenet to successfully introducing new products is the discipline to make the up-front investment in research and development. 3M has invested more than $1 billion per year over the past 3 years in research and development. This investment is directed both at broadening and strengthening 3M’s existing technology portfolio, as well as moving products rapidly through the commercialization process. Currently, approximately 20 percent of 3M’s research and development budget is directed at technology development and enhancement. Approximately 80 percent is directed at product scale-up and commercialization. This investment ensures the availability of critical technologies to turn into products, and the product development resources to focus on the corporation’s priority programs. In our quest to maintain technical and market leadership in the markets we serve, we have evolved into a laboratory structure which substantially segregates technology development from product development. Our experience has been, that the key technologies are used broadly across the corporation’s market groups. By focusing on technology development in a technology center, a critical mass of technology experts is created, and costly redundancy is eliminated. 3M has formally established 14 technology centers for the corporation’s most pervasive technologies. The responsibility of these centers is to establish and maintain world-class capabilities in these critical technologies. The second responsibility is to work with product development teams in the business units to integrate this technology into the product development programs to assure successful and timely product introduction. This step is critical, and a number of management tools have been established to en- VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00114 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 111 sure that it takes place. 3M is acutely aware that even our best technologies, when not applied to timely commercial product development, are of no value, and we work hard to maximize that application. Critical to the commercial success of our company is our ability to select and to focus on programs with the best possibility for changing the basis of competition in large market opportunities. Despite 3M’s large investment in research and development, our resources are finite and there are always more opportunities than there are resources. Important steps in program initiation are opportunity identification, development of a comprehensive product or system description, a business assessment of projected results once success has been realized, and a technology assessment which indicates that technologies are in place to meet performance requirements. All of these elements must be in place before a program is approved for scale-up and commercialization. At the initiation of the product development process, a formal review is held to assure that all elements are in place for success. Hence, success on these key programs is critical to the success of the corporation and our goal is to put these programs in the very best possible position to succeed. Once under way, the development team follows a detailed new product introduction plan which outlines all elements which must be accomplished during the product development process. Periodic program reviews are routinely conducted with the emphasis placed on encouraging and rewarding candor on the part of the product development teams and identifying and eliminating impediments to the success of the team. In addition, corporate review teams are employed. These review teams bring together experts in product development from across the corporation to identify potential roadblocks to successful commercialization and to identify resources to eliminate those roadblocks. In summary, 3M’s approach to technology integration and product development has proven to be very successful. This process, however, is far from being perfect. It’s a process that we will continue to evolve, and our goal, of course, is 100 percent success on all of our priority programs in the corporation. Thank you, Mr. Chairman. I’ll be happy to answer any questions. [The prepared statement of Mr. Larkins follows:] VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00115 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 112 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00116 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 113 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00117 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 114 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00118 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 115 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00119 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 116 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00120 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 117 Mr. SHAYS. Thank you, Mr. Larkins. You work for, I think, a pretty amazing company. When I think of your company I think of it as—though very large, it has tremendous innovation as a small company might have. And it’s great to have you here. I would love to know how we can seek comparables between what you do and what we need to do in government in general, and specifically with defense. Dr. Harris. I just want you to know, sir, I’m always awed when I see a ‘‘doctor,’’ and I see MIT next to it. Mr. HARRIS. Thank you. Good afternoon, Mr. Chairman. I very much appreciate the opportunity to address this distinguished subcommittee on—— Mr. SHAYS. I would ask you to move the mic a little closer to you. If you can turn it over the paper, as long as it doesn’t get in the way. Does it get in the way? OK. That’s perfect. Mr. HARRIS. I’m fine. Thank you. Again, I very much appreciate the opportunity to address this distinguished subcommittee on certain issues related to technology maturity and acquisition reform. I wish to state at the outset that I approach this important topic from the perspective of both an academic researcher and as a former government manager, not as an engineer practicing within the defense industry. It is my view that our defense acquisition policy and practices are complex. This complexity is a result of several factors, including a dynamic or shifting defense industrial base, a declining acquisition budget, a constantly evolving threat environment, a diverse force structure, and most importantly, an increasing rate of change of technology. The impact of the last factor, namely the increasing rate of change of technology, on technology maturity from an acquisition policy perspective, is difficult to overstate. The impact is made more profound when the global nature of much of this technology is considered in the acquisition of new weapons systems. I would like to add to this list the following elements that have impact on the acquisition of defensive systems: the current focus on greater life cycle value, the emphasis on more rapid deployment, the emphasis on upgradability, sustainment and maintenance. Mr. Chairman, while noting that today this subcommittee is addressing a specific program within the defense acquisition arena, I wish to state for the record that the impact of acquisition reform reaches beyond the procurement of defense systems and its related technology. Through our defense industry base, acquisition reform drives our national economy and impacts world peace, in short, our success in developing an effective and efficient acquisition strategy that captures mature technology, exposes the risk to control our future, to produce wealth and to continue to contribute to the advancement of humanity. Based on my research and government experiences, I wish to share with this subcommittee today several national successes. First, there exist several case studies of successful acquisition of defense systems in production. Second, there exist case studies of successful parallel development of advanced technology to high maturity levels where the government is customer. These two successes have many things in common, and I believe are related to VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00121 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 118 today’s issues of acquisition reform in the environment of technology maturity. My success or my research on economically incentivized contracts focused on several important programs including the Sensor Fused Weapons system, the Joint Direct Attack Munitions program, the C–17 program, C130J, the F–414 engine, development that goes within the F–18E/F airplane and the F–117 engine that goes in the C–17 airplane, as well as the Boeing 757 airplane. These programs were in production, were able to develop an economically incentivized contract or a win-win solution for both government and contractor. A few comments now on the development of a critical complex technology in advance of full system acquisition: In the early 1990’s, NASA’s Office of Aeronautics did develop and manage two technology development programs. These were the high-speed research program and the advanced subsonic technology program. During the same time period, NASA also worked jointly with industry and DOD to develop advanced gas turbine components within the integrated high-performance turbine engine technology program. These three programs of technology development were successful, and as stated in the prepared statement, over 12 reasons why they were successful. Mr. Chairman, the subcommittee may wish to note the very strong commonality between the factors leading to successful development of technology parallel to full-system acquisition and the factors that enable a win-win solution for programs already in production. At the most fundamental level, the environment for favorable development of advanced technology is very similar to the environment for acquisition of defense systems in production where technology risk is low, corresponding to technology at a high maturity level. These programs, both advanced technology development programs and acquisition of full defense systems, strongly suggest that advanced technology at a high maturity level is essential to the acquisition of affordable systems with requirements for superior performance. The importance of advanced technology at a high maturity level is so great, in my opinion, that the government must incentivize the contractor to develop advanced technology. This means that the government must place a premium on the development of technology to a high maturity level. The premium must compare favorably with other awards available to the contractor. The economic realities of a high premium will, or at least should, drive the government to a lean portfolio management condition. Advanced technologies selected for development to high maturity level must be, or should be, based on realistic projections of need. Mr. Chairman before concluding, I would like to add a few additional comments. I wish to note that to improve the chances of successfully developing advanced technology to a high level of maturity, one or we, those involved, should adopt common, quantitativebased language and assessment tools. Qualitative prescription of technology readiness, such as the technology readiness levels, are insufficient and inconsistent with realistic projections of need. The VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00122 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 119 desired quantitative-based language and assessment tools are recommended to be derived from probability in each technology readiness level. The TRL would be expressed or defined by probability bands. In conclusion, we, as a Nation, have demonstrated the capability to produce weapons systems driven by advanced technology. My colleagues have clearly confirmed that. To become more efficient at the acquisition process, we must continue to develop advanced technology in parallel with acquisition of full, complete systems. Key elements again in this efficiency are to incentivize industry with favorable premiums, a realistic projection of advanced technology needs, and the use of quantitative-based language and assessment tools. The bottom line is that government and industry know enough about each other and about advanced technology development to make affordable acquisition of full systems and technology insertion work, to make it work efficiently. The health and wealth of our Nation depends upon this working very efficiently. I would entertain questions, if there are any. [The prepared statement of Mr. Harris follows:] VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00123 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 120 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00124 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 121 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00125 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 122 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00126 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 123 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00127 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 124 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00128 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 125 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00129 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 126 VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00130 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 127 Mr. SHAYS. Thank you. There will be questions. And I’ll just throw it out to all of you. First off, you were all here for the entire hearing, and I thank you for that, because the value is that you can comment without me having to repeat the questions. Conceptually, a ‘‘best practice’’ model for acquisition that makes sense in the private sector for private development also makes sense for the private sector for a public customer. Would you all agree with that? And if not, if you would, qualify it where you would—I mean conceptually. I make the assumption nobody would disagree. Mr. MCNAUGHER. You’re saying that the commercial best practices is transferable to the public process of—— Mr. SHAYS. Yes, and that we should seek to use commercial best practices. Mr. MCNAUGHER. The only caveat I would raise, Mr. Chairman, I mean, obviously no commercial firm operates in the kind of political environment—— Mr. SHAYS. Just a little louder. I know you’re not feeling well, and I said I wasn’t sympathetic; but I didn’t want to give you sympathy because I didn’t want it to get worse. Mr. MCNAUGHER. There’s a lot of differences, obviously, between buying weapons as a public good and developing a private sector item. The only difference, though, that leads me to question the use, the extensive use of best practice models is I don’t think in the commercial world very often you systematically take these huge leaps out into the unknown, as we did during the cold war and, I think, still do; that is, not only the components of a system, but the integration of a system really represents a dramatic jump. Mr. SHAYS. But you don’t have any fear that we’re doing extensive use of this now. I mean, we’re not, this is the—I’m not aware that a commercial best practices model is running rampant through our government. Mr. MCNAUGHER. No. What I’m saying is, trying to apply it to the government, to this process, has limits. Most commercial firms insofar as they’re doing marginal improvements are aware of risks and costs at a fairly good level of certainty. You know, most of the discussions of risks in the early phases of cold war development programs proved in the long run to have been wrong. You don’t know what the numbers really are. You have an idea of a cost performance curve—— Mr. SHAYS. And they were almost always understated. Mr. MCNAUGHER. Yes. That’s not an indictment of anybody’s behavior. Technology just doesn’t give up its secrets very easily. So we have these very precise discussions during ‘‘dem-val,’’ but, the numbers and curves are going to shift. That’s again why you need more flexibility down the road. I think commercial practice may not have to deal with that kind of uncertainty. Mr. SHAYS. OK. We’ll touch on that a little bit more in a second. Mr. Larkins, when I think of 3M, I would think that you use a knowledge-based process, a best practices model for acquisition as defined by GAO when they talked about knowledge, point one, VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00131 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 128 match is made between the customer’s requirements and the available technology. Mr. LARKINS. Yes, sir. Mr. SHAYS. Knowledge, two, when the product design is determined to be capable of meeting performance requirements. And three, knowledge point three, when the product is determined to be producible with cost schedule and quality targets. I would think 3M would just be right in the center of that kind of philosophy. Mr. LARKINS. Yes, sir, that’s correct. Mr. SHAYS. Where it might differ is that I think of 3M, with no disrespect, but just in terms of putting it in perspective, that you are seeing customer needs and then you’re seeing how you can meet those needs with products that you have potentially available in technology that you have available. I make an assumption in most cases that technology is pretty developed, or sometimes do you have to—do you see a need and you just start out from scratch with a technology? Mr. LARKINS. That happens, sir. Mr. SHAYS. What happens? Mr. LARKINS. Where we will, in fact, start from scratch and develop the technology. A lot of it depends on how broad your window of opportunity is in the market. What we are finding today is that if you don’t meet a market need quickly, then the market need is met by someone else. So what we try to do is do a rapid technology assessment, determine whether in fact we have technologies in house to meet that need. If we don’t, then we will go out and make a technology partnership or acquire a technology. Mr. SHAYS. And admittedly the—so time is of the essence? Mr. LARKINS. Absolutely. Mr. SHAYS. But your technology requirements may not and certainly wouldn’t be as complex as developing an aircraft or—— Mr. LARKINS. That’s very fair to say. Mr. SHAYS. So we need to have some empathy for the task at hand, that applying best commercial practices is a very important effort, but certainly we can carry the commercial to the defense; and we may be—it may be unfair to say that you—DOD needs to do it just like we do it in the private sector. Mr. LARKINS. Yes, I agree that it would be unfair to say that their technology integration problems are as easy as ours, because they certainly are not. And—but I would like to agree with my colleague, Dr. Harris. I think one of the reasons we are successful and are able to achieve the type of record in product integration scale-up that we do is because we do have a significant investment in technology. We have a broad base of technology platforms that we can tap into, and they’re ready to be integrated. Mr. SHAYS. But it would be unlikely for 3M to go into development and production without the technology there to back it up? Mr. LARKINS. We absolutely would not. Mr. SHAYS. You would not. So the general principle that there is significant logic in developing the technology before you went into development and production still holds for the defense as well. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00132 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 129 Mr. LARKINS. Well, all I can say is that we do not go into a product scale-up mode until we have the technologies in hand. Mr. SHAYS. So, then, if you can’t meet the deadline, you just don’t produce the product? Mr. LARKINS. That’s correct. Mr. SHAYS. Dr. Harris, do you want to comment on the questions I have asked so far? Mr. HARRIS. Yes, I would like to comment, sir. The use of the words ‘‘commercial best practices’’ and our consideration of the—this environment of acquisition of defense systems, that phrase is a very loaded one. It has many meanings and usually it differs, depending upon the speaker. Mr. SHAYS. Is that—if I could interrupt—is that because there is no, one, so-called ‘‘commercial best practice’’; there are—different companies have their version of best practices? Mr. HARRIS. Well, that’s somewhat it, sir, but I think—my statement is based primarily on the fact that there’s no commonly agreed-upon body of knowledge as to what it really is. It’s a sort of catchall phrase that has caught many up into some more political stance than real substance. For example, the question of scaling along technology complexity, the question of scaling in terms of number of products, the questions of availability of markets all would impact commercial practices. And you don’t have a serious discussion of commercial best practices being transferable along what is in fact scalable. So I’m somewhat personally concerned as to whether—what we really mean when we say that DOD or the government ought to rush immediately into commercial practices. It just doesn’t have the substance that I think it must have in order to hold those who practice within government accountable when we say that they ought to use or should be using best practices. Mr. SHAYS. So you would be more sympathetic to the view that— that when we use TRL, technical readiness levels, that it be more a tool than a requirement. And you heard them refer—— Mr. HARRIS. Yes, I did hear that discussion, sir. Yes, Mr. Chairman. I think we’re wrapping ourselves around an axle with the qualitative discussions of technology readiness levels. Until we can move to a position where we can quantify what we mean by technology readiness, we can’t hold those who practice, who acquire the weapons systems, accountable. Because they and their contractors will always be at each other’s throats as to what they mean when they present an argument or position on technology readiness levels. So I’m calling for more quantitative language and assessment tools to enable all of us to understand precisely where we are within this arena called ‘‘technology readiness levels.’’ Mr. SHAYS. That’s spoken like a true academician. You’re fulfilling your role perfectly. Well, one of the things is—and I’m not burdened by being a lawyer, and I don’t have any perceived—and the reason why I love this committee and love the work as a legislator, I’m not trying to prove a point. I’m trying to understand what we as a committee need to recommend to the full Congress and to the executive branch. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00133 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 130 But intuitively, I have accepted as a fact—and, Dr. McNaugher, I’m going to try to have you separate a little bit the so-called politics side of this, and then we’ll get back into that. But I accept as just very logical, without a lot of empirical knowledge, but just your own life experiences, that if you are going to be developing technology and development, and fail to come to grips with some technology; and then you’ve got a point where the technology hasn’t been resolved and yet you’re still moving things along, that your costs are going to go up significantly. You’re going to begin to slow down your development after you started, particularly in production. And I accept the point that you made, Dr. McNaugher, that sometimes we haven’t perfected the product until we have produced 100 planes. And that’s a real negative. So, Dr. Harris, tell me why intuitively I shouldn’t accept the fact that we should develop the technology before we go into development, but particularly into production. Mr. HARRIS. Well, I can’t tell you that. I want my testimony to read very clearly that we—— Mr. SHAYS. Don’t put words in your mouth. Mr. HARRIS. We as a Nation have, in my opinion, been successful in developing technology in advance of acquisition of the full system. My experiences at NASA with the high-speed civil transport, the new 700,000-pound Mach 2.5 airplane, we developed that technology with industry as partners. The airplane was never built because it was a business decision, not because the technology was not there, or was designed to be there, in advance of building the airplane. So we know how to build or how to develop technology in advance of acquisition. Mr. SHAYS. But the question is, is it going to be cost effective? Mr. HARRIS. Oh, I would say it definitely is cost effective. And the effectiveness goes up, I believe, with the complexity of the final product. If you’re talking about very complicated systems that require a real stretch in technology, then clearly the effectiveness is going to be there, the cost effectiveness. Mr. SHAYS. I’ll allow the other two to jump in in a second, but where I’m not with you is, I would think that waiting to develop the technology before you did the development, but particularly production, would be to your advantage as long as you don’t have the time restraint of having to get into the marketplace because someone else is going to build the product. Mr. HARRIS. Well, I agree, as I know the threat, at least the military threat, that this Nation currently faces, we have that time. Mr. SHAYS. Right. OK. Any other responses to any points that either of you, Mr. Larkins or Dr. McNaugher, any comments you want to make? When you heard the debate between—the dialog between the first panel and the second panel, I want each of you to tell me your reaction not in—as concise as you can; and tell me how you reacted to the two different testimonies, because I think both testimonies were quite excellent. I think GAO gave us a wonderful vehicle in which to have this debate and dialog. But I would love to know how you reacted as you heard it. No criticism of the committee allowed, just the other side. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00134 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 131 Mr. HARRIS. Well, I’ll go first. Mr. MCNAUGHER. Yeah, feel free. I’m happy to defer. Mr. HARRIS. I thought the first panel was again a very excellent presentation. I think there was an honest mistake in interpreting what technology readiness levels were meant. Was it to be on a full-up system or a comparable system? And that led, I think, to a downgrading of the so-called ‘‘technology readiness levels.’’ What I thought was missing in the first presentation was a normalization of the two possible streams to the same system, one stream as to what we currently have, or at least what the first panel proposes that we have, that we buy this complex weapons system and develop the technology while we buy it. Another stream would be to develop the critical technology off stream, then insert it when we are ready to move forward with the full system. Now, my question is this: What is the cost going along the first stream and what is the total cost going along the second stream? And what are the time lines involved? When does the clock start? Mr. SHAYS. How would we know that if we don’t know if we can actually develop the technology? Mr. HARRIS. As far as I know, sir, no one has ever done a comparative analysis of that sort—unless Dr. McNaugher has done it. Yet we get this constant bombardment of how to develop systems and how to develop technologies that support systems off line. But the answer is, Mr. Chairman, no one really knows. Mr. SHAYS. What I heard, and then I’ll open it to the other two of you, when I found myself wondering if moving forward with development represents—where you still have high risk on technology, whether that represents a significant negative. But I clearly feel that if you went into production without the technology, then we are really opening up potential costs—I mean, that ratio. So I found myself more tolerant of DOD kind of wanting to go to that development stage. I would become very concerned if they wanted to go that one step further. In other words, I think there’s still this high risk in some of their technology. Mr. HARRIS. OK. I don’t know the specific technologies involved with this weapons system, as I am not a consultant to any of the primes. But there are two different ways of getting to the same end goal; a highly complex system is what we don’t have a comparative analysis to. Mr. SHAYS. Fair enough. Dr. McNaugher. Mr. MCNAUGHER. First of all, I don’t know enough about the TRL methodology to comment on it explicitly. I think it’s absolutely appropriate to be hammering on the readiness of these technologies. The disagreement between the two prior panels—and not everybody agreed that there was a disagreement—seemed to circle around the question of whether you’re assessing risk in the context of systems integration or the risk of specific technologies. I can at least understand why you get a different number in each of those cases, given what I said earlier. And so the one upshot I would have, and I think I’m echoing something Dr. Harris said, is whenever you move or this project moves from ‘‘dem-val’’ to EMD, don’t think that just because you VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00135 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 132 have reached TRL 7 for components, you’re out of the woods. There’s a lot of development work and there’s going to be a lot of sometimes nasty surprises up ahead. Mr. SHAYS. And I don’t think GAO was suggesting that. Mr. MCNAUGHER. Agreed. So I can understand why there might be a different measure, depending upon whether you’re thinking of the specific technology or that technology in the context of an overall system, because the system does pose its own uncertainties. Mr. SHAYS. Mr. Larkins. Mr. LARKINS. There seemed to me to be two basic issues that came up. I think—— Mr. SHAYS. Can you move the mic just a little closer. Mr. LARKINS. There seem to be two basic issues of disagreement. I think there was an awful lot of agreement, but there were two basic issues that I got that there was disagreement. One was that—one was the TRL as—the definition used in each individual category. And again I have to agree with Dr. Harris here; I think that if you’re going to use categories to evaluate a project, you should have general agreement by everybody, the people who are being evaluated and the evaluators, as to what those categories are and where they fit. And there obviously wasn’t any there. The other disagreement was that there is a substantial difference between a fully integrated system with eight apparent critical technologies versus evaluating each of the individual critical technologies. And frankly, at 3M, when we look at integrating several new technologies into a program, if we consider, for example, that each technology has a 90 percent chance of success, then you have to do a multiple of those chances of success to arrive at a final chance of success for your program. So if each of your programs has a 2 percent chance of failure and you have—or each of your technologies has a 2 percent chance of failure and there are four technologies, you’re looking at a 16 percent chance of failure right there. So I am completely sympathetic with what the Department of Defense is saying here. Mr. SHAYS. And yet you would be sympathetic with the fact this integration sometimes may mean that the technology doesn’t work? Mr. LARKINS. That’s correct. And that’s why we frankly take a very conservative view of that approach. We—and believe me, we do take risks, but they are calculated risks; and the greater the risk on a commercialization program, the tighter we monitor it. Mr. SHAYS. Do you have any questions? Mr. Chase would like to ask questions. Mr. CHASE. Mr. Larkins, what kind of incentives do you provide your teams, your project teams, to ensure that a new product that you’re trying to develop comes in on time and within budget? Mr. LARKINS. Well, there are, in my view anyway, three different types of incentives, peer recognition and reward—and we have programs in place which recognize individuals and teams for outstanding success in product introduction—and, of course, promotion. And when a person is a program manager and they’re involved in a successful product introduction, there are certainly promotions that are involved in that type of an activity. And finally, financial incen- VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00136 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 133 tives as well, which go along with the promotion and separate from that. So recognition and reward and promotion. Mr. SHAYS. Thank you. I was going to ask that question after you did. Because I just wonder if in the—the Federal Government, when they are utilizing the services of a private company, whether those companies themselves have their own internal programs to encourage as much innovation and extra effort and so on. Let me say this to you. I’ve really asked the questions I wanted to ask. I’m going to allow all of you to make some closing comments or ask yourself a question you wish I had asked, that you think I should have asked. I would also like to just ask if Mr. Rodrigues would mind coming to the corner here, just to kind of share your observations. Is there anyone from DOD that was here that would like to or— if they hear a comment—do we have a representative from DOD who would be? We had let them know they would be invited if they were able to stay. I realize they couldn’t necessarily stay. If you could, just sit on the corner here, if you could do that. Are there any comments that you would like to make or observations? If so, I would move the mic over to you. This is not—the purpose of this is not a debate, but just to say things that you would want us to focus in on. Mr. RODRIGUES. I guess I would like to say a couple of things to try to add a little clarification. The work that we did on the Joint Strike Fighter and the application of the TRLs, being able to do that was the culmination of years of work. It was not easy to define a model that reflected what we defined as, at this point, ‘‘commercial best practices’’ that can be applied in DOD. That was gleaned from a lot of work with a lot of companies, and working with the Department looking at successes and failures and trying to categorize what people do to come up with knowledge points that are critical to being able to move forward successfully. When we put that over at the Department of Defense for comment, the original comments that we were getting back were very, very negative because it would require a significant change in the way they do things. Dr. Gansler, coming on board, he took that on and rewrote the comments; and I can tell you I never get comments like this. At the end, he writes a personal note, on the formal comments coming over, ‘‘A very thoughtful and helpful report, Jack.’’ Believe me, I never get these. What we then did is, we said, OK, if you agree with—if you could put that chart up, please. Mr. SHAYS. You know, it’s interesting, just a nice little comment like that goes a long way. We all need to do it, all of us. Mr. RODRIGUES. That’s the only one I’ve ever gotten. So if you look at this chart, then we said, OK, if everybody can agree with this—and we had to come up with a model, run it back by all the commercial companies we did, because they didn’t know how to define what they did. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00137 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 134 They don’t think about it in this term. Nobody had a model for us to use. We had to work and find it. Find it, define it, get it understood by both parties, DOD and the commercial guys; and let’s agree to some things. Once we got agreement on that, we went back and said, if this is true, let’s peel the onion. If technology development, separating technology development from product development is so critical, how do you do it? Mr. SHAYS. I need to know what your bottom line is. Mr. RODRIGUES. My bottom line is, we did a whole study to come up with the technology readiness levels. We didn’t get there by accident. We did a whole thing focusing on knowledge point one. And we worked with a lot of individual technologies, both commercial and defense. What we found was that they work, that they can predict outcome, that if you meet the right technology level, you can get success. If you don’t, you are virtually guaranteed failure. Mr. SHAYS. You’re a brave guy to say that right next to Dr. Harris. Mr. RODRIGUES. I’m not guaranteeing there aren’t better ways to do it. I’m just saying they are a great indicator. Now, where I was going with this was or trying—— Mr. SHAYS. Just don’t get too deep here. Mr. RODRIGUES. In that report, before we applied TRLs to the JSF Program—see, we applied it in the abstract and didn’t criticize anything—we never said anything. We said, here they are. Here’s a tool. Isn’t it wonderful? Can’t you guys use this? Doesn’t it make sense? When we defined it in there—and this went to the Department for comment, and I’ll read you their comments. We said—we defined TRL 7. We were talking about where does integration fit. There are two types of integration, there is subsystem integration and there is product integration. Mr. SHAYS. OK. Mr. RODRIGUES. We got a whole bunch of technologies that are at subsystems. Integrating proven technologies into a product is a real challenge, and it is something that has to be done. Matching technology to requirement up front doesn’t stop that from happening. You still have to do it. It’s a challenge. What we’re trying to do is isolate that, because what should be done in product development—isolate that from the technology development which is proving the technologies in the subsystem forms in which they have to be, so you’re not trying to do that concurrently, especially on pacing items. So we said when radio components—this is in the TRL, report—are assembled inside a case that resembles a final radio design and then demonstrated aboard a surrogate of the intended aircraft, the radio reads TRL 7 now when they comment on this report, they say that the Department agrees—this is their writing, it’s not mine, reprinted in the report—the Department agrees that TRL is an important input and is necessary, but adds they’re not sufficient alone to decide when and where to insert new technologies in a weapons system programs. Military system development decisions require a total ownership cost approach through the entire life cycle system. Now I totally VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00138 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 135 agree with that. We weren’t trying to imply that once you reach 7 you have to put it on a product. We were saying if you don’t reach 7, you’ve got problems and you shouldn’t put it on a product. They went on to say in their comments—— Mr. SHAYS. I want you to bring this to a close. Mr. RODRIGUES. This is their comment. The Department concurs with GAO that the weapons system program manager should assure that technology is matured to TRL 7—I just read how we defined it in the report and they agreed—before insertion into a new system. They agreed with that here. The difference in the JSF study was we then cross-walked that, and said, hey, you know what? When you apply this on the program you’re doing, you’re not there and you shouldn’t move ahead. It wasn’t until we made that cross-walk in a draft report for comment that all of a sudden this issue of a misunderstanding, came up. It came up once they saw how it was used. If they had read this report in detail, which I had provided to them, they would have seen how it would be used. Mr. SHAYS. Let me just say I think they did read the report. The advantage that GAO or Members of Congress have is that we can look at something that someone else is building or doing, and we can sit with hindsight and all the other things, and really be pretty analytical about it. I think your report was done very tastefully. I think it served a tremendous use here and will be used by other committees because, as you know, your report was used by the Armed Services Committee even before we released it today. I think DOD was fairly respectful of your analysis. I think there are—I have a sense of where our disagreements are. And a lot of good is going to come from the report. And—but I’m excited that this is a program being used by Defense. I want to make sure that they’re using it to the fullest extent possible. Let me just—I’ll let you say another comment here, but let me just ask Dr. Harris or Dr. Larkins or Dr. McNaugher: Any question you want to ask yourself and then answer brilliantly? Mr. HARRIS. Mr. Chairman, no question, but I would like to conclude by simply repeating myself. We, as a Nation, have demonstrated the capability to produce weapons systems driven by advanced technology. To become more efficient at the acquisition process, we must continue to develop advanced technology in parallel to acquisition of full, complete systems. Key elements in this efficiency are, No. 1, incentivize industry with favorable premiums, develop this advanced technology; No. 2, realistic projections of advanced technology needs, meaning a management of our DOD technology advancement portfolio; third, use of quantitative-based language and assessment tools. Mr. SHAYS. Thank you very much. Mr. MCNAUGHER. Three points, the first taking off from Dr. Harris’ last remark; and it’s a subject of a different hearing, but one that should be kept in mind. We have never found how to make R&D per se profitable in the defense industry. The historic tendency is, firms sink their own money into development and get well in production. If you want to relax this process and maybe even cancel the occasional system, VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00139 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 136 you probably want to have a way, a mechanism, for making R&D profitable. We’ve never found that, and that’s the way you encourage industry. It’s a profound problem in the defense industrial base, precisely in this area we’re moving into. My second point has to do with the earlier discussion of the tenure of program managers. I must say I’m fascinated with the idea of a program manager who actually has to live with the results of his or her decisions 5 or 6 years later, I think it would be a profound change. Again, though, stepping back and looking at the full sweep of American history, remember that before World War II the Army had an Ordnance Department which was a bunch of full-time program managers and technocrats, if you will. They wore uniforms, but they didn’t rotate. I think the Army’s conclusion, as an institution coming out of World War II, was that the Ordnance Department was not attentive enough to the user, the actual operator. And it was more fixated on the technologies than on military capability. What the Army did in the 1950’s and 1960’s was begin to substitute line users as program managers, and but also as the chief of R&D, for example. Mr. SHAYS. Wasn’t there also another danger that they developed too cozy a relationship with the organization they were buying from? Mr. MCNAUGHER. Well—— Mr. SHAYS. Over time you just began to become almost their advocate, not necessarily their—— Mr. MCNAUGHER. My sense of the ordnance department, to the extent that I understand the history, is that it had a somewhat thorny relationship with commercial firms because it was doing its own R&D, and it was always being besieged by Members of Congress pushing Samuel Colt’s pistol or rifle. So it was a more combative relationship. Mr. SHAYS. So—I didn’t realize, they did the R&D; I just thought they were—— Mr. MCNAUGHER. Springfield Arsenal. The arsenals did R&D, so they were competing with industry to some extent. But the point is, a program manager tenured for 15 years or 10 years may become very attentive to sort of the permanent features in the environment, of which the political structure is one, and not as attentive to the military user. So maybe there’s a length of tenure that sort of balances those needs. I don’t think it’s 3 years either. I think it’s longer than that. And finally, at the risk of complicating things, I would just separate technologies into two bundles, slow-moving and fast-moving technologies. If you look at an airplane, the engine airframe combination had its fast-moving days back in World War I and the 1950’s, and that’s slow-moving technology. You can afford to stop, assess that technology, test it. If you look at the electronics in the avionics—— Mr. SHAYS. Let me just interrupt you a second so we can ask a question of the recorder. Thank you. Mr. MCNAUGHER. If we look at avionics in the cockpit of that airplane, which may come to 40 or 45 percent of the cost of the air- VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00140 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 137 plane, we’re looking at technologies where if you stop and test and decide, you know, they’re already obsolete—I mean, they’re turning over in a year, 18 months, 2 years—and I’ve always felt that we really need—now that electronics technologies are seen to be the key to the revolution in military affairs we really need to handle these by different commercial practices. One can be much slower and more judicious because you’re way out on the flat of the cost performance curve; you’re pushing for that marginal improvement in jet engines or airframes. The other is just so fast-moving that you almost have to be turning your design and your tests over constantly. The DOD needs a different approach entirely. Mr. SHAYS. I thank you. Mr. Larkins, I knew that would bring you out. Mr. LARKINS. If I could just make one. Mr. SHAYS. Would you move the mic, please, toward you. Mr. LARKINS. If I could just make one additional comment, I would say that our experience in industry with the importance of focusing on fewer larger programs, that having trained people who are—whom we call program managers or program development managers, whose expertise is scaling up new programs. We are beginning to see within industry a focus on that as a discipline. 3M traditionally has what we call a dual-ladder system where we have technology people on one side of the promotion scale and management people on the other side of the professional scale. So you have research and management moving up together, the same opportunities for advancement, the same opportunities for pay, this kind of thing. But we are now in industry beginning to see a third leg or a third ladder in this process, which is a focus on program management. Mr. SHAYS. Any other comment? Mr. RODRIGUES. I would like to make one, because this becomes a real area of concern; it is related to this. I have started looking at some data really hard-looking at this whole issue, this issue of profitability for research and development. In private industry, research and development is investment, the return is on sales. You don’t make money in R&D. You make money building a product and selling it to somebody. The same principles should apply in the Department of Defense and should apply in the defense industry. The reward for doing the right thing is you sell us something that you make a profit on. When I looked at this issue and was trying to study this, what I find is when I look at tactical aircraft, from 1973 to 1991 we built anywhere, annually, from 350 to 500 aircraft. From 1991 to today, we’re building handsful, 40, 50 aircraft, sometimes much, much less than that. And then we have an industry saying, we’re not profitable; Wall Street is beating us up. There are a lot of other factors, but one of them is, we’re not producing a lot. We got into a cycle where we’re heavily into R&D and so now the issue becomes, how do we make R&D profitable? The issue becomes, we have a modernization strategy—I think this is a real issue that we need to take a hard look at—that drives us to cycles that make the industry unprofitable at times. VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00141 Fmt 6633 Sfmt 6633 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1 138 Is the solution to pay them for R&D, pay profit rates, increase profit rates on something that is normally an investment account; or is the solution to better plan on modernization so that you equalize production, so people can make money and we can modernize? Mr. SHAYS. I think it’s a very fine point. I saw Dr. Harris start to waiver a little bit here. I am at overload. So whatever you guys are, I’m at overload; I see a new recorder, and we’re done. Julie Thomas, thank you for all your work. There’s no question you were working hard today. And I enjoyed this hearing. I learned a lot. I think our witnesses and all three panels were terrific. And I also appreciate the staff for both the majority and minority for all their good work. So, everyone, have a beautiful day. This hearing is over. [Whereupon, at 2 p.m., the subcommittee was adjourned.] Æ VerDate 11-MAY-2000 11:50 May 24, 2001 Jkt 000000 PO 00000 Frm 00142 Fmt 6633 Sfmt 6011 C:\DOCS\70782.TXT HGOVREF1 PsN: HGOVREF1