Committee on Oversight and Government Reform U.S. House of Representatives 114th Congress The OPM Data Breach: How the Government Jeopardized Our National Security for More than a Generation Majority Staff Report Hon. Jason Chaffetz, Chairman Committee on Oversight and Government Reform Hon. Mark Meadows, Chairman Subcommittee on Government Operations Hon. Will Hurd, Chairman Subcommittee on Information Technology September 7, 2016 www.oversight.house.gov A Letter from the Chairman September 7, 2016 To Federal Chief Information Officers: The advent of the information age presents a paradigm shift about how our federal institutions collect, store, distribute, and protect information. The data breach at the Office of Personnel Management (0PM) is a de?ning moment, and it is up to you?the community of federal chief information officers?to determine how the country will respond. The effectiveness of our country?s response depends on your answer to this questiontrusted with highly personal, highly sensitive data on millions of Arnericans?? Federal C105 possess expertise and technical knowledge that support the mission- related activities of their agency. As Departmental heads focus on managing the bureaucracy of the executive branch, substantive challenges of their agencies? mission, and Congress, CIOs play a critical role in keeping technology working for Americans, and in furtherance of the agencies? mission. Federal ClOs matter. In fact, your work has never been more important, and the margin for error has never been smaller. As we continue to confront the ongoing challenges of modernising antiquated systems, CiOs must remain constantly vigilant to protect the information of hundreds of millions of Americans in an environment where a single vulnerability is all a sophisticated actor needs to steal information, identities, and profoundly damage our national security. The mission of our Committee is to ensure the ef?ciency, effectiveness, and accountability of the federal government and its agencies. We have a constitutional duty to provide meaningful oversight of the executive branch and to recommend reforms that are informed by our investigative ?ndings- Taxpayers also rely on the Committee to bring a measure of accountability and transparency in cases where there is evidence of misconduct. That is why I am releasing this report to the American public. For those whose personal information was compromised, I hope this report provides some answers on the how and why. Most of all, however, it is my hope that the findings and recommendations contained herein will inform and motivate current and future CIDs and agency heads so we as a government can be smart about the way we acquire, deploy, maintain, and monitor our information technology- The 0PM data breach and the resulting generational national security consequences cannot happen again. It is up leaders like you and Congress to ensure it does not happen again. Sincerely, Jason Chaffeta Chairman ii The Damage Done ?This is crown jeweis materiai . . . a goia' minefor a foreign service. ?This is not the end of A meriean human intelligence, out it ?s a signi?cant biow. ?it Joel Brenner, former NSA Senior Counsel ?We cannot undo this damage. What is done is done and it take decades to ?it John Schindler, former NSA of?cer "[The SF 196] gives you any kind of information that might be a threat to [the empioyee security eiearonce. Jeff Neal, former DHS of?cial ?My 519-36 iists every piece I ?ve ever iivea' since i was 18, every foreign travei I ?ve ever taken, aft ofmy_??tmiiy, their addresses. So it?s not jastmy identity that?s a?eeted. I 've got sioiings. i ?ve got?ve kids. AH ofthat is in there. ?f James Conley, Director of the FBI data] remains a treasure trove ofinfortnation that is to the Chinese the peopie represented by the information age of]: mere ?s no ?xing it. Michael Hayden, former Director of the CIA David Perera Joseph Marks, Newty Disclosed Hack Got "Crown Jewel?s. POLITICO, June 12, available at: kground-checks-l 18954. U?icer.? 0PM Hack is Serious Breach of Worker Trust . NPR, June 13, 3015, available at: St??il 3t414 1496 l?tex-nsa-of?cer-opm-hac k-is-serious?breacb?o f?worker-trust. id. Maggie Ybarra, James Conley. FBI Chief: Savs His info was Hacked in 0PM Breach; it was "Enormous WASH. TIMES, July 9, 2o] 5, available at: save-his-own-info-was?haeked. Dan Tv?erton, impact of?FM Breach Conic! Last More Titan 4G Feats, July 12, 2015, available at: iv Executive Summary The of the United States of America has never before been more vulnerable to cyberattacks. No agency appears safe. In recent data breaches, hackers took information from the United States Postal Service; the State Department; the Nuclear Regulatory Commission; the lntemal Revenue Service; and even the White House. None of these data breaches though compare to the data breaches at the Of?ce of Personnel Management (0PM). In what appears to be a coordinated campaign to collect information on government employees, attackers ex?ltrated personnel files of 4.2 million former and current government employees and security clearance background investigation information on 21.5 million individuals.1 Additionally; ?ngerprint data of 5.6 million of these individuals was stolen. The loss of personally identi?able information (PH) is deeply troubling and citizens deserve greater protection from their government. Further, the damage done by the loss of the background investigation information and ?ngerprint data will harm counterintelligence efforts for at least a generation to come. The Signi?cance of What the Attackers Stole. Certain individuals apply for a security clearance to gain access to our country?s most sensitive national security secrets. These individuals are required to complete Standard Form 36 or and undergo a background investigation. Many applicants are obvious targets by adversaries for intelligence purposes by virtue of their holding some of the most sensitive positions in our government, including anyone accessing classified information and anyone employed in a ?national security sensitive position.? This encompasses a wide-range of federal employees and contractors at all federal agencies, including the U.S. Department of Defense and throughout the Intelligence Community. Background investigations conducted on these individuals are designed to identify the type of information that could be used to coerce an individual to betray their country. Therefore, applicants are required to provide a wealth of information about their past activities and lifestyle. For example, applicants are required to provide extensive ?nancial information, as well as employment history and home addresses for the past ten years. Applicants are also required to provide the names of any relatives, including step~siblings or half?siblings, and their home addresses. The also requests disclosure of some of the most intimate and potentially embarrassing aspects ofa person?s life, including whether the applicant: There is some overlap between the 4.2 million individuals impacted by the personnel records breach and the 21.5 million individuals impacted by the background investigation breach. 0f the 4.2 million individuals impacted by the personnel records breach, 3.5 million on these individuals also had their background investigation data stolen. See Letter from Jason Levine, Dir. Congressional, Legislative d: Intergov't Affairs, LLS. Office of Personnel Mgmt. to Jason Cliafl'eta, Chairman, H. Comm. on Oversight d: Gov?t Reform (Aug. 2015). The aggregate number of individuals impacted by this breach totals 22.1 million. It ?consult[ed] with a health care professional regarding an emotional or mentai health oondition;" 1- ?illegally used any drugs or controlled substances,? I abused alcohol resulting in ?a negative impact on your work performance or personal relationships, your ?nances, or result in intervention by law enforcementt'publie safety pcrsonnelf? and - ?experienced ?nancial problems due to gambling.? In short, the 313-36 asks individuals to turn over their most personal details; information that in the wrong hands could be used for espionage purposes. The intelligence and counterintelligence value of the stolen background investigation information for a foreign nation cannot be overstated, nor will it ever be fully known. The Director of the Federal Bureau of Investigation (FBI) James Comey described the data breach as a ?very big deal from a national security perspective and from a perspective. It?s a treasure trove of information about everybody who has worked for. tried to work for, or works for the United States government."2 Nor is there any way to remedy the problem now that the information is in the hands of our adversaries. Foimer Central Intelligence Agency Director Michael Hayden wamed he does not ?think there is recovery from what was lost? and ?it remains a treasure trove of infolmation that is available to the Chinese until the people represented by the information age off. There?s no ?xing it.?3 How the Breach Happened. Despite this high value information maintained by OPM, the agency failed to prioritize cybersccurity and adequately secure high value data. The 0PM Inspector General (IS) warned since at least 2005 that the information maintained by OPM was vulnerable to hackers. In 2014, the IG upgraded issues surrounding information security governance at 0PM from a ?material weakness? to a ?signi?cant de?ciency.? But fundamental aspects ofOPM?s information security posture, such as the absence of an effective managerial structure to implement reliable IT security policies. remained a ?signi?cant deficiency? or worse since 2007.4 Indeed, even after the data breach as of November 2015. the 0PM IG continued to report that continues to struggle to meet many FISMA requirements? and with ?overall lack ofcomplianee that seems to permeate the agency?s IT security program.?5 3 Ellen Nakashima. Hooks databases compromised peopie. federot' authorities save WASH. POST, July 9, 2t] 1 5, available at: ffeeted-Z -5-mi ll ion-people-fcd craI?authonties-sayt. Dan Verton, impact Breech Conic! Last More Titan 40 Years, FedScoopeom {Jul 12, EGIS) available at: .comio nm?losses-a-4U-vear?problem -for?inte liecnce-co mm unity. (1ch of Inapector Gen, US. Of?ce of Pers. Mgmt. No. ante-om, Flatter-tit iiy'brmatinn Security Management Act Audit Fi?i't'iitt (Nov. 12, and) available at: generaltreportsil? l4tfederal~i nformati on~security- manage ment-act-audit-fy-Z? ld-da?c 5 430-144] 1 Et .pdt'. Of?ce of Inspector Gem. LLS. Of?ce of Pets. No. I, Finni Audit Report. Federni Security Modernization Act Audit 5 (Nov. It], 2015] availabie at: ins nector? genera It re po rtsiZ? 1 5t fed oral-inf orm ation-securitv-mod ernization?aet-audit-fy-E?l nal-audit-repot?t-d a- ?ll-1561 l. hdl? [hereinafter Fl?? FISMA Audit]. vi The agency also failed to implement the Of?ce of Management and Budgct?s (OMB) longstanding requirement to use multi?factor authentication for employees and contractors who log on to the network. In a 2015 OMB report on IT security, 0PM was identi?ed at the end of ?scal year 2014 as one of several agencies with the ?weakest authentication profile[s]? and only having one percent of user accounts requiring personal identity verification cards for access.a The agency also allowed key IT systems, which were later compromised, to operate without a security assessment and valid Authority to Operate (ATO). In 2014, the it} called the increasing number of OPM IT systems operating without a valid ATO ?alarming?? The last state of information security left the agency?s information systems exposed for any experienced hacker to in?ltrate and compromise. On March 20, 2014, the US. Department of Homeland Security?s (DI-IS) United States Computer Emergency Response noti?ed Computer Incident Response Team that a third party had reported data ca?ltration from network. In an effort to better understand the threat posed by the hacker, 0PM monitored the adversary?s nrovements over a two?month period. The agency?s senior leadership failed to fully comprehend the extent of the compromise, allowing the trackers to remove manuals and other sensitive materials that essentially provided a roadmap to the 0PM IT environment and key users for potential compromise. While 0PM monitored the first hacker (for convenience here we will refer to this actor as Hacker X1), on May 2014 another hacker posed as an employee of an 0PM contractor performing background investigations, KcyPoint (which we can call Hacker X2). Hacker X2 used the contractor?s 0PM credentials to log into the 0PM system, install malware. and create a backdoor to the network. As the agency monitored Hacker ?s movements throughout the network, it noticed Hacker was getting dangerously close to the security clearance background information. 0PM, in conjunction with developed a plan to kick Hacker X1 out of the system. It termed this remediation ?the Big Bang.? The agency was confident the planned remediation effort in late May 2014 eliminated Hacker 's foothold on their systems. But Hacker X2, who had successfully established a foothold on systems and had not been detected due to gaps in IT security posture, remained in system post-Big Bang. The Eitfiltration of the Security Clearance Files Could Have Been Prevented. After the May 27 Big Bang, Hacker X2 moved around system until they began ea?ltrating data in July 2014. As Director oflT Security Operations Jeff Wagner explained, the KeyPoint credential was used for the initial attack vector and then the attacker used various tactics to obtain domain administrator credentials to ultimately perform operations and maintain persistence from malware. Beginning in July through August 2014, the Hacker X2 caliltrated the security clearance background investigation ?les. Then in December 2014, personnel records were exfiltrated, and in early 2015, ?ngerprint data was ex?ltrated. 6 Office if: Budget, Exec. Of?ce of the President, Fl" 20H Amino! Report to Giorgi-ass: ne'er-oi Ill?lil??l?i'lr?ll Security Management dot at 23, 2t} (Feb. 27, 2f? 5] available at: itehousegovfsitcsi?de sf?naljy 4_?sma 7_2{l 1 5.pd f. Of?ce of Personnel Mgmt. Of?ce of the Inspector General, Federal lipformort'on Scent-try Management .dc't? Audit Fl? rare at 9 (Nov. 12, 2?14)availahlc at: vii Had 0PM implemented basic, required security controls and more expeditiously deployed cutting edge security tools when they first learned hackers were targeting such sensitive data, they could have significantly delayed, potentially prevented, or signi?cantly mitigated the theft. Testimony from DHS made clear implementation oftwo-factor authentication for remote logons in early 2015, which had long been required of federal agencies, would have ?precluded continued access by the intruder into the OPM network.? Further, if 0PM had fully deployed in a preventative mode available security tools and had sufficient visibility to ?rlly monitor their network in the summer of 2014, they might have detected and stopped Hacker X2 before they had a chance to exfiltrate the security clearance background investigation ?les. Importantly, the damage also could have been mitigated if the security of the sensitive data in ?PMis critical IT systems had been prioritized and secured. The exact details on how and when the attackers (X1, X2) gained entry and established a persistent presence in OPlvl?s network are not entirely clear. This is in large part due to sloppy hygiene and inadequate security technologies that left 0PM with reduced visibility into the traf?c on its systems. The data breach by Hacker X1 in 2014 should have sounded a high level multi-agency national security alarm that a sophisticated, persistent actor was seeking to access highest-value data. It was not until April 15, 2015 that 0PM identified the first indicator its systems were compromised by Hacker X2. From April 16, 2015 through May 2015 (during the primary incident response period), security tools from an outside contractor, Cylance 1nc., consistently detected key malicious code and other threats to 0PM. While these types of security tools were generally available to (PM, the agency did not choose to deploy a preventative technology until rr?er the agency was severely compromised and until n?er the agency?s most sensitive information was lost to nefarious actors. Notably, Director of Security Operations, Jeff Wagner, recommended deploying Cylance?s preventative technology to insulate OPM's enterprise from additional attacks after the initial attack by Hacker in March 2014. The Committee obtained deeurnents and testimony proving information security posture was undermined by a woefully unsecure IT environment, internal politics and bureaucracy, and misplaced priorities related to the deployment of security tools that slowed vital security decisions. Swifter action by 0PM to harden the defenses of its IT architecture could have prevented or mitigated the damage that systems incurred. While 0PM continued its incident response efforts throughout April 2015, another outside contractor named CyTech Services, provided forensic support after conducting an onsite demonstration of its technology While 0PM and CyTeeh provide differing accounts ofthe role in detecting unknown malware on systems, it is clear CyTech detected rnalware and assisted for at least two week in the response to the 2015 data breaches. To date, CyTeeh has not been compensated for any of its work. The Anti~Dc?eicncy Act prohibits a federal agency from accepting voluntary services without payment and without obtaining an agreement in writing that the contractor will never seek payment. In this case, there was no such agreement. Most concerning, the agency destroyed [,035 ?les and directories located on CyTeeh?s device prior to returning the device to its owner while a request from the Committee for this information was pending. All of those ?les were material to the Committee's investigation, responsive to the Committee?s subpoena requests for information and documents, and subject to a preservation order by the Committee. 0PM Misled Congress and the Public to Diminish the Damage. As the agency assessed the damage caused by the hackers, 0PM downplayed the fallout. 0PM failed to proactively announce the 2014 breach to the public, and claimed the two cyberattaeks were not connected. The 2014 and 2015 incidents, however, appear to be connected and possibly coordinated. The ?rst confirmed adversarial activity for both incidents came within a two- month span in November and December 2013. The hack discovered in March 2014 by Hacker K1 appeared to move through the system looking for security clearance background investigation data and was removed when they got too close. Hacker XI did, however, cxfiltrate manuals and other sensitive materials, which would be useful or targeting background information data systems. Hacker X1 was cleared from the system in May 2014 during the Big Bang exercise. Within three months, Hacker X2 finished targeting and stealing background investigations data (by early August 2014). Hacker X2 later stole personnel records (in December 2014) and ?ngerprint data {in March 2015}. The two attackers shared the same target, conducted their attacks in a similarly sophisticated manner, and struck with similar timing. Further, the manuals estiltrated by Hacker X1 likely aided Hacker X2 in navigating the OPM environment. The Committee?s year-long investigation to understand now the attackers perpetrated their intrusion, movements, and ultimately the es?ltration of data began with hearings, wherein then-0PM Chief Information Of?cer [(310) Donna Seymour made a series of false and misleading statements under oath regarding the agency?s response to the incidents announced in 2015. Seymour testified that 0PM purchased CyTech licenses, hut 0PM did not make any purchases from CyTech. She also testi?ed that CyTech?s tool was installed in a quarantine environment for the demonstration, but this tool was running on a live environment at 0PM when it identi?ed malware on April 22, 20 5. Seymour also misled the public about the signi?cance of the data stolen in the 2014 attack. She testified on April 22, 2015 that ?our antiquated technologies may have helped us a little bit. ?3 Two months later, on one 24, 2015, she testified that the stolen manuals that were a roadmap to 0PM 5 systems were merely ?outdatcd security documents." 9 The Bottom Line. The longstanding failure of leadership to implement basic cyber hygiene, such as maintaining current authorities to operate and employing strong multi-faetor authentication, despite years of warnings from the Inspector General, represents a failure of culture and leadership, not technology. As 0PM discovered in April 2015, tools were available that could have prevented the breaches, but 0PM failed to leverage those tools to mitigate the agency?s extensive vulnerabilities. Enhancing vaeta'eern?itv oletird?Fm'tr Contractors and iv?endws: Hearing Re?ne the H. (711111111. on Oversight d: Co11g.[hpr. 22, 2015} Enhancing vaeizrecnrity Hearing] (statement of Donna Seymour, Chief Info. Officer of the 3. Office ofPers. Mgmt.) 9 0PM Darn Bi each Port Hero mg Before the H. anon. on Oversight (ii Gov' Rgfoi in, 1 14d: If.? bug. 619 [June 24, 2015] {hereinafter Henr mg on 0PM Data 31' each. For: It] {statement of Donna Chief Info. Officei of the US. Office ofPers. Mgmt.}. ix As a result, tens of millions of federal employees and their families paid the price. Indeed, the damage done to the Intelligence Conununity will never be trulj,?r blown. Due to the data breach at 0PM, adversaries are in possession of some of the most intimate and embarrassing details of the lives of individuals who our country trusts to protect our national security and its secrets. This report documents how the government allowed this unthinkable event to happen and makes recommendations in an attempt to ensure this never happens again. The Committee remains hopeful that 0PM, under the new leadership of Acting Director Beth Cohen, is in the'process of remedying decades of mismanagement. Table of Contents A Letter from the Chairman ii The Damage Done Executive Summary Table of Contents 1 Timeline of Key Events 5 Findings 14 Recommendations 20 Table of Names 28 Chapter I: IT Security Record Preceding Breaches 30 The Rise of Advanced Persistent Threat Hacking 3t:- Federal Contractors Holding Sensitive Federal Employee In Formation Targeted and Attacked 31 Federal Initiatives to Increase Information Security in Response to Increasing Attacks 34 0PM Failed to Recognize the Threat and Implement Effective IT Security Measures When It Mattered 35 C?yhersecority Spending Consistently Trailcd Other Federal Agenctesiio 0PM Attempts to Balance IT Security with Competing Priorities The Katherine Archulcta and Don na Seymour Era 42 0PM Failed to Prioritize the Security of Key Data and Systems 4? Chapter 2: The First Alarm Bell Attackers Discovered in 2014 Target Background Information Data and Eafiltrate System-Related Data 51 Discovery Incident Response for Attackers Discovered in 2014 52 Monitoring the Adversary and the May 20 4 ?Big Bang? to Expo] Attackers Discovered in NH 55 During the Incident Response Period the Ex?ltration ofFIPS-rclated In Formation Made Clear the Attackers? Target was Background Investigation Data Held in FIPS til Tactics Techniques 3: Procedures of Attackers Discovered in 2014: Hikit Malware and 3MB Protocol Network Logging Capabilities Limited Investigating the ?How" and ?How Long" for Attackers Chapter 3: 0PM Attempts to Mitigate the Security Gaps Identi?ed in 2014 While Iron Man and Captain America Go to Work (May 2014 April 2015) 75 IT Security Posture and Mitigation Efforts After the May 2014 ?Hi Bang? T5 Key 2614 Recommendations Highlighted 0PM IT Security Vulnerabilities 0PM Efforts to Buy Security Tools to Secure the Legacy Network and Rebuild "Very Insecure, Insecurely Architected Network? T9 Missed Key Developments 31 In April 2015. 0PM Realized They Were Under Attack Again 33 1 Captain America: The First Indicator that Led to the 2015 Discovery oIthe Background Investigation Data Breach 34 The Avengers: Anatomy of the Data Breach Discovered in 2015 35 Chapter 4: The Role of Cylance Inc. 91 ?Cyber Climate" During Cylance Product Demonstrations 91 Overview of the Cyiance Cyber Tools 93 April 15-16. 2Dl5: The First 24 Hours 94 April 12, 2015: Con?rms Plugx 98 April 17, 2015: CylanceProtect Deployed 1'30 April 13, 2015: Protect Lights Up Like a Christmas Tree 102 April 19, 2015: Severity of the Situation Becomes Clear 1113 April 2111-23, 2131 5 More Key Trojans Identi?ed; GIG First Noti?ed. IDS April 24-25, 2015 0PM Upgrades Protect to Auto-Quarantine Mode. 1 10 April 26 April 30, 21315: First Signs of Lost Background. Materials 113 The Decision to Purchase CylanceProtect 1 115 Political Challenges on the Desktop to Counterpoint? Lack of Compliance llSl Purchases Plotect After Nearly Losing Access to It 121 Chapter 5: The CyTech Story 125 CyTech Is a Small Business Contractor with Significant Cyber'l'ool Capabilities 126 CyTech Was Invited to Conduct a Demo at 0PM 12? Prior to the April 21 2015 Demonstration at CIPM 123 The April 21, 2015 April 22, 2015 Demonstration at 0PM 123 The CyTech Demo Turned into Incident Response and Forensic Support 135 CyTeeh Provided Onsite Incident Response and Forensic Support From April 23 to May 1, 2015 13S Was Deployed on the UPM Network beginning in April 5 and Remained on Network through August 2015 133 The Wolf Street Jennie! Reports on CyTech?s Role in the OPM incident on June It], 2915 141 Cy'l'ech Coordinated with 0PM Prior to the June it}, 2615 Story 142 0PM and Respond to the Article 143 DPl'v! Description oFCyTeeh?s Role Was Misleading 1415 Archuleta and Seymour Provided Misleading Testimony to Committee 1415 Data on CyTech's Appliance Collected During the 2015 Incident Response Period was Deleted 148 0PM Retained CyTech's Appliance Through August 2m 5 149 Before Returning the Appliance 0PM Deleted Key Data 149 0PM ?Sanitizod? the Appliance 151 0PM Violated the Anti-De?ciency Act 152 The prohibition on accepting voluntary services 152 The ?gratuitous? services exception 152 2 The ?emergencies" exception 153 The ADA applied to the 0PM and Situation 153 CyTech expected to be paid 154 Chapter 6: Connections Between the 2014 and 2015 Intrusions 157 One Group, Several Names 153 The 2014 Data Breach: The Unique Malware of the Axiom lGroup 159 Malwarc Discovered during the 21115 Data Breach 162 21114 a 21115: Likely Connected, Possibly Coordinated 163 Chapter 7: OCIO and its Federal Watchdog 173 The [G's Memorandum ofConcern 1T4 Four Instances Where the 01:10 Failed to Cooperate Fully 17?? Seymour failed to appropriately notify the of the April 212115 intrusion detection Seymour failed to notify the DIG of the loss of background investigation data in a timely manner 1311 Seymour failed to notify the BIG ahtJut the 21114 incident 132 Meetings Itvith Federal Law Enforcement Agencies 133 KeyPoint Audit 134 Noti?cation Concerning New IT Infrastructure 135 Five Incorrect anchor Misleading Statements 137 First Missratemcnt before the Senate Committee on Appropriations 1311 Second Misstatement Before the Senate Committee on Appropriations 133 Third Misstatelnent Before Senate Committee on Appropriations and House Committee on Clversight and Government Reform 133 Fourth Misstaternent Before the House Committee on ifftversiglit and Government Reform 139 Fifth Before the Senate 139 Current State of Relationship 139 Summary of GIG and relationship 193 Chapter 3: The IT Infrastructure Improvement Project: Key Weaknesses in Contracting Approach 194 The 1G Issues a Flash Audit Alert and Interim Reports on the IT Infrastructure Project 196 The [G?s Concerns Continued through the Fall of 211 I 5 193 10 Reports Progress in Responding to Concerns, but Challenges Remain as ofMay 21116 . 193 The Story of CIPtvt's IT infrastructure Improvement Project and the Sole Source Contract EDD Timeline: IT Infrastructure Improvement Project 21.11 0PM Initiates Contact with Imperatis and Awards Sole Source Contract 205 Imperatis and UFM Buy Security Tools to Secure the Legacy IT Environment 212115 Imperatis? Role in Responding to 0PM Data Breach Incidents 207 Sole Source, Schedule, and Cost 1G Concerns Related to IT Infrastructure Improvement Contract Validated 21:13 Summary of Investigation 214 3 Committee hearings on the data breaches 214 Committee request For information regarding identity theft services 21 5 Productions related to the 0PM data breaches and CyTe?ah 216 The Committee investigated the role oFCylance 219 The Committee investigated the role of SEA 221] The Committee Investigated OPM's IT Infrastructure Improvement Project and the Contract Awardee lnrperatis 22] Document productions by Department of Homeland Securityr 221 delays. restrictions, redactions and a congressional subpoena 222 Unnecessary delays 222 redactions 222 Subpoena issued to 0PM 224 Conclusion 225 Appendix: Cyber security Spending at 0PM (Fiscal Years 2012-2015) 227 Timeline of Key Events Jan 2012 r? Attackers had access to network, according to US-CERT found maiware {Hikit} resided on an 0PM server since 2012.3 November ZUIJ First evidence of adversarial activity by the attacker associated with the breach that USHCERT informed 0PM about in March 2014.3 December 2013 First evidence of adversarial activity associated with the 2015 breaches (including harvesting of credentials from 0PM contractors) by the attacker that was not identi?ed until April 2015.4 More}: 20, 201'4 a? US-CERT notifies 0PM oi"a data exiiltration from OPM's network.5 0PM, working with US-CERT, determines and implements a strategy to monitor the attackers? movements to gather This breach involved data that included manuals and IT system architecture information, but the full extent ot'esliltrated data is unknown. v? The strategy remains in place Until the ?Big Hang" on May 20M. Marci: 25, 20? v? Situation report takes place with CID Donna Seymour and US-CERT.E March 2014 s? As 0PM monitors the hackers, it develops a ?Plan for full shut down if needed."7 June 2014 0PM Incident Report at Production- Sent 13,3115] [hereinafter June 2014 0PM Incident Report]. Note: This Report was authored by CERT and provided to DPM 2 3. Dee of Homeland SecuritnyS? Digital Media Analysis Report-465355 (June 9 2m 5} at [101 154 CERT Production: Dec. 22, 2015) [Hereinafter June 9 ZUIS I"Heavier; on 0PM Doro Brena-:31 Pm! (statement of Donna Seymour, Chief lnf'o. Officer ofthe U. 3 Of?ce of Personnel "Brie?ng by to H. Comm. on Oversight dc Gov't Reform Stafleeh. 19, El] 16}. one 2014 incident Report at Id. 7' Id. April I1, 201 4 Tactical mitigation strategies and security remediation plan developed for brie?ng to Donna Seymour.S April 21, 20114 1? 0PM contractor (SEA) discovers a ?specific piece of malware,? which is brought to attention.? April 25, 2014 is registered to Steve Rogers, a.k.a. ?Captain The hackers later used this domain for command and control and data ea?ltration.? May 2014 1/ The attacker later associated with ea?ltrating background investigation data establishes their foothold into network. This attacker poses as a background investigations contractor employee (KeyPoint), uses an 0PM credential, remotely accesses network, and installs Plugx malware to create a backdoor.? v" OPM did not identify the attacker?s May 7 foothold despite the fact that 0PM was monitoring and removing another attacker {that US-CERT had noti?ed 0PM about in March 2014). May 2914 OPM shuts down its compromised systems in the ?Big Bang? event in an effort to remove the attacker. This decision was made after OPM observed the attacker ?load a key logger onto . . . several database administrators? workstations? and they got at. at Id. at HOGRUB I 84101242. '0 ThreatConneet Research Team, 0PM Breach [June 5, 212115}, available at: H. Comm. on Oversight and Oov?t Reform, Transcribed interview of Brendan Saulebury, Senior lCyher Security Engineer, SKA, Ex. 4 (Feb. H, 2016} [Hereinafter Saulshury Briefing by US- CERT to H. Comm. on Oversight dc Gov? Reform Staff {Feb 19 2016}; r. at 59. H. Comm. on Oversight dc Gov Reform, Transcribed Interview of .lef fP Wagner, U. 5. Of?ce of Personnel Mmgt., Dir. of Information Technology Operations at 121-128 (Feb. 18, Z?l ti) [hereinafter Wagner Tr.; Dep tof Homeland and Of?ce of Pers. Mgmt., 0PM Cybersecurity Events Timeline (Aug. 26, at HOORGEDB Production: May 13, 2016) [hereinafter 0PM Cybersecurity Events Timeline]; Brie?ng by LIB-CERT to H. Comm. on Oversight a Gov?l Reform Staff (Feb. 2016}. KeyPoint CEO testi?ed that ?there was an individual who had an OPM account who was a KeyPoint employee and l] the credentials of that individual were compromised to gain access to Hearing on 0PM Data Breech: For! [statement of Erie Hess, Chief Exec. Of?cer, KeyPoint). The OPM Director of IT Security Operations [Wagner] explained that ?a KeyPoint user credential [was] utilised for [the] initial vector infection," but that ?user did not have administrative credentials, so the adversary utilized tactics in order to gain domain administrator credentials? to move through the environment and conduct operations?related activities. Wagner Tr. at Ed. ?too close to getting access to the system," which held the background investigation data.? it? Meanwhile. the attacker that established a foothold on May 7, 2014 continues their presence on the OPM network. June 5, 2914 Malware is successfully installed on a KeyPoint web server; accounts differ as to whether or not administrator privileges were used to install this rnalware. '4 June It}, 2014 v? 0PM C10 Donna Seymour testifies before the Senate Homeland Security and Governmental r-?tf?fairs? Subcommittee on OPlvl?s Strategic Information Technology Pian atstd does not disclose at this hearing the ?manuals" breach discovered in March 2014. June I2, 2014 1" 0PM executes a Cvlance product evaluation agreement that allowed it to test the ?anctionality of both Cylance products (V and Protect} for a limited period of note.?5 June 20, 2014 Attackers conduct a remote desktop protocol (EDP) session, indicating contact with ?important and sensitive servers supporting . . . background investigation processes.? The remote session was not discovered until spring 2015. '7 June 22, 2014 r? DHS issues a ?nal incident report for the 0PM ?manuals? breach first discovered on March 20, 2014.? '3 Saulsbury Tr. at 15-26. '4 Brie?ng by H. Comm. on Oversight a new: Reform 19. 2016]: Letter from Kcyl?oint Government Solutions to the Hon. Elijah E. Cummings, Ranking Member, H. Comm. on Oversight 3t Gov?t Reform [July 2, Note: Keyl?oint maintains that ?No unaccounted security tokens were used during the time the malware was operational on KeyPoint?s network.? The Report of the KeyPoint intrusion disagrees stating that ?a domain administrator account was used to install the malware on the web server. reported that this ?administrator account" had ?full access privileges.? ?5 A More E?cient and Effective Government." Examining Federai IT Initiatives and the IT Workforce: Hearing Before the S. Sttocomnt. on the E?eienev and E?eetiveness of??ed. Programs d: tire Fed. Wot-Home oftire S. Comm. on tiotnet'and See. Gov 't Affairs, 1 13th Cong. {June It], 20 I4). H. Comm- on Gversight d: Gov?t Reform. Transcribed Interview of Stuart MeCiure, Chief Exec. Of?cer, President ?5 Founder, Cylanee, Ex. 2 (Feb. 4, EMS) [hereina?er McClure H. Comm. on Oversight 3L Gov?t Reform, ?l'ranseribed interview of hris Coulter, Managing Dir. of Incident Response and Forensics (Feb. 12. 24316), Ex. 18 [hereinafter Couiter ?5 June 2014 orM Incident Report at June 23, 2M4 v? US-CERTIOPM identi?es this as first known adversarial access to mainframe.? July August 2014 v? Attackers successfully ea?ltrate the background investigation data from systems?? Jul} 9, 2014' 0PM acknowledges the March 2014 ?manuals? breach to the New York i'i'nruar.ll This information had not previously been disclosed publicly. 0PM states that no was lost in the breach and does not disclose the ex?ltration of the manuals. Jul)? 29, 2014 ?opmlearningorg? is registered to Tony Stark, a.k.a. ?iron Man."12 The attackers used this domain for command and control during their intrusion into OPIWs environment. August 16, 20.? The malware installed on KevPoint systems on one 5, 2014 ceased operational capabilities. 1? Detober 201'4 9" FBI Cyber Division issocs a Cyber Flash Alert regarding ?a group of Chinese Government af?liated cyher actors who routinely steal high value information from US commercial and government networks through cyber espionage" and notes l: Dep?t of Homeland Brie?ng to Staff (Feb. 19, 2016:); Cybersecuritv Events Timelinc. Id. 2' Michael E. Schmidt, David E. Sanger Nicole Perlroth, Chinese [fuckers Pursue Kay Dam on LES. Workers, MY. TIM 9, 2014. available at: ThreatConneet1 Branch Heather's; Saulsburv TL. Ex. 4. Letter from KcyPoint Government Solutions to the Hon. Elijah E. Cummings, Ranking Member, H. Comm. on uh Gov?t Reform {July 2. {citing USHCERT Report (Aug. 30, KeyPc-int notes that ?signi?cantly. the malware was a ?zero day" attack?it had an electronic signature that was not known by anti? virusfanti?malware software at that time." activity associated with this group ?should be considered an indication ofa compromise requiring extensive r? Meanwhile, the attackers move through the OPM environment to the U.S. Department of interior data center where 0PM personnel records are stored?5 November 2014' A group ot'private-industry security companies warns about threats to the human resources components of federal government and releases a report on Chinese Advanced Persistent Threat (APT) activity.?5 December 2014 4.2 million personnel records are ex?ltrated after attackers moved around system and through the 001?s database, which holds 0PM personnel records.? March 3, 2015 v? is registered by attackers. Attackers would use this domain for C2 and data en?ltration in the final stage of the intrusion.23 March 201' 5 9' The last beaconing activity to the unknown domain ?opmsecurityorg? occurs. This domain was registered in April 20M to Steve Rogers, a.k.a. ?Captain America?? March 26, 2015 v" Fingerprint data appears to have been ex?ltrated on or around this date.? 3* Cyber on. Fed. Bureau ot'Investigation, rear eyes-Firms Aim (Oct. 15, 2014), httg: slideslIare. 15 0PM C?vhersecurit}r Events Timeline. "5 Novetta, Operation Edam. Ariana ThreatAcroI Group Report 9 (2014}, http: novetta. cmm?wp? contentfuploads??ltifi 1fExecutivejumInarv-FinaL -pdf {The report emphasises ?Hikit? malware, stating. ?Among the industries We. observed targeted or potentially infected by Hikit [included] Asian and Western government agencies responsible for [a variety of services such as] Personnel Management. Brie?ng by US-CERT to H. Comm. on Oversight 3: Gov Ref?rlt'l Staff (Feb. l9, OFM Cybersecurity Lventsl limeline. ?5 DOMAIN 2-: WUC NEWS-P051: COM one {last visited June 28. sold). available at: threatcrowd. orgfdornain. php?domain=wdc? ?news? post. com. 39 Sauisbury Tr. at 59. 3?June 9 2315 DMAR at 153; see ntso Dep?t of] iomcland Brie?ng to Staff (Feb. 19, E?id); 0PM Cybersee urity Events Timeline. April 15, 201'5 14? After being alerted by an 0PM contractor (SKA) working on IT security, 0PM notifies about suspicious network traf?c related to opmsecurityorg.? This domain was registered to Steve Rogers, a-k.a. ?Captain America" in April 2014 and the last beaconing activity occurred in March 2015. April 16, Billb- 1/ 0PM contacts Cylance for technical support on use oi?Cylance V, which was an endpoint detection tool that 0PM had purchased in Siptember 2014.32 Cvlance is not intended to be an enterprise?wide prevention tool. April I 2015 v? 0PM begins to deploy enterprise-wide (on a demonstration basis and in ?Alert? mode) a Cvlance tool called CylanceProteet. At this time CylaneeProtect was not in quarantine mode, but the tool would later identify,r and alert 0PM to the widespread presence of malware on their system. 0PM brings Cylancc onsite for incident responses"4 0PM does not upgrade this tool to the highest preventative setting. 35 April lE?l 9, ZEUS 1/ Cvlancel?roteet is deployed to over 2,000 devices as of this date, makes ?tons of findings," and as a Cvlance engineer described the tool, it ?lit up like a Christmas tree? indicating widespread malicious activities within the OPM system-? April 2 2015 1" CyTech Services arrives onsite to conduct a product demonstration with their CyTech Forensics and Incident Response tool, and remains onsite until May 1, 2015 to assist with incident response.? April 22, 2015 v? Donna Seymour testi?es before the Committee about cybersecuritv and publicly discussed the discovery of the ?manuals? breach saying, ?the adversaries in today?s environment are typically used to more modern technologies, and so in this case, potentially, our antiquated technologies may have helped us a little bit. But I 3? June a, 2015 name at 15s. 3: Coulter Tr., Ex. 1, 2. 33 McClure Tr. at 3. 34 McClure Tr. at 21-22. 35 111'. DPle upgraded from the Grimes tool to the Cylonce PROTECT tool. However, the tool remains in ?Alert" mode only, not ?Quarantine mode." 3" McClure Tr., Ex. Coullcr Tr. at 20-2]. 3? H. Comm. on Oversight Gov?t Reform, Transcribed Interview of Benjamin Cotton, CvTceh Services, Chief Executive Of?cer at NH 5 (Sept. 3t}, 2015} [hereinafter Cotton ll] think also it comes down to culture and leadership, and one of the things that we were able to do at OPM was to recognize the problem.?B r" OPM's Of?ce of the Inspector Genera! (OIG) learns of the breach for the first time after a staffer bumped into the OPM Director of Security Operations in the hallway. if The staffer testi?ed that Director of IT Security Operations said there was ?no need" to notify the public ofthe breach.? Aprif 23, 201' 5 0PM determines there had been a ?major incident? involving the cs?ltration of personnel records, which triggers a requirement to notify Congress.? OPM notifies Congress of a ?major incident" on April 30, 2015.41 April 24, 205 OPM orders a global quarantine to address malware identi?ed by CylanceFrotect.? April 215, 2015 i/ Cylance engineers identify adversarial activity related to an session to a background investigation database indicating this session took place in June Etllalf?3 May a, 2015 1/ establishes with a high degree of certainty that personnel records datafPII had been stolen.? May 20, ENE v? 0PM determines there was a major incident regarding the cx?ltration of background investigation data, which triggers a requirement to notify Congress. v? 0PM noti?es Congress on May 3'3 Enhancing Cybersecnriry Contractors and Pandora: Hearing Eefore the H. Comm. on Oversight Gov'r. Reform, I 14th Cong. (Apr. 22, 2015) (statement of Donna Seymour, Chief Info. Of?cer, .3. Office of Pers. Mgmt.) (testifying that OPM was hacked and that no PM was taken}. The word ?manuals? is not used at this time, though it is how we have since described the 2?14 breach. 3'9 H. Comm. on Oversight d: Oov't Reform, Transcribed Interview of US. Of?ce of Pers. Mgmt. Of?ce of Inspector Gen. Special Agent at N-IS (Oct. ti, EDIE) [hereinafter Special Agent Federal Information Security Modernization Act of 2e14, Pub. L. No. 1 13-2s3, 12s Stat. sass (201d). 4' OPM Cybersecurity Events Timeline. ?3 Coulter Tr., Ex is. '13 Coulter Tr., Ex. 13. Briefing by to ll. Comm. on Oversight 3r Gov?t Reform Staff (Feb. 19, sets); OPM Cybersecurity Events Timelinc. 11 v? 0PM indicates to the GIG that background investigation information may also be compromised.? June 4, 2015 v? 0PM briefs the media and releases a press statement that revealed the personnel records of 4.2 million former and current federal employees have been compromised." June 8, 2015 v? US-CERT establishes with a high degree of certainty that background investigation datai?PIl has been ex?lttated and stolen.43 June 16, 20.5 v? Then?0PM Director Katherine Archuleta acknowledges that background investigation data may be compromised.? June 24, 2015 Donna Seymour testi?es before the Committee and minimizes the importance of data removed in 2014 ?Manuals? breach, saying ?those documents were some outdated security documents about our systems and some manuals about our systems."5n June 29, ENE v? The American Federation of Govemment Employees (AFGE) ?les a class action suit against 0PM.51 *5 Briefing by u. Comm. on Oversight a Gov?t Reform amnesia. 19. 2:115); om Cybersecurity Events Timelinc- *5 Special Agent at 46. U.S. Office of Pcrs. Mgmt., Press Release. 0PM to Nomi: Empiovess oijvbeieecm-irv Incident (June 4, EMS), vine ses?Ol ?3 Brie?ng by to H. Comm. on Oversight Gov't Reform Staff (Feb. 19, 2016}; 0PM Cybersccurity Events Timeline. 49 0PM: Dora Breach: Hearing ?ne-fore the H. Comm. on Oversight it} Gov Re?in?. lath Cong. [Jone 16, 21315} statement of Katherine Archuleta, Din, US. Office ofPers. Hearing on 0PM Doro Breech: For: (statement of Donna Seymour, Chief Info. Of?cer, U.S. Office of Pete. Mgmt.). 5' American Federation of Government Employees v. US. Q?'ice ofPenv. Mgms, No. 1:15wev?l?15 ?led June 29'. 2015). 12 June 30, 2015 After 7'4 days of deployment to over 10,250 devices, CylanceProtect detected and blocked almost 2,000 pieces of malware (including critical samples related to the breach)?nearly one piece of malware for every five devices. ma: 9, ears 0PM issues a press release continuing background investigation data for 21.5 million individuals was compromised? Jute 1'0, 201' 5 0PM Director Katherine Archuleta resigns. July 2015 v? The Committee sends the first of a series of document requests to 0PM. August 20, 30} 5 v" 0PM returns the tool to CyTech with kc},t information deleted. The tool, before it was deleted, contained images from incident response of more than 11,000 ?les and directories. September 23, 2015 0PM updates its original estimate that 1.1 million ?ngerprint records were compromised. The new estimate: 5.6 million.53 February 22, 2016 v" Prior to testifying before the Committee, 0PM C10 Donna Seymour resigns. February 24, 2016 v? Committee?s planned hearing, Data Breach: Part is cancelled in the wake of OPM Donna SeymoUr?s resignation.54 53 Press Release, US. Of?ce cfPers. Mgmt., 0PMAmumncer Steps In Frans-ct Ferret-at Workers and Others me Cyber Threats [July 9, 2015} available at: 53' Press Release, U.S. Of?ce ofPers. Mgmt., Statement by (3PM Prem- Secretory Sent on Background Investt'gutt'ens Incident (Sept. 23, 2015} available at: 9231' . 5; 0PM Date Breeches: Part Ht: Hearing Before It. Comm. on (hereth re: Gov 't Refer-m, 1 14"? Cong. (Feb. 24, 2016) {hearing cancelled]. 13 Findings Che ter 1: later! PM IT Securi Record 0PM has iong been piagneti by afaiiare of management to prioritize information security in practice, and to retain ieariers that are committed to information security over the iong hani. FINDING: FINDING: FINDING: FINDING: FINDING: FINDING: 0PM leadership failed to heed repeated recommendations from its Inspector General 0PM has historically maintained a fragmented IT infrastructure, and still lacks a full, accurate inventory of all its major IT systems. As the IG noted in its FY2015 audit, ?failure to maintain an accurate inventory undermines all attempts at securing information systems?1 Over the EGGS-2015 timeframc, 0PM failed to suf?ciently respond to growing threats of sophisticated cyber attackers. 0PM failed to prioritize resources for cyber security. In FY 2013, FY 2014 and 2015, 0PM spent seven million each year on cybersecurity?spending that was consistently at the bottom relative to all other agencies that are required to report such expenditures to the Office of Management and Budget. Slow implementation of critical security requirements such as dual factor authentication is a true case of misplaced priorities. As early as 21305, 16 issued a warning in a semiannual report that given the sensitive data 0PM holds on former and current federal employees and family members, any attack or breakdown ?could compromise efficiency and effectiveness and ultimately increase the cost to the American taxpayer.? Key 0PM systems, including the Personnel Investigations Processing System (PIPE), Enterprise Server Infrastructure (E31), and the Local Area NetworWidc Area Nennork (LANKWAN) were all operating on expired Authorities to Operate at the time of the data breach. Fin i elat ta Br iscov 'n2 14? In the spring onDi 4 0PM sn??ered a data breach that restated in the toss of documents reiatt'ng to the most databases on 13" environment. FINDING: Due to security gaps in network and a failure to adequately log network activity, the country will never know with complete certainty all of the documents that the attackers cs?ltrated from 0PM in connection with the breach discovered in March of2'314. 14 FINDING: FINDING: FINDING The 2014 attackers used an uncommon toolkit designed for late-stage persistence and data ex?ltr'ation. The malware observed on systems in 2014 were two variants of Hikit maiware, termed Hikit A and Hikit B. During an approximately two?month period, 0PM watched the adversaries take sensitive data relating to high?valued targets on systems, the server" that holds background investigation materials, but was never able to determine how the adversary initially gained entry into their network. The documents taken by the 2014 attackers included information about systems that would have given an adversary an advantage in hacking the background investigation database and other sensitive systems in environment. [manger 3: QPM Attempt; te Mitigate the Seeu?m ?aps Identi?ed in 2911 ?hjle lren WW FINDING: FINDING: FINDING: FINDING: In June 2014, issued an incident report with 14 observations and recommendations to address the security gaps identified after the 2014 ?manuals" breach. LIE-CERT deemed network very insecure, insecurely architeeted, and found 0PM had a signi?cant amount of legacy infrastructure. US-CERT also said there was a gap in information technology leadership across 0PM as an agency and that it was not uncommon for existing security policies to be circumvented to execute business functions while exposing the entire agency to unnecessary risk- Had 0PM leaders ?rlly implemented basic, required security controls including multiufactor authentication when they ?rst learned attackers were targeting background investigation data, they could have signi?cantly delayed or mitigated the data breach of background information. in April 2015, an 0PM contract employee identi?ed a domain that was purposely named to emulate a legitimate looking website and upon further found the domain had a randomized email address and was registered to Steve Rogers, a.k.a. ?Captain America.? This was one of the ?rst indicators of compromise identified by OPM in April 2015. 15 er 4- Fin [an Information security toots onyionce Inc. detected critics! moticious code and other threats to 0PM in Aprii 20! 5 and thereofi?er player! it critical rote in responding to the (trite breaches in 20.35. FINDING: FINDING: FINDING: FINDING: FINDING: FINDING: ?Wi'rile Cylance tools were available to 0PM as early as June 2014, 0PM did not deploy its preventative technology until April 2015 after the agency was severely compromised and the nation's most sensitive information was lost. Swi?er action by OPM to deploy CylanceProtect would have prevented or mitigated the damage that UPM's systems incurred. Following the May 27, 2014 ?Big Bang? remediation, 0PM decided not to purchase and deploy CylanceProtect due to, as Cylanee CEO Stuart McClure put it, ?political challenges on the desktop,? meaning overcoming the tensions between IT security and program functionality. On April 15, 2015, 0PM found an indicator ofcomprornise and turned to Cylance for assistance. Cylanee tools inunediately found the most critical samples of malicious code present at 0PM related to the breaches and that correspond to ?ndings of DIIS US-CERT. As of April 13?19, 2015, CylanceProtect was deployed (in Alert mode) to over 2,11)th devices, made ?tons of findings,? and as a Cylance engineer described the tool it ?lit up like a Christmas tree" indicating widespread malicious activities in IT Environment. former Director, Katherine Archuleta and former CIO Donna Seymour made questionable statements under oath about use of a quarantine to isolate malware and malicious process during the incident response. 0PM eventually purchased CylanceProtect on June 31.1, 2015, but only as it was about to lose access to the product {as the demonstration period was ending). Despite Cylance?s proven value during the 2015 incident response, 0PM failed to timely make payments. 16 On Jane 1? 0 20115 the Wait Street (PERU repaired that yiecii Services inc. network farce piatfat in atittaiiy discounted that data breach at 0PM in mid? Apt it during a sates demonstration. FINDING: CyTeeh, a service disabled veteran?owned small business contractor, did participate in several meetings with 0PM in early 2015 to discuss the capabilities of their CyTech Forensics and Incident response tool and provided a demonstration of their too] on April 21, 2015 at 0PM headquarters. FINDING: During the April 2] demonstration CyTeeh did identify malware on the live 0PM IT environment related to the incident. CyTech was not aware at the time that 0PM had identi?ed on April 15 an unknown Secure Sockets Layer (SSL) certificate heaconing to a malicious domain not associated with 0PM. FINDING: Beginning on April 22. 2015, CyTech offered and began providing significant incident response and forensic suppoit to 0PM related to the 2015 incident. FINDING: CyTech did not leak information about their involvement with the 0PM incident to the press. FINDING: The testimony given by the (new fen-her) 0PM CID, Donna Seymour, before the Committee on .1 une 24, 21115 regarding the CyTech matter is inconsistent with the facts on the record. FINDING: Documents and testimony show CyTee?n provided a service to 0PM and 0PM did not pay. The Anti?de?ciency Act (ADA) prohibits a l'ederai agency from accepting voluntary services. to he no cc [2214 Intrusions a; QEM The data breaches 0PM Sig?'ered in H.114 and 2fti5 share commonaiities t'eievant not only to attribution, but more importantly 0PM '3 reaction or tacit thereof'in the wake cfthe 20M intrusion. FINDING: The data breach discovered in March 2014 was likely conducted by the Axiom Group. This conclusion is based on the presence of Hikit malware and other Tactics Techniques and Procedures associated with this group, which have been publicly reported. FINDING: The data breaches discovered in April 2015 were likely perpetrated by the group Deep Panda (aka. Shell_Crcw, a.k.a. Deputy Dog) as part of a broader campaign that targeted federal workers. This conclusion is based on commonalities in the 2015 adversary?s attack infrastructure and TTPs common to other hacks publicly FINDING: FINDING: attributed to Deep Panda. These groups include Wellpoino?nnthem, VAE Inc., and United Airlines. However, the cyber intrusion and data theft announced by Anthem in 2015 is a separate attack by a separate threat actor group unrelated to the back against 0PM discovered in 2015. As publicly reported, both the Axiom and Deep Panda groups are highly likely to be statc~sponsored threat-actor group supported by the same foreign government. It is highly likely that the 2014 and 2014:9015 cyber intrusions into networks were likely connected and possibly coordinated campaigns. 1" aria: a! watchdogs play a c: af'tcaf rofe in thefeder a! government partnering with ageames to improve and safeguard programs and operations including (hitting and after data breaches. FINDING: FINDING: FINDING: The relationship between the UPM Office of the Inspector General (01(3) and Office of the Chief Information lElfficer became strained during the tenure of former Director Katherine Arehuleta and former CID Donna Seymour. The relationship became so strained that on July 22, 2015, then?InSpectOr General Patrick McFarland issued a memorandum to Acting Director Beth Cobert to share ?serious concerns? regarding the DCIO. Former 0PM Director Katherine Archuleta and former 0PM CIDI Donna Seymour engaged in activities that hindered the work of the DIG, including when: {l OCIO failed to timely notify the DIG of the 2014 and 2015 data breaches or the data that was compromised; Director Archuleta stated that the DIG could not attend certain meetings relating to the data breaches because the 010's presence would ?interfere? with the FBI and work; (3) The OCID failed to notify and involved OIG in a major investment to develop a new IT infrastructure; and (4) The DIG delayed an audit of KeyPoint Government Solutions at the request of the DCIO after an October 16, 2014 meeting, only to learn later 0PM knew in early September 2014 that KeyPoint had been breached and did not disclose this information to the 01G. Former 0PM Director Katherine Archulcta and former 0PM CID Donna Seymour made five incorrect andror misleading statements to Congress. These WEFBI (1) Director Archuleta testilied June 23, 2015 before the Senate Committee on Appropriations, Subcommittee on Financial Services and General Government, that 0PM completed a Major 1T Business Case (formerly known as the OMB ?Exhibit 300"} for the infrastructure improvement project; contrary to the ?nding ofthe 0PM 18 FINDING: At the same June 23, 2015 hearing, Director Archuieta testi?ed that ?my CID has told me that we have, indeed, an inventory of systems and data,? contrary to the ?ndings of the GIG in both a ?ash audit alert and the FY 2014 FISMA audit; Director Archuleta and CID Donna Seymour testi?ed before the Senate Appropriations Committee and the House Committee on Oversight and Government Reform that the sole?source contract with contractor (Imperatis) for the IT Improvement project covered only the ?rst two phases of this multiphase IT Infrastructure Improvement project, and contracts for the later phases [migration and cleanup) of the project had not been awarded. However, the BIG found that the sole?source contract provided for work under all liour phases of the project; 0PM CID Seymour testi?ed before the House Committee on Oversight and Government Reform on June 16, 2015 that the II 0PM systems operating without authorization were no longer a concern because she had granted an interim authorisation to these systems. However, the 1G found that OMB does not allow interim or extended authorizations; and At a June 25, 2015 hearing held by the Senate Committee on Homeland Security and Goverrunental A?airs, Director Archuleta stated that 0PM had received a special exemption from OMB related to system authorization because of the ongoing IT Infrastructure Improvement project; however, this claim could not be substantiated. The relationship between the OPM DIG and 0PM leadership has improved under Acting Director Beth F. Cohert. Chapter 3: Findings. Related to the IT Infrastrn?ure Project In response to the data breach at 0PM in 201' 4, and o?er identi?ring serious in the OPM network. the agency. or the recommendation initiated the IT Improvement project. FINDING: FINDING: FINDING: IT Infrastructure Improvement project is a case study illustrating why agencies need to ensure robust communications with the OIG, panicularly in responding to cybersecurity incidents. Former 0PM CID Seymour said she was not aware ofa requirement ?to notify the 1G ofevery project that we take on.? use of a sole~source contract in an emergency situation illustrates why there should be pro?established contract vehicles for cyher incident response and related services. There is a pressing need for federal agencies to modernize legacy IT in order to mitigate the cybersecurity threat inherent in unsupported, end of life IT systems and applications. 19 Recommendations in 201? 5 0PM announced the iargest data breach ofpersonaiiv identi?abie information (Pit) of 22. i Americans. ihis?riiure ofcuiture anti ieadership cannot happen again. The federai government must recognise and mitigate the ever?increasing cyher threat and protect the information that Americans entrust to the government. White there was much that went wrong for years in the federoi government approach to information security, this episode presents an opportunity for Congress and other agencies to inject new ieariershi}? andr a cuiture of security in federai it". The recommendations iisted beiotv are aimed at taking iessons iearnerifrom the 0PM experience andT charting a path of ever vi giiant i1" security in order to secure the PH of Americans heia' hy the ferterat government. Recommendation 1 Ensure Agency CIOs are Empowered, Accountable, and Competent Each federal agency must ensure agency CIOs are empowered, accountable, competent and retained for more than the current average two year tenure. The C10 at federal agencies and independent executive agencies is a critical leader who should be accountable to the head of the agency. Under federal laws, such as the Federal Information Security Management Act (FISMA) and the Federal Information Technology Acquisition Reform Act CIDs are responsible for IT security and management functions within the agency. In the last two years, Congress revised FISMA and FITARA. to reflect the new prioritization agency heads should place on IT management and security. CIOs typically serve an average of two years, but greater priority should be placed on retaining these leaders for at least ?ve years.55 This Committee, and in particular the IT subcommittee, has made IT management and security an oversight priority to ensure vigorous implementation of FISMA and FITARA. Such oversight has included a FITARA scorecard to assess agencies? implementation ofthis law. This oversight will continue and agencies will he expected to ensure there is an empowered, accountable, and competent C10 serving in this critical role. Recommendation 2 Reprioritize Federal Information Security Efforts Toward a Zero Trust Model OMB should provide guidance to agencies to promote a zero trust IT security model. The 0PM data breaches discovered in 2014 and 2015 illustrate the challenge of securing large, and therefore high-value, data repositories when defenses are geared toward perimeter defenses. In both cases the attackers compromised user credentials to gain initial network access, utilised tactics to elevate their privileges, and once inside the perimeter, were able to move throughout network, and ultimately accessed the ?crewnjewel? data held by 0PM. The agency was unable to visualise and log network traffic which led to gaps in knowledge regarding how much data was actually ea?ltrated by attackers. To combat the advanced persistent threats seeking to compromise or exploit federal government 11" networks, agencies should move toward a ?zero trust" model of information security and Ti 55 Gov?t Accountability Of?ce, 1-1534, Fetter-at Chief information O?icers: Opportunities Exist to improve Role in hn?in'motion chhnoiogv Management (Del. 20] l} [stating the average ClD?s tenure is two years). 20 architecture. The zero trust model centers on the concept that users inside a network are no more trustworthy than users outside a network. 5? The zero trust model requires strictly enforced user controls to ensure limited access for all users and assumes that all traffic traveling over an organization's network is threat traffic until authorized by the 1T team. In order to effectively implement a zero trust model, organizations must implement measures to visualize and log all network traf?c, and implement and enforce strong access controls for federal employees and contractors who access government networks and applications. Recommendation 3 Reduce Use of SSNs by Federal Agencies Federal agencies should reduce the use of Social Security Numbers in order to mitigate the risk of identity theft. SSNs are key pieces of that can potentially be used to perpetrate identity theft. The potential for misuse of SSNs has raised questions about how the federal government obtains, uses, and protects the SSNs it obtains. In May 2007?, OMB required all federal agencies to review their use of SSNs in agency systems and programs in order to identify opportunities to reduce such use.57 Agencies were required to establish a plan, within 120 days of the memo, to eliminate the unnecessary collection and use of SSNs within 18 months. They were also required to participate in government-wide efforts to explore alternatives to the use of SSNs as a personal identifier for federal employees and in the administration of federal programs. In response to a 2016 request by Chairman Challeta, the US. General Accountability Of?ce (GAO) is currently reviewing actions agencies have taken to reduce the use of SSNS actions OMB has taken to ensure agencies have adhered to its directive, and what progress has been made in reducing the use of SSNs across the federal government. Congress should carefully monitor the progress of these important actions, and work with agencies to ensure steps are taken to efficiently and effectively reduce agency use of SSNs. Recommendation 4 Require Timelv Justifieations for Lapsed Authorities to Operate Agencies that fail to re?authorize the authorities to operate [or their critical federal systems should be required to provide Congress, within [5 days of the system?s authorization expiring, a justification as to why the system authorisation was allowed to lapse. Designated critical information systems lacking adequate justification for a lapsed ATO should he removed immediately from the production environment. ATOs provide a comprehensive assessment ofthc IT system?s security controls and are a vital part of ensuring federal systems operate securely. FISMA requires agencies to assess the effectiveness of their information security controls, the frequency of which is based on risk but no less than annually. (3th Circular A-130, Appendix required agencies to assess and authorize (formerly referred to as certify and accredit) their systems before placing them into operational environment and whenever there is a major change to the system, but no less tinni- 5" This model was proposed by Forrester Research inc, an American?owned independent research and advisory ?rm. in response to a 2313 National Institute of Science and Technology (MIST) request for information entitled, ?Developing a Framework to Improve Critical Infrastructure Cybersecurity? NIST 1332G31 19?31 19-01. See T3 Fed. Reg. 131324 (Feb. 26, 21313) available at: commentsf040313 ferrester 57 Memorandum from Office of Mgmt. it}. Budget, Exec. Of?ce of the President, to the Heads of Exec. Dep'ts Agencies, d, Safeguarding Against and Responding to the ofPenronnr?tv frienry'r'ahie fight-motion (May 22, available at? 2 I every three cars thereafter. 53 At 0PM, critical systems were operating in FY 2014 without a valid ATO. 9 Of the 21 0PM systems due for reauthorization in FY 2014, 11 were not completed On time and were operating without a valid authorization,? and several were among the most critical, containing the agency?s most sensitive information.Ell This led the IG to warn 0PM that ?[t]he drastic increase in the number operating without a valid Authorization is alarming, and represents a systemic issue of inadequate planning by 0PM program of?ces to authorize the information systems that they A failure to maintain current ATOs negatively impacts the security of federal information systems. As the OPM pointed out, ?there are currently no consequences for 0PM systems that do not have a valid Authorization to Consequently, agencies should account for lapses to Congress and be prepared to take critical systems out of production. Further, at 0PM, the IG reconunended the adoption of administrative sanctions for the failure to meet security authorization requirements.?34 Congress and the Administration should consider options (including legislation or policy guidance) to ensure there are appropriate consequences for lapsed ATDs. Recommendation 5 Ensure Accountability and Empower DOD IT Of?cials Implementing Necessary Security Improvements for NBIB Clear rules for accountability and dedicated fending should be established by the end of FY 2017 to ensure the US. Department of Defense is successful in securing the background investigation materials that will now be held at the new National Background Investigations Bureau (N BIB). In an effort to reform the background investigation process and secure related data, this function will now reside at the new NBIB and the DOD CID will be responsible for The DOD CIO has testi?ed that he will ultimately answer to the Secretary of Defense in matters relating to were and that non will provide short-term funding rat IT at note.? it a Budget, Eaee. ornee of the President, oars Circular a?isc, Management errederai information Resources (Nov. 28, available at: alJD al3?tran34l. OMB Circular tat-13G was recently Updated and includes new guidance for agencies on Authorization to Operate and Continuous Monitoring. Of?ce of 3.: Budget Exec. Office of the President, OMB Circular tit?Bil Management of Federal Infonnation Resources {July 27, available at: The Committee expects to continue oversight in the areas covered by the revised A-l3?. 5" oniee of the Inspector oen, us. or?ee of Fcrs. Mgint., Rupert No. eat-er aura-ma, retreat in amtatiea t'iiccurityJ Management Act Audit FY Jill 4 (Nov. 12, 2014} available at: goviour?i nsnector? ma na gemen -act- a_udit-fy-_2tl 14-4 a-ei -lli.i- 1 4-0 1 ?pdf Id. at 9. E-mail from Inspector Gen. Staff, U5. Office ofPers. Mgnit., to H. Comm. on lit. Clov't Reform Staff Epee. 4, 21115} (on ?le with the Committee). '3ch of the Inspector Gen, ILLS. lL?Jlfi'ice of Pers. Mgmt., Report No. art?c1 ??U?l4?Dl 6, ecleral liner-titration Security Management Act Audit FY al 9 (Nov. l2, 2014) available at: l4itederal-informat i on-sec uritv- man agent ent-act-audit-fv-ZO 1-1-4a-c i lit-til tied 3 in. at to. 5? Iii. at i. '55 White House, Press Release, The Way Forwarn?for Federal Background lurertigatimrs (Jan. 22, 2016), 1 till wa y?forwa rd ions. Security Clearance Reform: The Performance Accountability Council '3 Pail: Forum-ti: Hearing Before the House (forum. on Over-sight til Gut: 't Re?u-m, I 14th Cong. (Feb. 25, Kilo) (testimony of Terry Halvorsen, Chief Info. Officer, U.S. Dep?t of Defense}. 22 However, it is not yet cica1 whetl1e1 future 1T funding will come fiom DOD, 0PM, or another source" It 1s also unclear how disagreements between DOD and 0PM regarding IT security spending would be resolved.mi To ensure that security is appropi iately prioritized at NBIB, 0PM and DOD should establish clear soulces of funding and decision? making processes for IT security, and the DIG at both 0PM and DOD should work to oversee such implementation and management. Recommendation 11 Eliminate Information Security Roadblocks Faced by Agencies To the extent there are non-security related bureaucratic hurdles to quickly implementing IT security policies and deploying cyber tools, agencies should make every effort to streamline processes and prioritize security. The federal government?s most important responsibility is to protect this nation and our citizens including when it comes to protecting this nation against The process of deploying security tools can be cumbersome and requires navigating a bureaucratic process that may involve notifying unions and overcoming program manager opposition.'5 9Congress should enact legislation sponsored by Rep Gary Palmer 1n the House 4361) and Senator Joni Ernst 2935) to clarify agencies authority under FISMA by Stating the heads of federal agencies are able to take timely action to secure their IT networks, and without being required to ?rst provide unions with the opportunity to bargain. Recommendation 7 Strengthen Security of Federal Websites and Breach Noti?cations Congress should enact H.111. 451, the Safe and Secure Federal Websites Act of 2015, legislation sponsored by Rep. Chuck leischn1a1u1 that increases the certi?cation requirements for public federal websites that process or contain The bill requires an agency?s C10 to certify the website for security and functionality prior to making it publicly accessible. The bill also increases the requirements for agencies when responding to an information security breach that involves PII. The events that unfolded at 0PM in 2014 and 2015 demonstrated an unwillingness by some of?cials to notify the public ofa PII compromise in a timely manner. The bill directs OMB to develop and oversee implementation of the certification requirements, which include reporting the breach to a federal cyber security center and notifying individuals affected by a compromise. Recommendation 8 Financial Education and Counseling Services Through Employee Assistance Programs Congress should encourage federal agencies to provide federal employees with ?nancial education and counseling services that are designed to help employees recognize, prevent and mitigate identity theft through existing Employee Assistance Programs (BAP). An EAP is a voluntary, work-based program that offers free and confidential assessments, short-term 5? at frf. 69 in the case erPM?s efforts to deploy a tool called Forescout {which is a tool to manage network access control for devices}, there were deployment delays due in part to the need to notify unions. Imperatis Weekly Report (Aug. 3, 201 S-Aug. T, 2015}, Attach. at ?tltl942 (lrnperatis Production: Sept. 1, [stating ?project sponsor is in noti?cation stage with the Union? and mitigation was to "prepare updated project timelinc, plan 3: memo to pilot ForeScout to non-union agency users?). 23 counseling, referrals, and follow-up services to empioyees who have personal andtor work- reiated problems.ml Recommendation 9 Establish Government-wide Contracting Vehicle for Cyber Incident Response Services OMB and the General Services Administration (GSA) should lead efforts to establish a govemment-wide contracting vehicle for Cyber Incident Response Services or Congress should establish a statutory requirement for such a vehicle. After the data breach discovered in March 2014, 0PM awarded a sole source contract for a multi-phased IT Infrastructure Improvement project. Under this contract, 0PM procured cyberseeurity tools to secure their legacy IT environment. Instead of duplicative sole source contracts across various agencies, the federal govemment should have pre-established contracting vehicles that have the bene?t of competition and are available to provide incident response services, including tools to secure IT environments post?breach. Agencies should not be in the process of establishing contracts [or these services during the incident response period. In October 2015, OMB published a Cyber Security Strategy and Implementation Plan for the federal civilian government agencies.? The CSTP included a nutnber of deliverablcs, including one related to establishing contracting vehicles providing incident response services. A govemment?wide contracting vehicle for incident response services sltould be established as soon as possible and before another agency faces the same situation as 0PM. This will ensure such contracting vehicles have the bene?t of competition and provide a robust suite of services to assist agencies in an incident response scenario. Recommendation 10 Improve and Update Cybersccuritv Requirements for Federal Aeguisition OMB should refocus etforts on improving and updating the current patchwork and outdated eyberseeurity requirements in existing federal security and acquisition rules. There have been a number of initiatives launched over the last few years to update and improve cyberseeurity requirements in federal acquisition- To date, few of these efforts have been ?nalized. Thus, the Committee recommends that the Administration prioritize and complete efforts to develop and implement clear cyberseeurity requirements for federal acquisition as soon as possible. The importance of the partnership between agencies and federal contractors in securing sensitive data held by agencies and contractor-operated systems cannot be overstated. Existing eyberseeurity rules and requirements in federal acquisition are ad hoe, overlapping, potentially con?ict and are in need of updating. In Febmary 21113, the President issued Executive Order 13636, Improving Critters! Cyberseettrt'ty and Presidential Policy Directive (RFD) 21, Critieoi Security and Robotics, that directed agencies to complete a broad range of tasks to enhance national 7? What is on Emptoyee Assistance Progrotti, U.S. DFt-?It?e OF Peas. Mtitvrr, available at: govt 1' aqstDA .asns'i'tid=43 I 3c? I E?a?oe?a-eEe-b?i?tt- 1 new 2a] Dd? Sroid=2c2b I 4949: b4?3?34035}a1e1 1T4. 1' Meanolaltduln from Shaun Donovan, Din, and Tony Scott, Fed. Chief Info. Of?cer, Office of Mgmt. 3: Budget, Exec. Of?ce of the President, to Agency lleads, M?lo??d, vaetsecttritv Strategy and fnmfementottrm Ptonfor the Fetter-oi Civilian Government (Get. 312}, 2015] available at: go vt'sitestdc l'aulnlr lestom ht emoranda??l dtm- f. 24 cybersecurity and resilience. 72 lOne group of deliverables included a mandate to incorporate cybersecurity requirements into the federal acquisition process. In January 2014, GSA and DOD delivered a report, Improving anti Reliance tin-engii Acquisition that made recommendations to achieve this obj ective.? These report recommendations have not been implemented to date- The existing framework for cyberseeurity requirements in federal acquisition should be reviewed and updated immediately. The January Zill 4 report recommendations provide useful guidance to inform such an update. Recommendation 11 Modernize Existing Legacy Federal Information Technology Assets Federal agencies should utilise existing tools and Congress should consider new tools to ineentiviae the transition from legacy to modernised IT solutions. Federal agencies spend over $89 billion annually on IT, with the majority of this spending focused on maintaining and 7-4 . . . 75 operating legacy IT systems. Over I5 percent of this spending is focused on legacy IT costs. GAO reported legacy IT investments are becoming increasingly obsolete with outdated software languages and hardware parts that are not supported.? Such reliance on legacy IT can result in security vulnerabilities where old software or operating systems are no longer supported by vendors and aging IT infrastructure becomes dif?cult and expensive to secure. OFM testi?ed before the Committee there ?are some ofour legacy systems that may not be capable of accepting these types of The solution to this legacy IT challenge must be multifaceted and should include the use of existing and new tools to incentiviae modernization. FITARA provides important tools for IT management and acquisition, including facilitating the transition from legacy IT to modernised solutions.?8 In terms of new tools, incentives for agencies to achieve savings modernization and innovative financing options to promote modernisation should be considered. Recommendation 12 Agencies Should Consider Using Critical Pav for IT Securigy Specialists: Agencies may request and be granted ?critical position pay? authority. Agencies may request critical position pay authority only after determining the position in question cannot be ?lled Lines. Order No. l3?3ti, TE Fed. Reg. 1139 {Felt}. 19, Z?lli}; White House, Press Release, Presidential Policy Directive El, CriticruT Security Rciiouce r(Feb. ll, 2013). 73? Gen. Serv?s Admin. Dep't of Barnes, improving anti ?it-cogs Acquisition (Nov. 2?13), available at: rtalimedialdi roving_cybersecurity and resi Ii cnce_through_acquisition. action. 7" The annual total billion For IT understates the federal government's total IT investment because it does not include: DOD classi?ed IT systems, IT investments by 58 independent executive branch agencies [including the and IT investments by the legislative orjudicial branches. Data available through the IT Dashboard, and OMB Df?ce ofE-Gov and Information Technology, T5 Gov?t Accountability Of?ce, GAD-145463. .iqfonnoiion Technologv Fedcroi Agencies Need to Address Aging Legacy Systems, {May ?5 Id. Dom Brooch: Hearing Before the H. Comm. on Uvemigfti Gov 'i Reform [June l6, 2m 5] (testimony of Donna Seymour, D?iccr, US. Of?ce of Pers. it National Defense Authorization no Fv 2m 5. Pub. L. No. 113-291, Title vm, Subtitle n, 123 star. 3292, 3433- 513 (Dec. 19, 21314]. 25 with an ?Exceptionally well-quali?ed individual" through the use ofother available human resource ?exibilitics and pay authorities. 0PM, in consultation with OMB, reviews agency requests. When approving a request, 0PM must determine whether the position requires an ?extremely high level of expertise? in a ?scientific, technical, professional, or administrative field? and is mission critical. Authority is used to recruit andfor retain exceptional talent, and is capped at 300 positions at any one time. Generally, critical pay may be established up to Cabinet Secretary pay levels ($205,700) and can be increased with approval by the President (but pay and bonus generally cannot exceed the vice president?s salary). The Committee intends to collect more information on the use of critical pay authority in order to conduct appropriate oversight and tnake adjustments to the authority, and to ensure it provides agencies the necessary ?exibility for recruitment and retention of lT security talent. 0PM should also consider establishing a pay band for information Technology Security Specialists. Recommendation 13 Improve Federal Recruitment, Training and Retention of Cyber Security Specialists Recruiting, training, and retaining cyber security specialists should be a critical national security priority. Following the oyberattacits at 0PM, the federal CIO and the OMB Director issued a Memorandum concerning a cybersecurity strategy and implementation plan (CS IP) for the federal civilian government? The CSIP included several federal cyber workforce related taskings, including directing: I. 0PM and OMB to compile special hiring authorities by agency that can be used to hire cyber and IT professionals across government. 2. Agencies to participate in Cyber Worig?irrce Project an effort to code cyberseeurityjobs by specialty tor the purpose of gaining knowledge about the gaps and challenges in cyber recruitment and retention. 3. DHS to pilot an Automated Cybersecurity Position Description Hiring Tool to assist in implementation of the National initiative for Cybersecurity Education (NICE) framework, and posting analysis of the cyber workforce on the C10 Council?s knowledge portal as a best practice for other agencies to follow. 4. 0PM, DHS, and OMB to map the entire cyber workforce across all agencies using the NICE National Cybersecurity Workforce Framework. 5. 0PM, D115, and 0MB to develop recommendations for federal workforce training and professional development. The Administration and Congress must work together to complete these tasks and swiftly take steps needed to recruit, train, and retain a world class cyber workforce. The Committee notes Memorandum From Shaun Donovan, Din, and Tony Scott, Fed. Chieflnfo. Of?cer, Of?ce of Mgmt. Budget, Exec. Df?ce of the President, to Agency Heads, bide?04, Err-mega and implementation Planjbr the Civil'irm Government (Del. 3D, 2015} available at: still] 1 trim?l o?Udpd f. 26 OMB and 0PM jointly transmitted a memorandum to agency heads on a Federal Cybersecuritv Workforce Strategy on July 12, 2016 and appreciates this opportunity to continue the dialogue in this area. Finally, Congress and the Administration should consider non-traditional mechanisms to recruit and retain cyber talent. Such mechanisms should complement private sector experience rather than compete with the private sector, recognize the need to quickly hire top talent, and provide an opportunity for public service to those in the private sector. 27 Table of Names Of?ce of Personnel Management Name Title Katherine L. Archuleta Director (May 21313 - July EDIE) Morrell John Berry Director [April 21309 - April 2013) Beth F. Cobert Acting Director {July Zill 5 present) Jason K. Levine Director of?f?ce of Congressional, Legislative, and Intergovernmental Affairs [August 2015 present] Patrick McFarland Inspector General {August 199?~February 21316] Lisa Acting Chief Information Of?cer (March - August 2016) Donna K. Seymour Chief Information Of?cer {December 20 E?February 2.016} Special Agent in Charge Of?ce of Inspector General Linda M. Springer Director (June Clifton {?Clif?) N. Senior Cyber and Information Technology Advisor Norbert [?Eert") E. Vint Acting Inapector General [February 21316 - present) Deputy Inspector General EU 1 I5) Jeff P. 1 blir'agner Director of Information Technology Security Operations Assurance Data, Inc. Name Title Matthew Morrison President and Chief Executive Of?cer Cyicnce Inc. am: Title Chris Coulter Managing Director of Incident Response and Forensics Stuart McClure Chief Executive Of?cer, President and Founder [3 rant Moersc hel Director of Sales Engineering Nicholas Warner 1'ir'ice President of Worldwide Sales Services Name Title Juan Eonilla Sr. Security Consultant Solutions Engineering (with 0PM April 23-May 1, 5) Ben Cotton Chief Executive Ut'?cer SBA Name Title Brendan Saulsbury Senior Cyber Security Engineer [March 21312 May 2016] Jonathan Tonda 0PM Branch Chief, Security Engineering (September 2015- present); Network Security Team Lead, ERA [May Zilli- Septernher. 2t] 1 5] 28 Imperoris Name Title Patrick Technical Lead for CHM contract Misc. Name Title Joel Brenner Former National Security Agency Senior Counsel James B. Conley. Jr. Director of the Federal Bureau of Investigations Michael V. Hayden Former Director of the Central Intelligence ngnoyir James Andrew Lewis Senior 1'rr'ice President and Director, Strategic Technologies Program, Center for Strategic and International Studies Jeff'Neal Former Chief Human Capital at the US. Department of Homeland Security John Schindler Former National Security Agency of?cer Richard A. Spires Former Chief Information Of?cer at the US. Department of Homeland Setsuril}r and the Internal Revenue Service 29 Chapter 1: IT Security Record Preceding Breaches The attackers who successfully penetrated the US. Of?ce of Personnel Management network were sophisticated, but neither their methods nor their ambition was unprecedented. The federal government had been subject to attacks for years by the same or similar groups using similar variants of malware. In fact, 0PM had reportedly been hacked in 2012. A vast amount of publicly available information on similar hacks within the past decade was available that should have put 0PM on notice. Furthermore, 0PM had every incentive to prioritize information security given the volume of sensitive information and PII it holds. Despite red ?ags that began as early as 2005, appropriated IT security funding consistently lagged behind other agencies, its most sensitive data was inadequately protected, and 0PM leadership failed to heed recommendations from The Rise of Advanced Persistent Threat Hacking The longstanding 0PM cyber security failures that culminated in the theft ofpersonnel records, background investigation data, and ?ngerprint data began a decade earlier when the federal government was put on notice regarding the nature ofthe threat. In July 2005, the LLB. Computer Emergency Response Team issued an alert regarding sophisticated, multi? year efforts in which hackers send targeted, socially-engineered emails (commonly called ?Spear phishing" emails) for the purpose of having a user download a ?le that would eventually lead to the ex?ltration of sensitive information.Eml Though the term would not emerge for several years, the alert described what would come to be known as an ?advanced persistent threat" attack. Such attacks are focused on a particular set of high-value assets or physical systems with the explicit purpose of maintaining access and of stealing data overtime. Because the attackers are sophisticated, they can learn how to jump from system to system within a given network, often attempting to compromise administrator accounts in order to gain wider and higher levels of access and creating new footholds to maintain their access. When a particular security precaution or obstacle prevents further compromise, the attackers change tactics and maintain a presence on the network until they reach their ultimate objective. The 2005 alert noted that APT attacks had already taken place, and that they often used malware speci?cally designed to elude anti-virus software and firewalls.BI The alert specifically noted the use of?McAfec" and ?Symantec? names in connection with APT hacks, foreshadowing the ?McAfee? name that would later be relevant in the 0PM breach? Since 2005, the federal government has been repeatedly victimized by sophisticated, sustained 1? attackers. In 2005, an APT intrusion gathered data from Vehicle :0 Technical thither Security Alert fingered i'i?ojrrn Attacks {July 2005). Id. 31 Id; see {rise Saulsbury Tr. at 60. 30 Assembly Building? Media outlets reported that Chinese involvement in the back was likely. 34 In 2002, James A. Lewis of the Center for Strategic and Intemational Studies testi?ed before Congress that intrusions occurred at the Defense Department, State Department and the Commerce Department";5 In late 2014, a media report catalogued a number of recent attacks against federal entities, including the White House, the State De artment, the United States Postal Service, 0PM, and the Nuclear Regulatory Commission. 6 Federal Contractors Holding Sensitive Federal Employee Information Targeted and Attacked In addition to the targeting of federal agencies, the government contractors that provide services to these agencies and hold sensitive federal employee information increasingly have been targeted by APTs, including several 0PM contractors that provide background investigation and healthcarc services. The ?rst public reports of data breaches involving 0PM contractors surfaced in the summer of 2014. In August 20 4, the largest background investigation contractor, U.S. Investigations Services, LLC publicly acknowledged a data breach impacting employees of the Department ofHorneland Security.BE Documents and testimony provided to the Committee indicate that USIS ?self?detected? this cyber?attack in June 2014, immediatel noti lied 0PM, and by early July 2014 had mitigated the attackers? activity on their systems. 9 In a June 22, 2015 document provided to the Committee, USIS said based on the results of an investigation, conducted by a company called Stros Friedberg, it was determined that [.1813 had been the tar at of an attack ?carried out by a state sponsored actor,? commonly referred to as an APT attack. USIS told the Committee that for otter 31,000 individuals associated with Keith Epstein a Ben Elgin, Network Seanrity Breaches Plague sass, Bus. WEEK, Nov- 2o, seas. id. 3?5 Hoiistia Approaches to Critter-security to Enabie Nearer}: Centric Operations: Hearing they?ll-re tire [in Terrorism, Unconventionai Threats and Capabilities afiire H. Comm. 0n Serif. I I 1th Cong. (Apr. 1, [statement of James Andrew Lewis]. 9'6 Jack Moore, Titre Yearafrtie Bleach: Federai' agency Data Breaches in 2914, Nexroov (Dec. 3i], 2GI4), 3" In 1996, USIS was established as a result ofthe privatization ofOPM?s Investigations Services and over the years was awarded a series of contracts to perform security clearance background investigations for more than 95 federal agencies. There were a variety oftransition iSsues when the privatization first occurred, including questions about USIS cmployees? access to government databases. See General Accounting Of?ce, Privatisation investigations Service (Aug. 22, 1996}. In September 2Dl 4, decided to end these contracts with USIS. In early 2015, parent company filed for bankruptcy. See ill Aitoro, it is G?ictat: USES is No More with Fianna-rt Aitegrity Banks-apron, Fins. 1., Feb. 4, 2015, 33 Ellen Nakashit?na, DHS Contractor Sii?'m?s Major Computer Breach, D?iciais Say, WASH. POST, Aug 6, 2014, available at: breaclt-of?ciaIs-sayi2t] ed 1 3 b4? :139? I 1e4-ae54atlcfe f9'i4f8 a_story.html. 39 Hearing on 0PM Data Breach: Part Ii [statement of Robert Giannetta, Chief Info. Of?cer, U.S. Investigations Services. LLC). an Letter from Counsel for U.S. Investigations Serv?s, LLC to the Hon. Elijah E. Cummings, Ranking Member, H. Comm. on Oversight it: Gov't Reform [June 22, 2D15): Id, Ex. 12, [Sires Friedberg Summary of investigation (Dec. 2014}. 31 USIS background investigation work for Customs and Border Protection, the National Geospatialdnteliigence Agency, Immigration and Customs Enforcement, and the U.S. Capitol Police ?may have suffered compromise in the [.1313 indicated this APT began in in late December 2013 and the last attacker activity was observed on July 4, 2014.92 The US IS investigation also determined that this APT was focused on access to computer systems related to the background investigations business of U818, which should have made it very clear to all stakeholders that the target was background investigation As a consequence of the USIS activity in the summer of 21] 14, LIB-CERT visited the facilities of Keyl?oint Government Solutions {KeyPoint} to do a network assessment, which found items of concern that prompted additional review.94 In December 2014, press reports indicated that KeyPoint had been breached resulting in the possible PII exposure of over 434100 federal employees:is In June 2015, KeyPoint CEO Eric Hess testified before the Committee saying, ?there was an individual who had an 0PM account that happened to be a KeyPoint employee and that the credentials of that individual were compromised to gain access to At the time of the 2015 data breach, 0PM gave contractors a username and password and investigators would log-in with this 0PM credential-? In addition, 0PM contractors holding sensitive healthcare information of federal employees have been the targets of APTs. In February 2015, Anthem, one of the largest health insurers in the country and provides coverage for 1.3 million federal employees, announced a data breach involving 30 million records ofcurrent and former customers and employees?:5 Then in March 2015, Premera, another health insurance company that has an 0PM contract (covering about 130,000 federal workers in Washington state and Alaska}, announced a data Letter from Counsel for US. investigations Serv?s, LLC (U313) to the Hon. Elijah E. Cummings. Ranking Member, i1. Comm. on Oversight Sr. Gov?t Reform at 5 [June 22, 2015]. 91 kt. at 543. In describing USIS activities related to the June 2014 discovery, [1315 noted that an employee of the forensic investigation ?rm (Stroz Friedberg] they hired attempted to provide additional forensic copies of hard drives with evidence of the attack on September 9, 2014, but the LIE-CERT employee declined saying CERT [was] on a stand downStroz Friedberg Summary of Investigation (Dee. 2014). 9" Hearing on 0PM Date Breach: Pant {statement of Ann Barron-DiCamillo, Director). ?5 See Christian Davenport, Keanr'nt Network Breech Cent's! Affect Thousands ofFea?ernt Workers, WASH. POST, Dec. id, 2014, t'ectediIf 21] 1 4f 12? Efedc'l' disc-Ede] -l 9" Hearing on 0PM Dem Breach: Part II (statement ofEric Hess, CEO KeyPoint Government Solutions); 011 June 29, 2015, the American Federation of Government Employees sued 0PM over the data breach and also named KeyPoint as a defendant in the lawsuit. 9? Saulsbury at Till?T1. Wagner. the UPM Director of IT Security Operations said multiple credentials were compromised during the 1015 incident, but a KeyPoint credential was likely used for the initial attack vector. Wagner added ?the adversary, utilizing a hosting server in California, created their own F18 [Federal Investigator Service, background] investigator laptop virtually. They built a virtual machine on the hosting server that mimicked and looked like a FIS investigator?s they utilized a compromise key point user credential to enter the nettvorlt through the FIS contractor VPN portal.? Wagner Tr. at 815, 123. 9" Reed Abelson a Matthew Goldstein, Mr'Hions' UfAHHtem Customers rage-tee a: NY. TIMES, Feb. 5, 2015, available at: Aliya 0PM Monitoring Anthem Hock; Feds Might be A?ected (Feb. 5, 2015] avaiiahle at: neatgov .co St??teaclusi ve?opm?monitori ng-anthem- hack-b reaclt?could?impact- 3 I'I?t?f?db'tl'l 1 32 breach that exposed medical data and ?nancial information [or 11 million customers!? These attacks highlight the persistent target that federal employee data presents and the need to secure such data whether it is maintained in a federal or a contractor-operating IT system. 0PM, as well as other agencies, faces the challenge of seeming their systems as well as overseeing the systems that government contractors operate on behalf of the government- In a 2014 report, GAO found that while agencies established security requirements and planned for assessments, the agencies reviewed (including 0PM) failed to consistently oversee the execution and review of these assessments.?m in response to recommendation to 0PM "to develop, document and implement oversight procedures for ensuring that a system test is fully executed for each contractonoperatot system. 0PM promised to review? ?existing security policies and p1oecdures"tol enhance their oversight. According to GAO 5 website this recommendation remains op en.' In the case of the OPM background investigation contractors who experienced data breaches in 2014 and 2015, 0PM had approved IT security plans for both USIS and KeyPcint. '03 In April 2015, GAO repeated the message about the need to address the cybersecurity challenge of ensuring effective oversight oi contractors implementation of security controls for systems contractors operate on behalf oi agencies. MBascd on testimony and documents submitted to the Committee, the record indicates that 0PM had not informed USIS or KeyPoint about the March 2014 data breach before it became public. '05 It is unclear whether the attack could have been mitigated if 0PM had informed their background investigation contractors, but given the threat environment and the background investiggtion systems targeted, it would have been prudent to alert the contractors ?itr11nediately.I 95' Blue Cross Says Ditto Breech Erpcsed Medical Dotti N.Y. TIMES, Mar. 1'1, 2?15, Elise Workers Might be Vietirrts Dem Breech, THF. 1-111 L, Mar.19,2015, m'?ov Accountability Of?ce. GAO- 14-612 ,Agericr'es Neetl to improve ?vexriglrt ofCorrtr?tretor Controls (Aug. http: pdf. lotw't Accountability Of?ce, one 14 612 ,Agettcim? Ntterlto centres as (Aug. 2014), http: gao govlassetsiISTDlE-o??l? .pdt. I02 Open {Lil U?l Iii-5i Agencies Need to improve Oversight ofCOttlt?tIolot' Controls {last visited July 2, 29115}, so. ovlrec rnrnendation 'lsearch?searched=ldrhiti? order by 10&now sort=seorerdesc3toaae [and] ts Hearing on ore: om arma- i?t?tt'l rt (testimony by Robert Giannetta, Chief Info. Of?cer, us. Services, Letter to the Hon. Elijah F.. Cummings, Ranking Member. H. Comm. on Oversight and Gov't Reform from Counsel for 1.1.5. Investigations Services, LLC (U515) {1 one 22, 2315], Ex. 8, 9, 113 (ATOs signed by 0PM and May 2014 Site Survey Assessment liorm); Hearing on Dotti Breech: Port [statement of Eric Hess, CED KeyPnint Government Solutions}; Fmail from KeyPoint Counsel to Majority Staff, H. Comm. on Oversight lit Gov Reform (Feb. 22,2(116) [on ?le with the Committee)- mENiI?HCltig Cybeiseetrr ity olerr'rti Forty .r and Vendors: Heating Before H. Comm. on Oversight Gov R?f?t'm 1 14th Cong. (Apr. 22, 2015} [testimony of Gregory C. 1Wilshusen, Dir. Info. See. Issues, Gov Accountability Of?ce]. n15 Hearing on 0PM Dotti Breech: Per-t ll [statement oI'Rol?rert Giannelta, Chief Info. Officer, U.S. Investigations Serv?s, Deepite a contractual obligation to notify contractors immediately of a ?new or unanticipated threat or hazard,? 0PM did not notify their contractors {KeyPoint and ofthe March 2014 incident. id '05 Hearing on 0PM Dotti Eretreli: Perl (Rep. Gowdy questioning of OPM contractors and 0PM of?cials on the de?nition of 33 Agencies today rely on federal contractors to operate IT systems on behalf of the federal govermnent and must access federal systems in order to perform services for the federal government. The potential risk of unauthorised access to IT systems operated by federal contractors on behalf of the federal government or contractors* IT systems should not have been surprising to 0PM in the years leading up to the data breaches. Federal Initiatives to Increase Information Security in Response to Increasing Attacks As the first warnings of APT attacks began in 2005, the federal government was beginning to strengthen access controls. On August 5, 2005, OMB issued guidance to implement a Directive requiring the development and implementation of a mandatory, govemment?wide standard for secure and reliable forms of identification for federal employees and contractors. The guidance (?Implementation of Homeland Security Presidential Directive (HSPD) l2 Policy for a Conunon identi?cation Standard for Federal Employees and Contractors") advised the heads of all departments and agencies that ?[ijnconsistent agency approaches to facility security and computer security are inef?cient and costly, and increase risks to the Federal government??UH The Administration issued implementation guidance in the immediate years after the 2005 Directive was issued. me In response to multiple attacks, in 2003, the federal government began a major new initiative to improve the security of its systems. 1 Meanwhile, attacks on federal systems continued and increased in volume and sophistication. Federal agencies only needed to look at attacks on government contractors and other private sector entities for a playbook about what they needed to able to counteract. In 2009, Chinese groups with ties to the People?s Liberation Army reportedly carried out dozens of APT attacks against, inter a'Iia, Notthrop Grumman, Lockheed Mania, and Dow Chemical.1 1' Memorandum from Joshua Bolton, Dir. Office of Mgmt. Budget, Exec. Of?ce oItIte President, to Dep?t and Agency Heads, [vi-05424, Implementation of?omeiand Seeurirv Presidentiai Directive (HSPD) i2 PoIiev?n- a Common Identification Standard for ederai Empioyees and Contractors (Aug. 5, 2005}. On August 2004, the President signed 2 ?Policy for a Common identi?cation Standard for Federal Employees and Contractors" {the Directive}. ?33 Memorandum from Joshua Bolton, Dir. Office of Mgmt. Budget, Exec. Of?ce of the President, to Dep?t and Agency Heads, Pvt-0544, Implementation ofHomeiand Security I?residentiai Directive I2 Poiievfor a Common Identification Standard Jfor Federai Employees and Cotiit'oci'rnw (Aug. 5, 2005]. Memorandum from Karen S. Evans, Admin'r, Office ofE-Gov't EL Info. Tech, Exec. Of?ce ofthe President, to Chief lnfo. Of?cers, and Senior Agency Of?cials for Privacy, Sangria Privacy Documents for Agencv Implementation (if-Homeland Secttriot I?t'aririeittiai Directive i 2 {Feb. l'i, 2006}, See aiso Exec. Of?ce of the President, PICSS Release, HSPD-IE Certi?ed Products and Services Now Avoiiobi'eforAgency Acquisition {July 5, 2006:), no National Security Presidential Directive 54 Policy (Jan. 3, 2008] available at: 1" ayyaz Rajpari, Finding the Advanced Fettrt'stent Adversary, SANS (Sept, 29, 2014), 5512. 34 Four years later, the situation had not improved and appeared to be getting worse. A 2012 white paper by FireEye stated: Federal agencies are increasingly the victims of advanced persistent threats, often comprised of multi-staged, coordinated attacks that feature dynamic malware and targeted spear phishing emails. In fact, in spite of massive investments in IT security infrastructure, on a weekly basis, over 95% of organizations have at least 10 malicious infections bypass existing security mechanisms and enter the network. Further, 80% experience more than 100 new infections each week. Every day, mission-critical systems are compromised, and sensitive and classi?ed data is enliltratcd from federal government and civilian networks' '2 0PM itself was also targeted in the years leading up to the breaches discovered in 2014 and 2015. In May 2012, a hacker reportedly broke into an 0PM database and stole 3? user 1135 and passwords. That breach was reportedly carried out by a group called ?@klldetec,? an activist af?liated with the hacking group Anonymous' In 2011, the Department ofHomeland Security issued a cybersecurity bulletin that called Anonymous ?script kiddies? using ?rudimentary? exploits. If true, Anonymous did not need advanced technical proficiency to gain access to an 0PM database.115 0PM Failed to Recognize the Threat and Implement Effective IT Security Measures When It Mattered The threat of APTs was well?known throughout the federal government and 0PM was a prime target given the sensitive information it held on current and former federal employees and contractors. Thus, 0PM should have made infomation security a top priority. In the years preceding the breaches at 0PM in 2014 and 2015, however, information security was just one of several competing agency priorities, and network vulnerabilities became more acute. In late 2013 and early 2014, under Director Katherine Archuleta and (310 Donna Seymour, 0PM attempted to re-l'oeus on improving IT security. It did not work. Ineffective leadership and poor decision-making plagued the agency during a critical period in 2014, leaving the agency in a weak position to prevent the breaches. ?1 Cram-snacks on Government: How APTAttncil?s are Contprontiring Federal Agencies end How to Stop Them {20 12), rcyel?imagesf? reeye-cyher?attac Paul Rosenzweig, The Alarming Trend onybei?smrm'ry Branches and Failures in the Government Continues, HERITAGE Fotmn. (Nov. 13, available at: 2i? 1 continue {citing Privacy Rights Clearinghouse Chronology of Data Breaches available at: see also Plaintiff?s Class Action Complaint and Demand for Jury Trial, 2] {Aug 14, 2015), Krippendorf v. US. Of?ce efPersonncl Mgmt., D.D.C. (No, 1:15 CV {11321) at 21 available at: http?ble gsreuterseeml? alison?frankeli'?les1 2t} 1 5.1031? kripp 1? Lee Johnstene, LES. O?iee ofPersonne.? Management Hacked (E Dem Leaked by @k?detee, CYBER Was NEWS, May 13, 201 2, available at: That individual also carried out an attack on the Glade County Florida Sheriff?s department ?5 Nat?l Cybersecul?lty ill: Comm?n Integration Ctr., Dep?t ofHomeland Sec., Bulletin Will). 35 0PM consistently reported spending less than other federal agencies on In 2013, FY 2014 and FY 2015, 0PM spent seven million each year on eybersecurity? spending that was consistently at the bottom relative to all other agencies that are required to report such expenditures to the Of?ce of Management and Budget. 1 '5 The previous ?scal year, 2012, 0PM also lagged behind other federal agencies. 0PM sought additional funds for eybersecurity, but only after US-CERT noti?ed the agency about the damaging breach in 2014. On March 20, 2014, Computer Incident Reaponse Team (CIRT) received notification from that data was being ex?ltrated from network. 1 In the weeks that followed, 0PM leadership would become aware the intrusion led to the breach of background investigation data in 0PM systems holding the ?crown jewels? ofthe American federal workforce and national security personnel. I ?5 0PM requested additional cyberseeurity ?mding in its FY 2016 Budget Justi?cation (released February 2015), and only then {ten years after 0PM took over the background investigation function) acknowledged it was a target rich environment. In a February 2, 2015 letter to the House Appropriations Subconunittee on Financial Services and General Government concerning its budget request, then-Director Katherine Arehuleta noted: Y201 6 request is $32 million above our FY 2015 appropriation, Most of these funds will be directed towards investments in IT network infrastructure and security. As a puoprictor of sensitive data?including personally identi?able information for 32 million federal employees and has an obligation to maintain contemporary and robust cybersocurity controls." I After years of neglect, the request for increased funding in February 2015 was too little too late. it came more than one year after attackers stole security documents that provided a roadmap to systems. '20 And the request came after hackers had already successfully eafiltrated sensitive data, including background investigations data in July and August of 2014 and federal employee personnel records in December 2014. '21 See lnf'ra, Report Appendix: Cyber security Spending at 0PM (Fiscal Years 2012-2015]; see also Uf?ce of 3.: Budget, Exec. foice of the President, Annual Report to Congress: Federal lin?orniatlon Security Management Act 32 (Mar. 13, 2015} available at: doesl f1 nal fy 2015 ?sma report to congress 03 13 2015.pdf.3ee also Office of Mgmt. 8.: Budget, Exec. Of?ce of the President, Report to Congress: Federal lnforrnatlon Security Management Act 33 (Feb. 2015) available at: docsl?nal_fv14__?sma report 02 27 2015.udf. ?i June 2014 oeM Incident Report at Hoones 13-001233. ns June 2014 0PM incident Report at HOURUSIE $01245. nu U5. Of?ce ochrs. 0PM Congresnlonal Badge! Justification l?erfonnanee Badge! 1?20! at 2 (Feb. 20 15}, ww.oprn. guvl about-uslbud get-performanceIr bud getslcon gressionai?budget?j ustification?fy20l o.pdf. June sow Incident Repelt, at Hooaosls amass. 0PM ILlyberseeurity Events Timeline. 36 The year 2005 was a key year for both 0PM and federal cybersecurity. The IG and US- CERT issued a general technical alert, which should have made 0PM aware of the need to increase IT security in the face of increasing APT threats, '22 and 0MB was gearing up to announce and begin implementation of HSPD-IZ.123 The 0PM 10 also issued a warning in a semiannual report that would be repeated in subsequent reports. It warned: 0PM relies on computer technologies and information systems to administer programs that distribute health and retirement bene?ts to millions of current and former federal employees and eligible family members. Any breakdowns or malicious attacks hacking, worms or viruses) affecting these federal computer based programs could compromise ef?ciency and effectiveness and ultimately increase the cost to the American tartpayer.?24 Amidst efforts to tbrti t'y federal cybersecurity, 0PM was also working in 2005 to assume responsibility for the processing and storage of federal background investigations. 0PM accepted the transfer of the Personnel Security Investigations function and personnel from the Department of Defense?s Defense Security Service authorised by the National Defense Authorization Act of 2004 (at. 108-136). '15 The transfer from ass to Federal Investigative Services (F18) division ?brought under one roofa unit that is conducting 90 percent of background investigations for the entire Federal Government.?u? Congress applied pressure on 0PM to process the background investigation caseload more ef?ciently by tasking with meeting timeframes imposed under The intelligence Reform and Terrorism Prevention Act (PL. 103-458). This was an important function in the wake of '33 Lia-cant", Tecitnicai Cyber SecarityAiert mortars; rergereer Trojan amen Attacks [July 2005). '13 Memorandum from Joshua Bolton, Dir. Of?ce Budget, Exec. Of?ce of the President, to Don?t and Agency Heads, impiententatirm oernneitmri Security Directive 1'3 Poiicyfor a Common identy?ication Standard for Federai Empioyees and Contractors {Aug 5, On August the President signed lfSl?D-lE ?Policy for a Common Identi?cation Standard for Federal Employees and Contractors? (the Directive). 124 Of?ce ofthe Inspector Gen, US. Office of Pete. Mgmt., Report to Congress Detoiaer i Marci: 3 i. 11 (May 1, BUGS) available at: Leportsisar?lpdf. ?3 us. other at Pets. Mg mt., arenas Congressionai Performance 9 (Feb. 5, 200?] available at: US. Of?ce ofl?ers. Press Release, 0PM Consoiiciates Brtiir ofFetierai Security Ciearance Process with Transfer ry??'ver Lilith} Empioyees from Defense Department: Vast Majority ofFerierai Background Intestigationr to be Centered at 0PM (Nov. 22, {?The LLS. Of?ce of Personnel Management and Department of Defense announced today the transfer of over 1,343!) personnel security investigation staff from DOD to UPM. This move will consolidate the vast majority of background investigations for the Federal government with 126 US. Office of l?ers. Congresaianai Budget Justi?cation Performance Budget 9 (Feb. 5, 200?] available at: ?7 Intelligence neterm and Terrorism Prevention Act of zoo-i, Pub. L. No. res-453, so use. see ciao Rebeca Lafl urc, How Congress Screwed Up America '3 Security Cieoronca System, FOREIGN POLICY, Oct. t, 2131} available at: 3? the terrorist attacks in September 1 l, 2001. Various federal agencies and defense contractors increased their counter-terrorism staff. '13 That staf?ng surge caused a backlog in processing background investigations. The backlog was at least 133,000 by 2004.119 The Intelligence Reform and Terrorism Prevention Act 10841-53) required that 90 percent of clearance applications had to be resolved within 60 days by 2009, a reduction of 84 percent from the then- 375 day average wait time.am Clearing the background investigation backlog was a priority, but there was also a clear need for 0PM to prioritize the information security of its data. Over the 2005-200? timcframe, the 10?s annual auditing identi?ed weaknesses in the security of the agency's information systems which would deteriorate to ?material weakness? status in 2001'? In March 2003, the HTS Report to Congress recognized a need [or the agency to focus on protecting sensitive information and over the Unfortunately, in today?s high tech world, inappropriate access to this sensitive information can lead to adverse consequences for the American public we are sworn to protect and serve. Consequently, the Office of the Inspector General has identi?ed and reported the protection of personally identi?able information as a top management challenge for the U.S. Office of Personnel Management (0PM), and we believe it is a challenge that will be ongoing because of the dynamic and ever-evolving nature of information security. Recognizing the adverse consequences of lost or stolen Pll, including substantial harm, embarrassment and inconvenience to individuals, as well as potential identity thell, Director, the Honorable Linda M. Springer, initiated a series of actions beginning last fall. She wanted to make sure that all 0PM employees clearly understood what meant, the impedance of protecting Pl], and their responsibilities in protecting it. system?; 1.1.3. Of?ce of Per-3.. Mgmt., FYEWS Congressional Budget Justi?cation Pel?rrmance Budget 9 (Feb. 5, 2013?], res See, Rebeca La?urc, How Congress Screwed Up America?s Security Clearance System, FOREIGN POLICY (Clot. l, 2013} available at: clearance-systemt. '19 Id. no Intelligence Reform and Terrorism Prevention Act of 2004, Pub. L. No. 103?458, 5U U.S.C. 334] {2012); see also Rebeca Lallure, How Congress Screwed Up America 's Seem-int Clearance System, FOREIGN PULICY, Dct. I, it] I 3, 1'1" Of?ce of Inspector Gen, LLS. Of?ce of Pers. Mgmt., Report to Congress .4!er I, 290? September 3H. at H) (Sept. available at: df. Of?ce of Inspector Gen, U.S. Office of Pers. Mgmt., Seariannual' Report to Congress October I, to March Elliott, at i (Mar. 2003) available at: reportafsarl?pdf. '3 Of?ce of inspector Gen, LLS. Uil'ice of Pets. Mgmt., Sentimental Report to Congress Detaher I, 2th?)? to March 31, Etltl'?t, at I (Mar. available at: 1n. ow'newsr're ortsr nblicalionstsemi-annual- rcportatsar??pd 1When the agency made a push in EDGE to ensure ?all 0PM employees clearly understand what PII meant, the importance of protecting PH, and their responsibilities in protecting it?, 0PM security stat't'that were 38 In the fall of 2003, however, the 10 reported that the material weakness from the prior year had not been fully addressed, and that it had ?some signi?cant concerns? with aspects of the agency?s information security program. '34 The warned that major elements of policies had not been updated in ?ve years, found signi?cant de?ciencies existing in the control structure of management of major system certi?cation and accreditation, as well as in the plan of action and milestones process, and that the agency operated without a permanent IT security of?cer for over six months.135 In the spring of 2009, 0PM underwent a leadership transition. At John Berry's Senate con?rmation hearin in March 2009, Mr. Berry was questioned extensively on the security clearance backlog,1 however, Congress did not pose any questions to him about information security 11? Berry was con?rmed in April 2009333 and in September zoos he testi?ed at length on the need to modemize the security clearance system and to eliminate the clearance backlog. His prepared testimony noted that work to improve background investigation processing would include efforts to strengthen access controls. Berry testi?ed: We are working to bring the benefits of access to the veri?cation system to new user types to support agencies in Personal Identity Veri?cation (PW) credentialing. We are working with the stakeholder community to identify potential enhancement to the veri?cation system to permit greater reciprocity. We are developing a web-based automated tool to assist agencies in identifying the appropriate level of investigation. Meanwhile in September 2009, the reported that the state of information security at 0PM was worsening. The 1G stated: In our FY 2007 and 2003 FISMA audit reports, we reported the lack of policies and procedures as a material weal-mess. While some progress was made in FY 2009, detailed guidance is still lacking. . . This yearbreach response were already working at 0PM. For example, Jeff Wagner, UPI?vl's current Director of IT Security Operations, began working at in June 2005. In transcribed interviews, Mr. Wagner also admitted that he had been on a Performance Improvement Plan in 2012 or 2013- He said, believe the PIP that I was placed on was became, in my aggressive nature towards IT security, I had offended a few ccple.? See Wagner Resume, at 00000] Production: Aug. 23, 2015}; Wagner Tr. at 141-142. 3" Of?ce of Inspector Gen, US. Office of Pets. Mgmt., Sertrtormunl Report to Congress April l, 2003 September it} 2003, at 15 (2008} available at: m. ovlnewsl orts- df. til. ?6 Nomination qt'l-lrm. M. Jolm Benjy to he Director, [lilies ofPers-ormel Management: Hem-mg Before the S. Sperm. on Homeland Sac. d?r Gov 'tA?nlrs. 11 1th Cong. (Mar. 26, 2009]. Id. ?3 us. orsee ofPers. Mgmt., Press Release, ,lolm Hertjt CorI?tmerl or 0PM Dlrector (Apr. a, zone) Clearance Reform: Moving Pot-word rm Hearing Be?rre ott of Gov 't tlte Peal Worlgforee, rll' D. C. oftlre 3. Comm. Cm Homeland See. (E Gov 't A?nit?s, ll 1th Cong. (Sept. 2009} [statement of John Berry, Director, US. Of?ce of Pets. lrl. 39 expanded the material weakness to include the agency?s overall information security governance program and included our concerns about the agency?s information security management structure. For example, in the last 18 months, there has not been a permanent Senior Agency Information Security Of?cial or a Privacy Program Manager, resulting in a serious decline in the quality of the agency?s information security and privacy programs. With the recent appointment of the new SAISO, and the planned Office of Chief Information Of?cer reorganisation which may involve increased staf?n levels, we will reevaluate this issue during the FY 2010 FISMA sstait.1 In the spring of 201 U, the continued to report ?significant concerns? regarding the overall quality of the information security program at 0PM. '42 The warned that the agency had not fully documented information security policies and procedures or established appropriate roles and responsibilities, and that while an updated Information Security and Privacy Policy was ?nalized in August it did not speci?cally address IT environment and lacked detailed procedures and implementing guidance. ?3 The 16 also questioned in 21310 whether 0PM leadership was conunitted to information security over the long-term. The stated: This year we expandcd the material weakness to include the agency?s overall information security governance program and incorporated our concerns about the agency?s information security management structure. . . . The agency appointed a new SAISO in September 2009; however, the individual left in January 2010. Another new SAISU was appointed in late April 2010. With a new Chief In formation Of?cer also recently selected, 0PM may ?nally be in a position to make long needed improvements to its IT security program. However, given this turbulent history it remains to be seen whether senior management is fully committed to strong IT security governance for the long term.?44 In 2012, 0PM Director Ben'y ordered the centralization of security duties to a team within Office ofChieflnformation Of?cer (DCIO). In March 2012, the 16 reported that ?Our audit showed that the agenc continues to struggle with improving the quality of its information security program?? The lG also found that the agency?s OCIO lacked the authority it needed to manage security matters effectively, and that the agency needed to move to a more centralized system ?because the fundamental design of the program is flawed?!? The 1G Office of inspector Gen, US. Of?ce of Pcrs. Mgmt., Report to Congress Aprt't I, Still? to September 2909, at {Sept. 2DU9), Of?ce of Iuleector Gem, US. Of?ce ochIs. Sentiments! Report to Congress October 20:79 More}: 20m, at (Mar. 2010}, 14] Id. 1-H ?5 Office of inspector Gen, LLS. Office of Pets. Mgmt., Sentimental Report to Congress- October J, 2i? t? to March ENE, at (Mar. 21]] 2), n1. ovtnews?re orts~ licalienstsemi-annual-re g?s?sa??pdf. '45 1.1.3. Uf?ce offersonnel Mgmt. IL'lf?l?ice General Sentient-mot Report to October t. to March 31'. 2W3, at 8-9 (Mar. 2013) available at: 4D pointed out that owns ?designated security of?cers? were appointed by, and report to, the program offices that own the systems, but ?very few of the [)3th have any background in information security, and most are only managing their security responsibilities as a secondary duty to their primary job function."m The It] found that IT security at 0PM was limited because ?the OCIO has no authority to enforce security requirements" and concluded: IT security is a shared responsibility between the OCIO and program of?ces. The OCIO is responsible for overall information security governance while program of?ces are responsible for the security of the systems that they own. There is a balance that must be maintained between a consolidated and a distributed approach to managing IT security, but it is our opinion that approach is too decentralized. 0PM program offices should continue to be responsible for maintaining security of the systems that they own, but the D30 responsibility for documenting, testing, and monitoring system security should be centralized within the onto. '43 In other words, there were increasing calls iior centralizing and fortifying authority and power under the OCIO by the BIG. By the end of FY2013, the centralised structure for information system security of?cers remained understaffed and hampered by budget restrictions. t4 And in 2013, as the agency prepared to transition to new leadership, the IG released two key reports. First, its newest audit found that the security of information systems remained a material weakness. is? Second, the It] also issued a warning about the information system where background investigation materials are stored. In June 2013, the audited Federal Investigative Services? Personnel investigations Processing System (PIPE). The made clear the importance of this system: Approximately 15 million records of investigations conducted by and for 0PM, the Federal Bureau of Investigations (FBI), the US. Department of State, the 11.8. Secret Service, and other customer agencies are maintained in FIPS. Furthermore, the system interfaces with several other FIS systems to process applications while its data ?ow relies on both the UPM Local Area Network.t Wide Area Network (LANIWAN) and Enterprise Server Infrastructure (531) general support systems. '51 amine HIS "w Office of inspector Gen, LLS. Of?ce of Pete. Fedora! Security Management dot/1mm FF 25.13, at 5 (Nov. 2 I 2-313), man 3 4321 .pdf. 15? Office of Inspector Gen, LLS. Of?ce of Pers. Mgmt., Report to Congress October I. to Mot-cit 3i, HIM, at [Man Edit-i}, m. ovt'newsfre orts- ublicati nstsemi?annual-re Office of Inspector Gen, US. Office of Pers. Mgmt., Semiommot Report to Congress Apt-ti t. Etit?j to September 30. 2W3, at 1" (Sept. 2013) available at: 41 In the case of PIPS, we Found that there were a number of controls inappropriately labeled in the system security plan as or inherited. As a result, these controls were never tested, increasing the risk that these controls may not be functioning as intended, and therefore posing a potential security threat to the system. This omission is particularly concerning given the purpose of the system and the nature of the data the system contains. '52 The [G?s warning about the weakness in and the need to protect the background investigations data was prescient. The 10?s warnings were in effect when, in 2013, the agency welcomed new senior leadership. On May 23, 2W3, Katherine Arehuleta was nominated to serve as Director The LLS. Senate con?rmed Archuleta on October 30, 2013f? and she was sworn into of?ce on November 4, 2(113.155 Archuleta was a former teacher, public administrator, community leader from Colorado and the National Political Director For President Dbama?s reelection campaignd?' Shortly thereafter, in December 2013, Donna Seymour began her tenure as C10. 57 During her Senate confirmation hearing on July 16, 2013, Archuleta made a commitment to work with her senior management team to create a plan for modernizing IT within 100 days of assuming of?ce, and to identify new IT leadership using existing agency expertise and with advice from government experts. '53 As Archuleta and Seymour began their tenure, modernization was a key part of the Director?s early agenda. Director Archuleta announced a new Strategic Information Tecnnot'ogy In: L53 White House, Press Release, President Announces His intent to Katherine Arcino?eto as Director of the O?'ice ofPersoni-tei Management [May 23, 2013), officeiil] "4 Lisa Rein, ?Senate Con?rms as the New Fetter-oi Personae! Chief," WASH. POST, lDiet. EU, 21113 available at: ton ost.comi oliticst? federal-p ersonnel-ehief'i 3! 1 a??l le3?a624-41d??l bt'th b'i'R storv.html . US. Of?ce of Pers. Press Release, U18. O?ice ofPet?s. Katherine Sworn-in as i tit}: Director oftite Q?'ice ofPers-onnei Management: Greets Empinyees or the Nest! Director and Gets to Wot-it (Nov. 4, 2013) available at: Uth-director-of? ?5 Cecilia Munoz, Weicotning KatherineArcitoteto, the First Latino Director oftite ??ice offers-outta! Management, THt-z (Nev. 4, 2013, 4:39 pm.) available at: se.govfblogt'l? 1 3! nel- management. missus Miller, crossing Continues one can, Fen. NEws IUUJIU (Dec. as, sets), ht lpziti'ederalnewsradio. convtechnology??l 3! Eicio-shuf?c-contin uesel?sba?d hsrop mi. ?3 Of?ce of Pars. Strategic Information I'ecimoiogy Finn (Feb. 2014} available at: cet'strategi c?pla nsistrategic- it-pian.pd F. 42 Pian in as working days (12? calendar days after being sworn in on November 4, The Plan listed ?information Security? as one of sin IT ?Enabling lnitiatives??that is, initiatives to "provide the strong foundation necessary for successful operation, development, and management of IT that increases accountability, ef?ciency, and The sixty-nine page report includes a brief discussion of the background investigation systems, '51 but the overall discussion related to background investigations focused largely on process reform and automation.?52 The Plan also included two-and-a-half pages on information security, wherein 0PM stated it will: 0 follow guidance from the Federai information Security Management Act, NIST SUD-53 (?Security and Privacy Controls for Federal Information Systems and Organizations"); ?33 I follow guidance from OMB to ensure protection of these systems that contain PII and [protected health information]; i work with DHS to implement continuous diagnostic monitoring and use information security continuous monitoring (ISCM) tools; I implement a three-phase plan to carry out its ISCM strategy; and Ir attempt to secure additional resources to hireitrain IT staff. Seymour later recounted early efforts to assemble the Strategic tight-motion Technology Piotr with Archuteta. In June 2014, Seymour testi?ed to the Senate Committee on Homeland Security and Governmental Affairs: its Chief Information Of?cer [or the Office of Personnel Management (0PM), 1 am responsible for the IT and innovative solutions that support mission to recruit, retain, and honor a '59 Joe Davidson, 0PM Unvaiis iT Pian to improve Feder?ai Retirement Operations, Recruitment, WASH. Post, Mar. to, 2014 available at: imo rove?fetleral-retirement-on motions-recruit mentiEUl 4i?3i liaeerbSZ-a?ii Ef?l LLB. Office of Pets. Mgmt., Strategic Itnrot?ntotion Tochnoiogv Finn, at vii (Feb. Elite). at 32. The Plan?s reference to background investigations included one line on security: ?'i?he initiative will also support reform in the investigative process and, drawing on the enabling initiative of information security, protect and secure the volume of sensitive information in the EPIC systems [the automated suite ofbachground investigation systems]. U.S. Of?ce of Pers. Mgmt., Strategic Ity?omtation Tecnnoiogv Fiat: 32 (Feb. 2014}. ILS. Dcp?t of Commerce, HIST Spec. Publ?n Still-53 Rev. 4, Security and Privacy Controisfor eater-oi {information Systems and Organisations (Apr. 2GB) available at: littp:iinvl pubs. nist . govt nistpubsi IST. SP3 00-53r4.pdf. no 1.1.3. Of?ce ofFers- l'vlganL, Strategic information Technoiogv Pia-n at lit?19 (Feb. 2014}. Note: While OPM worked to craft the new Plan, key corresponding updates to key internal security guidance and protocols and Authority to Operation For example, ?Incident and Response and Reporting Guide? was not updated guide issued in 21309. The Guide contains protocols for responding to breaches, among other things. See US. Of?ce ofPers. Mgmt., incident Response and Reporting Guide 3 [July See oiso Special Agent Tr. at 8. The OPM OIG special agent testi?ed on October 6, 2015 that the incident Response and Reporting Guide issued in Bill)? was still the guidance in cfibct at OPM, as of October 2015. 43 world class workforce. Director Katherine Archuleta tasked me with conducting a thorough assessment of the state of IT at 0PM including how existing systems are managed and how new projects are developed. This process has led us to identify numerous opportunities for improvement in the way we manage IT. Ful?lling the Director?s promise, 0PM released a Strategic lT Plan in March 2014. We developed the Strategic IT Plan to ensure our IT supports and aligns to our agency?s Strategic Plan and that mission is ful?lled. It provides a framework for the use of data throughout the human resources lifecycle and establishes enabling success?il practices and initiatives that de?ne lT modernization efforts. The plan also creates a flexible and sustainable Chief lnfennatien Of?cer organization led by a strong senior executive with Federal experience in infonnatien technology, program management, and HR policy. 0PM also understands that new lT implementation will be done in a way that leverages cybersecurity best practices and protects the personally identi?able information 0PM is responsible for. '55 MS . SEY MOUR 1 Donna Sana-near testi?es to the Coamn'nec on Oversight and Government Reform When Seymour testified before Congress in June 2014, however, she did not mention that the agency learned in March 2014 of a signi?cant data breach at the agency; nor did A More Efficient mind E?entive Government: Federal IT Initiatives and the IT Wer?force: Hearing Before the thcomm. on h?ictenev Lit hfectiveness ofFed. :31 Fed. Wot-Horne ofrhe .3. Conan. on Homeland Sec. 1 13th Cong. (June It}, 2014) [statement of Donna Seymour, Chieflnfo. Of?cer", US. Of?ce of Pets. 44 she mention that the agency, under her and Archuleta?s watch, had spent the previous two months monitoring attackers and remediating a signi?cant incident. 1'55 On July 9, 2014, The New York Times broke the news, previously unknown to the public, that 0PM suffered a breach?? The Times drew attention to the severe implications of the breach for anyone who had ever applied for a security clearance. The story stated: The intrusion at the Oltice of Personnel Management was particularly disturbing because it oversees a system called in which federal employees applying for security clearances enter their most personal information, including ?nancial data. Federal employees who have had security clearances for some time are often required to update their personal infotrnation through the website. The agencies and the contractors use the information from to investigate the employees and ultimately determine whether they should be granted security clearances, or have them updated. '53 While Tire Times immediately grasped the potential implications for the country, CID was trumpeting the merits of the agency?s IT Modernization plan. in fact, downplayed the damage from the breach to the The Times: The story stated: But in this case there was no announcement about the attack. ?The administration has never advocated that all intrusions be made public,? said Caitlin Hayden, a spokeswoman for the Obama administration- ?We have advocated that businesses that have suffered an intrusion notify customers if the intruder had access to consumers? personal information. We have also advocated that companies and agencies voluntarily share information about intrusions.? Ms. Hayden noted that the agency had intrusion?detection systems in place and notified other federal agencies, state and local governments about the attack, then shared relevant threat information with some in the security industry. Four months after the attack, Ms. Hayden said the Obarna administration had no reason to believe personally identi?able information for employees was compromised. ?None of this differs from our normal response to similar threats,? Ms. Hayden said. 1'56 June Edi-4 DPM Incident Report; see oi?so, A More E?icietit and E?'eetive Government: Examining Fetter-oi IT initiatives and tire TT Workforce: Hearing Before tire Subcomm. on E?ieietiey dihfectiveness ofFeti. Programs :5 Tied. Workforce oftiie 3. Comm. or.- i-iometood See. Gov ?t A?hirs, 113th Cong. [June it}, Edi-4) {statement of Donna Seymour, Chief Info. Of?cer, LLS. Of?ce of Pets. MngJ. Michael B. Schmidt, David E. Sanger tit Nicole Perlroth, Chinese Hookers i?tostre Kev Data on US. Workers, MY. July 9, 2014, available at: Ft]. Isa id. at. 45 Archuleta and Seymour later testi?ed in 2015 that no PII was exfiltrated during the 2014 data breach. Documents and testimony show gaps in audit logging practices led DHS to conclude the country will never know with complete certainty all of the documents the attackers ex?ltrated during the breach discovered in March 2014.?1 It is clear, however, sensitive data was ea?ltrated by the hackers.W2 As discussed in the following chapter, 0PM watched the attackers steal documents related to 0PM IT systems, including PlPs, contractor information, and documents containing names and the last four digits of associated Social Security numbers.?3 Archuleta and Seymour did make some progress in addressing security governance issues by continuing to centralize IT security responsibility. They committed to make IT a priority with the release of their IT Modemization plan in early 2014, and arguably had more ownership of its IT security at this point than ever before. However, they failed to prioritize data security and implementation of basic cyber hygiene measures at a time when it became critically important to meet the increasing cyber threat. . Katherine res-?ies to the Cornmi?ee on Oversight and Government Reform 0PM Dam Branch: Seymour, Chief Info. foicer, U3. Uf?cc of Pcrs. Mgmt.)_ During this hearing, then-Director Katherine Archuleta, and then-CID of 0PM, Donna Seymour, testi?ed nine times in a single exchange with Chairman Jason that no personally identi?able information was stolen, June 2014 0PM Incident Report at ?1 The sensitivity ofthese documents is evidenced in part by the fact that 0PM refused to produce these documents to the Committee in unredacted form until February 16, 2016. The Committee initially requested this information on August 13, 2015. ?3 June 2014 0PM Incident Report at I-Ioonc31 a -001245-1246. 46 0PM Failed to Prioritize the Security of Key Data and Systems OPM's failure to prioritize high-value targets like the background investigations data compounded the problems caused by inadequately investing in cybersecurit'y in the ?rst place. Neither the data held by OPM, nor the access to 0PM systems, were adequately protected. Indeed, REM did not even have a complete IT inventory of servers, databases, and network devices. Further, on the system level 0PM had not implemented multi factor authentication, making weak access controls a vulnerability that attackers were able to exploit. ?5 OPM's failure to prioritize multifactor authentication implementation was a key observation that US-CERT made in their analysis of the data breach discovered in 2014. ?5 0PM was pressed about these and other issues during congressional hearings. For example, the background investigations data was not is the foundation of data-level security. During a June 16, 2015 hearing before the Committee, Chairman Jason Chaffetz asked Director Archuleta why 0PM did not use an industry best practice, and Director Archuleta said, ?It is not feasible to implement on networks that are too Similarly, C10 Seymour told Ranking Member Elijah Cummings that the agency was working to use She testi?ed: 0PM has procured the tools, both for of its databases, and we are in the process of applying these tools within our environment. But there are some of our legacy systems that may not be capable of accepting these types of in the environment that they exist in today. ?9 In addition, key systems were also operating in FY 2014 without a valid Security Assessment and idtuthorieation.Ia?l Also called authorizations to operatetauthorities to operate provide a comprehensive assessment of the IT system?s security controls. The 0PM IG ?t oniee of Inspector General, us. since ofPers. Mgml., Report No. 4a-c1-co-is-tn LFca'erai Inf-tanttatiatt Security Andi: Fi? Edi-t at i Olav. 2015] available at: 5- 01 ?5 Information Technology Spending and Data Security at the O?icc afFet-mnnet Management: Hearing Before the Sitbcatnnt. (in inanciat Sara. '3 and Gen. Gov. oftire Sen. Comm. on Appropriations. 1 Mill Cong. (June 23, [testimony of Richard Spires, fon?ner CID of the Internal Revenue Serv.). ?6 See infra Chapter 2. 1? Teciinatagy Eileen-ding and Data Security at tt'ie U?ice ofPersonnei Management: Hearing Before tire Sititcomm. 0t! Sent. '5 and Gen. Gov. cftiic Sen. Comm. an Appropriations, 114th Cong. {Julie 23, 2015} (testimony of Richard Spires, former ?310 of the Internal Revenue Scer. ?3 Data tilt-cacti, Hearing Before the H. Conan. on Oversight ri- Gov't Reform, ?41s Cong. {June In, (statement ofKalherine Archuleta, Din, LLS. Office of Pets. Mgmt.]. can Data Breach. Hearing Before the n. Comm. on Oversight a Gov't Reform, 1141h Cong. {June is, 2015) {statement of Katherine t?trchulcta, Din, LLS. Of?ce of Pets. Mgml.). Of?ce of the inspector General, IDfiice ochrs. Fedcrai ItJortnatian Security Management Act rlndit l" 20H (Nov. 12, 2W4) available at: information-security?m it?f Jill 4-4a-ci-?il considers the authorization process to be a ?critical step toward preventing security breaches and data Of the 21 0PM systems due for reauthorization in FY 2014, were not completed on time and were operating without a valid itsuthorization,?32 and several were among the most critical, containing the agency?s most sensitive information.133 This led the IG to warn 0PM that ?The drastic increase in the number of systems operating without a valid Authorization is alarming, and represents a systemic issue of inadcq?uate planning by OPM program of?ces to authorize the information systems that they own.?l 4 ISMA requires agencies to assess the effectiveness of their information security controls, the frequency of which is based on risk but no less than annually. '35 Appendix of OMB Circular A?lJt], in place at the time, requires that agencies assess and authorize (formerly referred to as certify and accredit) their systems before placing them into operation and whenever there is a major change to the system, but no less than every three.- years rirararr?er. '35 In November 2014, the 10?s FISMA audit stated: ?We therefore also recommend that 0PM consider shutting down systems that do not have a current and valid Authorizationmm 0PM CID Donna Seymour responded, however, that ?The IT Program Managers will work with [830s to ensure that 0PM systems maintain current ATOs and that there are no interruptions to OPt?vt's mission and operations??33 Of the eleven major 0PM information systems that were operating without a valid Authorization in three ofthese systems should have been an irru'rrcdiate priority for Director Archuleta and C10 Seymour to ensure were addressed: Personnel Investigations Processing System (PIPS), Enterprise Server Infrastructure (E31), and the Local Area Networltf Wide Area Network (LANFWAN). The security ofthese systems is critical because the ?ow of background investigation data through relies on both the 0PM LANHWAN and Enterprise Server Infrastructure general support systems. LANMAN serves as the hardware and software infrastructure at at 11. .ft'f. at 9. '33 E-mail from ethos of Pots. Mgmt. Inspector csu. Staffto House oversight a. Gov?t asrotm Staff (Dec. 4, zeta} (on file with the Committee}. Office of Personnel Of?ce of the Inspector General, Federal Iryfiminti'on Security Management Act Audit FY2314 at 9 (Nov. 12, available at: infonnation?see 44a-ci-DIJ- ?.pdf. roast-s1 Information Security Management not otzcez. Pub. L. No. Wit-34?, 44 use. a 3541 [sure]. Of?ce of Mgmt. 8.: Budget, Exec. Of?ce of the President, OMB Circular J's?13ft, Management of Federal Information Resources (Nov. 23, 20%) available at: at3? a1 see also US. Dep?t of Homeland See, Security Authoriration Process Guide 1 (Mar. 16, Etll?} available at: lesfuublicationsfs ccurity%2?Authori2ation%2 DProecss%2thuide v] 1 Led f. Of?ce ortrto tusosstor out, us. cross or rots. Mgmt., Report No. mercenaries, Federal iitromuuisu Security Management Act Audit FY 2014 at 2, [4 (Nov. 12, 2914} available at: l4-4a-ci-??- l4-t'i ?.pdf. Inf. '39 to. at s. 48 environment, supporting systems housed at Washington, Macon, Georgia; and Boyers, Pit facilities. also supports the cars tarps imaging and FTS [Fingerprint Transactional System}. ESI is the general mainframe environment that supports PIPS. OPl?vl?s mainframe is considered a separate infrastructure or ?general support system" from PIPS, and 1331 were all operating on expired Authorities to Operate. The need to prioritize the security of these systems was well-known after the IG warned in June 2013 that PIPS had vulnerabilities, and that the system interfaces with several other FIS systems to process applications while its data ?ow relies on both the OPM Local Area Neonorki Wide Area Network and Enterprise Server Infrastructure general support However, the ATO for PIPS was not reauthoriacd in 2014, and the 16?s FY2015 FISMA showed that management of system Authorizations has deteriorated even Experts from outside 0PM also criticized choices regarding IT security following the breach. On June 23, 20 5, Richard Spires, the former CIO of the Internal Revenue Service and at testi?ed before a Senate Committee on Appropriations? Subcommittee on Financial Services and General Government that 0PM should have set better priorities and focused on securing the data itself rather than the systems as an initial priority- Spires stated: [I]f I had walked in there as the ClO?and, you know, again, I?m speculating a bit, but?and I saw the kinds of lack of protections on very sensitive data, the first thing we would have been working on is how do we protect that data? Not even talking about necessarily the '90 OPIS was also operating with an invalid authorization to operate. See Of?ce ofInspector Gen, LLB. Of?ce of Pet's. Mgmt., Report No. 4A-IS-0fls?tiutl24, ity?ortnation Itseitnaiagy Sentry-tar Cantrois of tire {wire ofi?ersonnei Management 's Personnei Investigations Processing imaging Svstetn {July 1 l, see aiso mail from U.S. Office of Pers. Mgr-at. inspector Gen. Staff to House Oversight tie Gov?t Reform Sta?it?ec. 4, an I 5] (on ?le with the Committee). Of?ce of Inspector Gcn., US. Office of Pers. Mgmt., Report No. 4A?lS?U?vl 343122, Ana'it oftite information ieennaiogy Security Controls of tire US. D?iee ofPersonnei t'u-ianagetnent 's Persotrnei investigations Processing System Willi 3 (June 24, 2013] available at: Of?ce of Inspector Gem, U5. Dt??cc nt'Pers. Mgmt., Report No. ti, Faciat'tti Security Management Act Audit 1" 29M (Nov. 5, 21312) avai Iablc at: .oprngoviou 2012.pdf; Office Gem, 1.1.5. Of?ce of Pet's. Mgml., Report No. Andi! oftire information Teehnoiogy Security Contrais oftt're DES. O?ice ofPetsannei Monagenrentlr Lanai Area Nenvorir t? Wide Area Network Generai Support .Evstem FY Edi? (May 16, EDIE) available at: manageme i lpdf. '91 lIL'EIf?cc of the Inspector General, LLS. Of?ce of Pets. Mgmt., Semionnooi Report to Congress Aprii i, to September 30, 2013, at 'i (Sept. 20E 3] available at: re ortsisar49. df. ?1 Of?ce of Inspector General, U.S. of Pets. Mgmt., Report No. l, i?erierai information Security Management AetAotiit FY Edi-t (Nov. 10, 2615} available at: ge neralireportsiZ? Sifederal?in fonnation1secu ritv-mo dcrnization?ae t?au dit-fv-ZU 15 ?fma ?andi t-report-4a-ci-DD ?l 01 l.pdf. 49 systems. How is it we get better protections and then control access to that data bet?terl'm4 Spires also stated that management issues posed a greater obstacle than resource problems in solving IT security problems. Spires testi?ed: A focused effort on protecting the sensitive data with the right and the right access-control capabilities, if you put the focus there, I think most federal agencies would have the funds, have the resources to be able to accomplish that. Because of the sparse nature of the way IT has been mo in a lot of agencies there are so many, let?s say, inefficiencies that have crept into this system that I don?t believe we effectively spend the IT dollars that we receive. So I believe that with the proper drive towards management you can actually derive a lot of savings from existing budgets.'95 0PM has long been plagued by management?s failure to prioritize information security in practice, and to retain leaders that are committed to information security over the long haul. Years of neglect, compounded by an abject failure of key leaders to make the right decisions at 0PM in 2014, led to the worst data breach the federal government has ever experienced. no Technologv Spending and Data SecurityJ at the ??lee ofPersnn-ne.? Management: Hearing Before the Snbconmt. on Financial" Sew. 's and Genera! Gov. ofrhe Conan. on Appropriations, 114th Cong. [June 23, 2015} [Eestimony of Richard Spires, Former Chief Info. Officer, Internal Revenue Serv.}. I "l M. 50 Chapter 2: The First Alarm Bell Attackers Discovered in 2014 Target Background Information Data and Exfiltrate System-Related Data In the March 2014, USHCERT alerted 0PM to an intrusion that laid the groundwork for the breach of OPM systems holding background investigation data, the ?crown jewels? of current and fortner federal employees, contractors, and national security personnel. '95 0PM considered their reaponse to the data breach, which they learned about from in 2014, a success. C10 Donna Seymour touted the response strategy: ?one of the things we were able to do immediately at 0PM [in 2014] was recognize the problem. We were able to react to it by partnering with DHS . . . to put mitigations in place to better protect information."m However, the data breach of background investigation data and personnel records ?rst announced in June and July of 2015193 raises serious questions about response to the data breach discovered in EDI 4. Documents and testimony obtained by the Committee show successes and failures, but some of the tnost important questions were unanswerable. For example, while 0PM testified that no personally identi?able in formation was ea?ltrated during the 2014 data breach,1w gaps in audit logging practices led DHS to conclude that the country will never know with complete certainty the universe of documents the attackers enlillraled.2m Documents and testimony show the materials esfiltrated from 0PM likely would have given an adversary an advantage in hacking systemsml This evidence calls Donna Seymour?s testimony into question. She told the Committee ?the adversaries in today?s environment are typically [able] to use more modern technologies, and so in this case, potentially our antiquated technologies may have helped a little bit.?202 [a putting forward a ?security through obscurity" defense, the C10 downplayed the reality that 0PM was facing a determined and sophisticated actor while only having minimal visibility into their environment. June E?lti 0PM Incident Report; see also David Perera 5: Joseph Marks, sewn Disclosed Huck (in! ?Crown Jewels, June IE, 2015, available at: security-background-checits-l 18954. no Enhancing of ?tint-Forty Contractors and Vendors: ?ooring Before the H. Comm. on Oversight ct: Ger Reform, 114th Cong. (Apr. 22, 2015] [Question by Mr. Cummings]. U.S. Of?ce of Pers. Press Release, 0PM to Nott?' Employees ofCvaorsooto?t'ty Incident [June 4, it?ll 5] available at: U.S. Of?ce of Pers. Mgmt., Press Release, 0PM Announces Steps to Protoct en?sral Workers and Gaiters Front Cyber- threats, (July 9, 21315] available at: Hearing on 0PM Doro Branch: Port {statement of Donna Seymour, Chief Info. Of?cer, US. Of?ce of Pers. During this hearing, then-Director of 0PM, Katherine Arehulcta, and then-CID of OPM, Donna Seymour, testi?ed nine times in a single exchange with Chairman Jason Chaffetz that no personally identi?able information was stolen. In? June 2014 0PM Incident Report or HUGHES 3-0131233 124d. :tn Saulsbury Tr. at It'll?28. 3m Enhancing ofotiinl?Pnt'tv Contractors and Vendors: Hearing Hefm?c the H. Comm. on Oversight t?i (For 't Reform, 114th Cong. [2015) (Question by Mr. Cummings}. 51 In the aftermath of their 2014 response, available threat intelligence about the relevant actor groups targeting federal employee information and the types of malwarc discovered in 2014 also raised the stakes for 0PM. In the fall of 2014, Novetta and a number of supporting industry organizations produced a detailed report containing information pertinent to Chinese APT activity with an emphasis on Hikit malware. This malware was found during the 2014 incident response. The Novetta paper speci?cally looked at the Axiom Threat Actor Group, which according to public reports, was responsible for the OPM data breach discovered in 2014.1? The analysis wamed that among the industries being targeted or infected by Hikit were Western government agencies with responsibility for personnel management. The report also warned that ?[w]ithin these targets, Axiom has been observed as going out of its way to ensure continued access regardless of changes to its target?s network topology or security controls?m 0PM leadership downplayed the signi?cance of the 2014 breach. Instead, 0PM should have raised the alarm and recognized this initial attack as a serious and potentially devastating precursor given how close the early attackers got to the background investigation systems and the related data taken during this breach. The following discussion describes 2014 discovery and incident response efforts, and how Hikjt malware was found and sensitive data related to the background investigation function was taken from OPM's systems. Further, this discussion highlights key observations that were made about the weaknesses and vulnerabilities ofOPM?s IT security during this incident response period. On March 20, 2014, 0PM's Computer Incident Response Team received noti?cation from that data had been eitfiltratecl from network.? Beginning March 2014 and through May 2014, 0PM (in consultation with investigated the incident, monitored the attacker, developed and implemented a mitigation plan, and removed this initial attacker from system. noti?ed 0PM that a third party had reported data being es ?ltrated from system to a known command and control server Jeffrey Wagner, Director of IT Security, testi?ed about 0PM activities upon notice them [T]he initial response [to the 2014 data breach] is a call from DHS. All right. So on 312(5) DHS called us and let know, hey, we think this is bad. We began pulling logs, and records, and things of that nature, and on 3f25 is when we veri?ed that it was a malicious activity.2m i: Novetta Operation SMN: Axiom Threat Actor Group Report. Jr}. 3-9. June 2o14 Incident Report at Hooaos ts-ootzss- M. 0PM contractor Brendan Saulsbury stated that ?[the 21114 incident] was ?rst detected by LIE-CERT via the Einstein appliances that they have on network. And that was communicated to 0PM via email." Saujsbury Tr. at 13. The 0PM Incident Report states that a ?third party? reported the data cx?ltration to June 2cm Incident Report at IIUGRUBI 8 41101233. It is possible that both accounts are correct and that the ?third party" referenced in the 2111114 Incident Report is an Internet Service Provider who reported network activity collected by an Einstein sensor. an Wagner Tr. at 13. 52 Wagner also described process for analyzing and elevating information security reporting or alerts to a cybersecurity incident. He stated: Once we get forensic evidence that there?s actual adversary activity within the environment, it escalates the level of response. So, for instance, on a regular basis we get alerts or reports of an email trying to be sent to us that has a maliciorrs link. it creates an alert. We?ll do initial forensics on that alert, and we?ll see that our cerrent tools will stop that malicious link from being able to connect or downloading anything. And it de-escalates the situation. So from an incident response perspective, everything rises to a critical level, and then once we have forensics evidence and identify speci?cally what is going on, and it then escalates into the speci?c response As incidem reaponse activities began, documents show that as of March 20, 2014, the following facts were among those known to 0PM: I PIS Investigator accounts had been compromised. . The malicious C2 server was communicating with an 0PM server. I The malicious C2 servers? conununications with 0PM were During the incident response period, 0PM learned the C2 server was connecting with an 0PM network monitoring server between the hours of 10 pm. and 10 am; then the attackers were using this server and a compromised Windows domain administrator credential to search for PIPs-relatcd files on network.2m An initial examination of the network trallic between the_ server and the C2 server found that the communications were utilizing a four byte XOR key, indicating a speci?c intent to disguise themselves amongst network traffic?? Brendan Sautsbury, an 0PM contractor working in the OPle IT Security Operation group, testi?ed that 0PM used the security tool to identify what devices on OPlvl?s network were actively communicating, or ?beaconing?? to the C2 server.?2 Using the network traf?c information gathered by NetWitness, Saulsbury was able to design a custom script to ?reverse engineer the obfuscation algorithm the attackers were using to mask their traf?c so it would not be detected by sensors, like [DPle's] security tools.?213 Saulsbury?s team could then 3? Id. June 2014 one Incident Report at uocaesrs rs. at Bouncers-001233exclusive?or is a form of private key that relies upon a simple binary formula to develop its obfuscation of the underlying data. 2'3 Saulsbury Tr. at 3'9. 3? Saulsbury Tr. at 4B. 53 observe the infected machines communicating with the C2 server, and also the commands that - 1.1. 1'12 were bemg sent down irorn the actual attacker s1tt1ng at the keyboard. '4 Thus, 0PM and their interagency team were able to identify the adversary?s initial foothold in network?where the attackers had established a persistent presence in the environment. Once it was determined which devices on network were beaconing to the hackers? C2 server, 0PM was in a position to begin a Full forensic investigation and look for malware on the compromised machines.?5 On or about March 25, in the words of OPM Director of Security Operations Jeff Wagner, a ?critical level?216 was reached and 0PM was able to make a ?full determination on the who and what?217 of the data breach, to know where the hackers are ?going, what they ate seeing, and most importantly what [the hackers] are interested 1n."213 As a result, 0PM determined the incident was malicious on March 25, 2014, moved DHS onsite to assist the response, and 1began a full monitoring phase to gather information to answer the question of ?how.?1 During the three-month incident response period, 0PM undertook a number of other incident response activities. For example, according to 2014 Report timeline, on March 26, 2014 0PM searched fo1 embedded malware on end points at its Washington, D. C. headquarters at its Boyers, data center, and at a back-up data center in Macon, Georgia.22 l113m March 2014, 0PM took steps to remediate the 0PM Personnel Investigations Processing System Imaging System system that provides an electronic representation of ease paper ?les to expedite the 1uproeessing of background investigations and performed this remediation work in late i'viarel't.2 011 March 28, 2014, in recognition of the fact that 0PM did not have the ability to monitor traf?c in and out of HTS the system that held background investigation data? 0PM installed a fiber tap to begin to monitor such traf?c- Finally, during this period 0PM watched the attackers take sensitive data relating to high-valued targets on 3 systems such as the PIPS system. 222 0PM was never able to determine how the adversary initially entered their systems. Then from late March through April 21114 the incident response2 team continued to identify additional infected workstations and malware on key systems.22 3Specifically, 0PM found Hikit malware on several 0PM systems}? Hikit' 1s a variant of rootkit malware (which ts ?an extremely stealthy form of malware designed to hide its malicious processes and programs from the detection of commodity intrusion detection and antimvirus As 1? Saulshury Tr. at 46. 3? Saulsbury Tr. at 351-411. '6 Wagner Tr. at 13. f: iune 2014 Incident Report at scandal 3 ecu-411. :11 13? June 2:114 0PM Incident Report at Hoeacsts 41111241. see also Office of Pers. Mgmt., 0PM Petite:nael Investigations Processing Imaging System Prime}- Assessment available at; olic 1" i s-ima in stem. df. "3111111: 21114 DPM Incident Report at nocaostsemssat. "31111111 21114 incident Report at nooaosls 111111241 1242. June 2014 0PM Incident Report at at Appendix C. June 2614 GEM Incident Report at 8 I 234. 54 explained in the June 2014 0PM Incident Report, ?HiKit allows the attacker to run commands and perform functions from a remote location as if they had the equivalent of a monitor and keyboard connected to the compromised 0PM systemi?? Time is crucial in an incident response scenario. According to NIST, ?organizations should strive to detect and validate malware incidents rapidly because infections can spread through an organization within a matter of The agency?s slow response made matters worse. According to NIST, ?minimizing the number of infected systems, which will lessen the magnitude of the recovery effort.?128 Once the incident was identi?ed and 0PM, along with their interagency partners, entered into an advanced monitoring phase necessary intelligence was gathered on the adversaries? tactics, techniques, and procedures, the kind of threat infonnation necessary to harden information security not only at 0PM but at other agencies. From March 25, 2014 to May 27, 2014, 0PM, upon the advice of US-CERT, engaged in a prolonged intelligence gathering phase. The goal of this advanced monitoring phase was to ?carefully observe all ofthe malicious actors? activities in order to gain an understanding oftheir tactics, techniques, and procedures as well as to identify all of their other unknown or inactive infected systems within network.?329 The advanced moniton'ng of the adversary ended in a ?Big Bang? on May 27, 2014?-an effort that commenced once the hackers got ?too close? to the background investigation material accessible from the PIPE system.130 Saulsbury described the comprehensive monitoring strategy during a transcribed interview with Committee investigators. He testified: advice was to basically do an ongoing investigation and ?gure out, do our best to find the entire attacker foothold in the network and then remediate them all at once to prevent the attacker from realizing that you are aware of them, and then changing their tactics and techniques to ?irther avoid detection?? Wagner also described the scope of the monitoring phase. He testi?ed that 0PM was not just looking for TTPS, but other indicators. Wagner stated: 31* June 2014 over Incident Report at nominate-amass in Peter Mell, Karen Kent Joseph Nusbaum, Nat'l Inst. of Standards 'l'ech., Spec. Publication sea?33. Gm?de to Monitor's Incident Prevention and Handling 3 (Nov. 21305} available at: govir 33-? June sore oeru Incident Report at Hooaems caress. 13? Ssulsbury Tr- at 26. 23] Saulsbury at 25-26. 55 You?re trying to ?nd speci?c actions they?re doing to give you an indication ofwhat they?re doing and what they want. You?re also looking for as a former pen tester, usually what you try to do to try to prevent people from catching youother back doors or means in which you can create a persistent attack. It?s just making sure you always have a secondary way in?? In June 20 4 0PM Incident Report, there is almost a daily catalogue of monitoring efforts. As part of the monitoring effort, 0PM established a series of alerts and system rules to watch the adversary, employing a full packet capture glogging data] tool to gather network traf?c between the infected machines and the C2 server.? An interageney team, including FBI, and was involved in the incident response effort. The team received automatic noti?cations during the monitoring phasem During this 2014 incident response period, 0PM used its existing set of security tools and infrastructure to conduct their monitoring effortm? in addition to monitoring, 0PM was prepared to implement preventative measures. For example, Wagner testi?ed that they were instructed to shutoff internet access if any PII was leaving the networkm By March 2014, reported that 0PM had ?heightened proactive readiness" and was developing plans for ?full shutdown.?233 By April It, 2014, tactical mitigation strategy and security remediation plans were being developed to eliminate the adversary"s foothold on network?? The process of setting up alerts and tipping points, identifying infected workstations, and elevating monitoring technology continued until the ?Big Bang? on May 27, 2014. While the timeline is helpful to understand the 2014 incident response activities. some entries illustrate gaps in visibility their systems and applications, including the highly sensitive system which housed the sensitive background investigation data. For example, the March 23, 2014 timeline entry states (PM ?did not have [the] ability to monitor traf?c intent of PIPS Installed PIPS ?ber taps?? Wagner responded to this entry by testifying: So in that speci?c instance -- a mainframe functions significantly different 13! Wagner Tr. at lit ?3 June sore Incident Report at Hooaes rs eorsrto. Saulsbury Tr. at 43 brought the NBA Blue Team :35 Wagner Tr. at 59 (?So ifthe adversary's activity was front Itwas normally in a period of3 to at am. where they were active, when they would throw something on our network or send a script to the network. 1 would get a phone call. I would then call DHS and FBI. So it was a concerted effort. lt wasn?t simply 0PM by June 2014 0PM Incident Report at rroeaosrs corsair. Wagner Tr. at 10 (The question posed to Mr. Wagner was whether or not the security staff at 0PM had the authority to make operational decisions; his answer stated that ?i guess a good example would be during the NM or 2?15 breaches, the security operations group was under a standing order from the director that if we indicated that information was leaving, we could shut down the interact at any time?). 1? June sore: Incident Report at parser. in M. M. 56 ?'om a standard distributing environment, say Linux, or Windows, or like you have at your home. A mainframe is a giant cloud computer, which runs on a proprietary type operating system, and it communicates in a far different method than a standard distributing environment. So at the time we did not have equipment installed to try to navigate between distributed and mainframe. We had a project to implement these pieces, and what we did is we sped up the project to get the ?ber taps installed to be able to set up a communication method to where we could see the traf?c as it traversed between the distributing environment and the mainframe environment?? Saulsbury also described limited ability to monitor Internet traf?c during and prior to the 2014 incident. He testi?ed: 0PM had the ability to monitor traffic going out to the Internet at all times or at least going back prior to the 2014 incident. The reason for putting a network tap on the PIPES segment is to be able to monitor what is called, what we refer to as east-west traf?c, so internal-to-internal traf?c, from the general network going in and out It was not until March 31, 2014 that 0PM was able to ?turn on? the monitoring capabilities for all PIPS and Federal Investigative Services (PIS) related systems.243 In other words, it took almost eleven days from the time 0PM was noti?ed on March 20, 2014 about the data breach for 0PM to deploy the capabilities necessary to monitor one of the most high value targets on their IT environment PIPs. The timeline also highlights other gaps in information sccurity posture that made 0PM vulnerable to attack and put sensitive data 0PM held at risk. For example, a March 31, 2014 entry states: ?high value, targeted users only needed to authenticate with username and password, which could be compromised remotely Enforced PW access for 5 high-value users?? Jeff Wagner testi?ed about challenges related to implementing PW functionality: Q. Were they not being enforced prior to that? A. No. Why was that? A. It was a project that was on the list, and to completely change the culture and the ?nictionality of some systems, it takes planning. 3? 1l?v'agner at 3" Saulsbury Tr. at 35. 3? June 2014 can: Incident Report at access is 1? June 2cm oria Incident Roporl at Hooacais coma. 57 Q. When you say the culture of some systems, what do you mean by that? A. So as users have built systems throughout years or decades, they have become accustomed, and there?s business or operational procedures that rely on speci?c methods. In order to change authentication methods from like user name password to some of those processes have to get rede?ned and republished.245 Thus, the challenge of fully enforcing multi factor authentication through the use of Pl?v? cards arose in part from the agency?s culture. Wagner testi?ed that maintaining the ?metionality of the production environment was related challenge in deploying PW. He said: ?full deployment of PW, caused certain applications and certain functionalities to brealt.HM Wagner testi?ed that in response to the 2014 breach remediation plan, 100 percent of windows administrators began utilising Pl?v' cards through an Xceedium appliance,? and by September 2014, all 0PM users were PW compliant.241g According to an OMB Report on Fiscal Year 2014 activities, 0PM still had not fully implemented PW card access rules. 0PM was identi?ed in this OMB Report as one of several agencies with the ?weakest authentication pro?lelsj" meaning a majority of the agency's unprivileged users logged on only with a user ID and password, making an unauthorized access more likely?? While 0PM monitored the situation in 2014 to the extent their 2014 security posture allowed, the next step was to develop a remediation plan to eliminate the attackers presence on the network. Prior to the May 2014 ?Big Bang" effort to eliminate the attackers from network. 0PM began taking other ad hoe measures to mitigate the damage. In early May, 0PM began setting up ?green zones" ?the security team?s effort to ?eliminate certain administrators from being on the network to be esploited."m Wagner described the green zone during his testimony. He stated the green zone was: E45 Wagner Tr. at 33. 3?16 Id. Wagner Tr. at T4 (Mr. 1Wagner testi?ed that, ?There is a piece of network equipment that needs to get purchased and installed to ?nalize the last couple pieces at the Macon site. But to clarify, they?re all Forced to utilize PW through the Xeccdium Appliance. There just happens to be a potential workaround that we have mitigation pieces in lace to prevent"). '43 Wagner Tr. at "t5 [explaining that the exact date that all administrator accounts began PW compliant varied based upon the location]. As of April Bil-?15, 0PM reported to OMB that ltlt't percent of their privileged users were required to use PW cards and only 41 percent of their unprivileged users were required to use PW cards. After a 30 day sprint launched in July EDIE, 0PM reported 9? percent PW card compliance as of July 5. Office of Mgmt. dc: Budget, Exec. Of?ce of the President, Cybet-Spn'nt Results {July 31, 2015) [On ?le with the Committee}. 249 Uf?ee of Mgmt. 3: Budget, Exec. Office of the President, Annual Report to Congress: federal! Information Management Act 23 (Feb. 2015} available at: defaultt?lestom btassetstegov?doest?nal fy14_fisma_rep on_32_2?_2ll 5.pdl'. cards tacilitate multifactor authentication credentials to control access. Such technology can at a minimum slow attackers who attempt to use unsecure credentials to move around an IT network. Memorandum ?'om Jacob J. Low, Din, Of?ce 3: Budget, Exec. Office of the President, to Heads of Exec. Dep?ts. and Agencies, M~l 1 1, Continued Implementation oannretrn-trl Seem-try Directive (HSPD) IE?Pott'cy?n- a Common Identgr?ientt'on Entptovees and Conn-newer (Feb. 3, 201 I), defaultt'fi lesto In emorandat? 201 1 ll . . 35? Wagner Tr. at Bil-133. 53 creation of independent machines that the database administrators utilizing that was wholly separate from the normal network so that all database access of the database that we knew [the adversaries] were looking for could only be accessed through this one controlled machine, which was not on the network?? Green none machines were configured at locations in Washington, DC. and Beyers, Deployment and con?guration of? the green zone workstations continued through May 23, 201 4. Between May 23 and May 27, the timeline does not provide a clear description of activities prior to the May 27, 2014 ?Big Bang" effort to eliminate the attackers nor provide the reason after two months of monitor May was the designated date.152 However, testimony given before the Committee does ?ll in some of this gap. Wagner testi?ed: We needed preparation to do the Big Bang. The three-day weekend was coming up. It was something that looked like a perfect time to prestage everything. However, we wanted to ensure that the users were involved and we could get full direct identity of the users when changing passwords. We didn?t want to just get a phone call from somebody saying, hey, I need my password changed. We wanted to be able to physically verify that passwords were being changed by users. So that date was specifically chose to prestage all the back?end processes that needed to be in place in order for a ?ill?user reset.253 Wagner stated the decision to remove the adversary from the agency?s network on May 2? was made as a result of the forensic analysis process and not necessarily related to how close the adversary got to the background investigation system (PIPs). He testi tied: Q. So beyond the period of time to stage the event, were the attackers moving in the network they gave you an indication that you needed to kick them out at this point? Were they getting close to Were they getting close to A. It was a point of presence in which the interagency response team felt that there was nothing more to be gleaned from the presence of the adversary. We weren?t learning anything new. They weren?t searching for anything different. And so the risk of kicking them out too early had come and gone, and new the risk was becoming having them in too long, and we didn?t want to keep them around any longer than we had to.254 Wagner Tr. at 131-133. if? June rota can: incident RBpurt at Heeaests 43131243. 22.3 Wagner at 39. :54 Wagner Tr. at 59 Wagnefs testimony?that 0PM and their partners were no longer gaining useful intelligence from the monitoring phase?is at odds with the testimony ofBrendan Saulshury, an 0PM contractor with IT Security Operations who played a signi?cant role in monitoring the attackers during this period. Saulsbury stated: Q. And you and your team were monitoring their penetration. And was there any particular danger that precipitated the decision to conduct the Big Bang when it was conducted? A. Yes. So we would sort of observe the attacker every day or, you know, every couple of days get on the network and perform various commands. And so we could sort of see what they were looking for. They might take some documentation, come back, and then access, you know, somebody else's tile share that might be a little bit closer or have more access into the system. We would sort of see them progress as we are doing our investigation. And then it got to the point where we observed them load a key logger onto a database administrator?s work station, or actually several database administrators5 workstations. At that point, the decision was made that they are too close and 0PM needs to remove whatever they were aware of at the time. Q. Okay. And that precipitated the Big Bang. When you say too close? A. They were too close to getting access to the l?IPs system.255 The distinction is signi?cant on two levels. First, if Mr. Saulsbury is correct, it is possible that 0PM had not yet identified all of the infected systems on their network, i.e. the agency had not yet identi?ed the scope of the hacker?s foothold. Second, if the adversary was getting ?too close? to the PIPS system it is likely the hacker had conducted sufficient reconnaissance of network to access that application, but had not yet successfully executed the end-stage of their back and successfully exfiltrated data. Regardless of the instigating events, the ?rst phase of the remediation plan (the ?Big Bang?) was completed on May 27, 2014.256 0PM took a number of steps in collaboration with to ?eradicate the malicious actor, at least temporarily, from network.? These steps included: removing all known compromised systems, creating new accounts for 150 known or potentially compromised users and disabling their old accounts, and forcing all Windows administrators to use PW card for authentication?? 155 Saulsbury Tr. at 2546. 2a; Saulsbury Tr. at 48; Wagner Tr. at (Wagner referring to the end of the monitoring phase as the ?Big Bang?). 3? June 2cm orM Incident Report at Hooaesis-oe I 235. 60 In addition, the ?Big Bang? effort included: resetting administrative accounts; PIV- enf?orcing all admin accounts; building new accounts for compromised users; resetting all local accounts on all servers; taking the compromised systems off line; and a ?statefnl? reset of all intemet routers.2515 0PM and their interagency partners were effectively attempting to press the reset button and eliminate the adversary?s foothold in environment by eliminating their means of mobility (user accounts) and presence (compromised systems}. 0PM continued remediation efforts and was con?dent the adversary had been removed from their environment. Jeff 1Wagner, Director of IT Security Operations testi?ed: DHS remained with their Mandiant tool for another 30 or 45 days. We even had regular checkups with where I?d go over to the - - and talk to them to see if there was any communication throughout DHS, FBI, the 1C conununity, if anything that was being identi?ed related to 0PM, and there was no communication whatsoever.qu Documents and testimony show 0PM leveraged both interagency partners and private sector technologies, including iviandiant,2m to ensure their systems, particularly the PIPS system, were clean of any malicious presence. Saulsbury testi?ed: ?The NSA blue team came into 0PM and they 1rylgelre performing both vulnerability scans, and scans for malware artifacts on the nenvork.? 1Wagner and Saulsbury admitted, however, that the attack 0PM discovered in 2015 - which led to the ea?ltration of background investigation data in the summer of 2014 was already underway during the 2014 incident response period and continued after the Big Bang?51 On or about May 2014 and while 0PM was closely monitoring the 0PM network, the attackers had established a foothold and dropped malware?? Jed Wagner new 0! It Sent-n. If)" 1 n1 FMnm~ M1uor~rn4~rl 35? June 2014 can Incident Report at Hoonosls some. :59 Wagner Tr. at 40. . 2m Wagner at 54 (?They also deployed some of their technical staff to deploy the Marrdiant tool. We didn't have at the time a deployed endpoint search mechanism. So they deployed their Mandiant to our environment to do the search for malware. Actually, there's another component. They also utilized their forensics team to do some of the forensic imaging and then malware analysis once they took the drives -- occasionally took the drives back to DHS headquarters DEIS of?ce on Globe to do analysis, forensics analysis?). 26' Saulsbury Tr. at Wagner Tr. at Saulsbury Tr. at Till-Tl. 3 re. 61 . -. - . . . . . .: -- fgjt.? (?if 1! 1'1; If?: 11": It 111-41.- 1'5. ?1 ft Jnnund if)" J'lf-j" Flap)" During the 2014 incident response period while 0PM was monitoring the attackers, 0PM observed the ex?ltration of data related to the PIPs system. The fact that this information was taken makes clear the target; ?rrther, this information likely informed the background investigation data ex?ltration that was later discovered in 2015. June 2014 Incident Report Appendix lists the data ex?itrated while 0PM monitored their network in 2014. - .. mes. PM ?a PIPE: IS Burmese Processing Brliriln?? l'ur CID 115 pp! on-w UNC- onlv. [to own Ins Modernlullon Miami-i131; Line her?ommendamn _v 15 3160?. doc 1' ?"9315? . ELL 7 Vein-Phasi- intimation Erect-anpdf Ho ?Wit-In 1 . emails? . L- V's-o P1111311 3 Investigation pd! ?ye 1 time-Plus: at Last gleam: pol Ho PM ettsiuhmtemn- tit?? vat-.152 .1 "e #1 1" 5mm? 8- Sam-cps on rSerit-s 11].pr I Ho tor. Network ozone-us we No i '1 Pin Data to: Match Het?emes. test 4 SEN) fed- Employ}: in: IT 1011!. bit: 1 Into i PIPE Dul?l?l" new ml for {1 Awareness 2314.111 Last-I 1 ?siest?rfm?rrtt . .. -. Programmer Groups doc-1 No pips contractor lot 2009.?; . . - . __Pii'S P?gr?roup Met-53.111 I Ho ?3 aP group list It! hip 1 PIPE Ust_r Malrin Mo ..-. A For- UPH. Positive-writ Protected [Unable 10011011] I Hit {or IT Awareness- 31314 lit 1 {oi-111a 11;} l: . ?mp5 1 . . . . . Pli?s Batch lab 5 requenw arts i No trips :11 1.1199"; sin: 1 Ho De?cit-rue Usage out: I?ti?itilin ult 1 Ho PIPSJIustra-?Convenion? Plan Itlt Ha By way of background, PIPE is a mainframe application on the OPM environment that stores the background investigation information provided by employees and perspective employees on forms SF 86 and PIPS interacts with several other 3? Wagner Tr. at 19; LLB. Of?ce ofPers. Federur' Investigative Sen?ice Division Information Technolog- Impact Assessment 43 (Oct. 62 Federal Investigative Services (FIS) systems and the connected and component databases contain information and materials that are considered the ?crown jewels? for a foreign intelligence - 2155 service. Based on the nature of the information held in the PIPS and related systems it was clearly a target. but Jeff Wagner Director of IT Security Operations seemed to downplay the signi?cant of PIPS as a target. He testified: Q. What is the PIP server or system? A. PIPE is an application that sits on the mainframe. Q. Why would that be a target for an adversary, that particular application? A. It?s a large data repository. Q. It?s a high-value target? A. It?s currently assessed as a high-value assessment, but it?s a large data repository. Any large data repository is always a target.255 The PIPs system is more than simply a ?large data repository." The data it stores?sensitive background investigation information gathered from forms?is some of the government?s most valuable I Documents that could in form attackers about the nature of and the architecture of PIPS and related systems should not have been permitted to be ea?ltrated from network. Appendix (as shown above) lists documents that were ex?ltrated during monitoring effort in 2014. The documents relate to 0PM IT systems, including PlPs, contractor information, and documents with names and the last four digits of those individuals? Social Security numberst Additionally, the documents listed in Appendix contain information relevant to large repositories information. The list of ?Exfiltrated 0PM Data? in Appendix identi?es 34 documents?? Appendix indicates none of the documents contained PII (except in one case where the was password protected and the adversary was unable to open 255 David Perera 3.: Joseph Marks, his?trip Disclosed Hot-l: Got ?Crown Jewels,? PoLrneo, June 12, EUIS, available at: http?w ?f?drhackers-fed era]-employees-security-baek ground?checked 395 4. Wagner Tr. at IQ. is? According to guidance, ?Fll is ?any information about an individual maintained by an agency, including any information that can be used to distinguish or trace an individual's identity, such as name, social security number, date and place of' birth, mother?s maiden name, or biometric records; and any other information that is linked or linkable to an individual, such as medical, educational, ?nancial, and employment information.? See National Institute for Standards and Technology, Special Publication Still-122, Guide to Protecting the Confidentiality ol?PersenalIy Identi?able Information I Elgdf. has .Iune 2'314 0PM incident Report Appendix at 8 {1011454246. 1 Isl. 63 it). Four of the documents, however, included the last four digits of individual Social Security numbers.Em in describing the items estiltrated in Appendix. D, June 2014 incident Report makes clear the target was PIPE. The Report stated: The attackers primarily focused on utilizing SMB [Server Message Block] commands to map network ?le shares of 0PM users who had administrator access or were knowledgeable of system. The attackers would create a shopping list of the availabte documents contained on the network ?le shares. After reviewing the shopping list of available documents, the attackers would return to copy, compress, and es?ltrate the documents of interest from a compromised 0PM system to a C2 server?? Further, there remains the important caveat from US-CERT that additional documents may have been ex?ltrated prior to OPM's monitoring phase which began in March 2014. stated: In should be noted the attackers had access to network since uly 2012 and the documents were esfiltrated during the time period of March 2014 to May 2014 when 0PM stated their advanced monitoring of the infected systems. Addition a] documents may have been exfiltrated prior to March 20?, but there is no way to determine with exact eertainty.2?2 1Wagner downplayed the signi?cance of the information ex?ltrated in 2014 and testi?ed that the information was ?standard? and would not necessarily give an adversary an advantage in a subsequent attack?? He testi?ed2014, the adversary was utilizing a visual basic script to scan all of our unstructured data. So the data comes in two forms. it?s either structured, Le, a database, or unstructured, like tile shares or the home drive of your computer, things of that nature. All the data that is listed here, all came out of personal ?le shares that were stored in the domain storage network. And when I went back to the program of?ces and had them sit down with us and do an assessment of it and look at the age and the amount of data within these, it was not recognized to be critical data or critical information. it?s pretty standard documentation, for the most part. no Id. June 2014 incident Report at Hoosesm amass-1235. 3:3 June 2014 one Incident Report at senses: a amass. 3?3 Notably, 0PM produced these documents from Appendix to the Committee in the Fall of 20'] 5 with redactions and in camera. It was only under subpoena that produced these documents without redactions in February 201d. 64 When you say ?standard documentation,? documentation that would be public accessible? I don?t necessarily knew if it would totally be publicly accessible. I don't know what everyone publishes. But like Adda and 03m packages, for the most part, are available for review; they?re traded amongst agencies. It?s not something you would be, you know, overly freaked out over. 274 When questioned further about the signi?cance of the Appendix documents, Wagner continued to downplay the significance of these documents in his testimony: Q. One of the entries includes a document that was es?ltrated PIPS contractor list? Is that the kind of information that yen would want in the hands -- not that you would want in the hands of an attacker but that would give an attacker an advantage? The Eist of contractors from was just simply a user name list of the system. It?s not something that?s it wouldn?t necessarily give them an advantage. I mean Would knowing the users on a network for a particular system Finding users is not dif?cult. For the most part, if you think about it, most companies or agencies utilize a naming scheme. So it?s fairly easy from a pen tester or an adversary standpoint to glean this information, either from initial presence or half the time you can just Goegle it. For instance, everybody?s Facebook account utilizes a Yahoo or a Google email address. It wouldn?t be dif?cult to ?nd anyone, any individual?s credentials in sop}? form to figure out what your user name to your Faceboek 1s. Saulsbury, however, disagreed with Wagner?s assessment of the sensitivity of the Appendix documents that were esliltrated. He testified that the documents could be useful to the hackers in a subsequent attack. He stated: Q. So tell me ?rst of all, are these public things that 0PM would be concerned about if they were put out into the open? Yes, those are not documents that are meant to be public. And what kind of documents are these if you could generally characterize them?l 2" Wagner Tr. at 3? Wagner Tr. at 42. 65 A. They are basically, sort of system documentation, various processes, and related to the background investigation systems. Q. So if an attacker were able to extiltrate this type of data, which it appears they did, would this give them an advantage for a future attack? A. Yes. And how so? A. It gives them more familiarity with how the systems are architected. Potentially some of these documents may contain accounts, account names, or machine names, or IP addresses, that are relevant to these critical systems.m Saulsbury's testimony indicates the exfiltrated documents in Appendix contained information relevant to understanding ?how the system works.? These documents included among other things a 2014 list of contractors with access to the PIPS system, a CEO?level brie?ng on the EPIC system and a discussion of the interface between the PTPS and Joint Pcrsotmcl Adjudication System (JPAS) systems. These documents would have improved an adversary?s understanding of system, its architecture, and information on who has access to the background investigation information contained on the PIPS system. The Appendix information is significant because it would be useful to an attacker and it provides ?irther evidence that the hackers were targeting PIPs. Nonetheless, Mr. Wagner?s characterization seems to downplay the significance oi" the Appendix D. Given the near certainty that PIPS and the information it held was a target before and continued during the 2014 incident response period, it is noteworthy that network monitoring technology did not have total visibility into PIPS. Wagner testi?ed, "i guess it would be fair to say that there was minimum visibility ofthe PIPS application Despite this lack of visibility, 0PM asserted they were con?dent no PII was taken during the course of the 20 4 data breach. Wagner testi?ed: Q. 1Without monitoring tools on the PIP server at that point, at least insofar as this is described, could data from the PIPS application have been taken prior to March 28th and 0PM had not been aware of that? 'I?hat would not be possible. Why is that? 21a Saulsbury Tr. at 2123. 27? Wagner at it}. 66 A. Because it would have to pass through the distributing environment to do so. The mainframe sits within the center of the distributed nucleus, so in order to get data out, it would have to pass through all the other monitoring techniques- And why would that allow you to see it? Because we had seen large sums of data leaving. And that would be - -- we?ve seen large spikes and things of that nature, and D118 and us, both, looked for those large spikes at that time, and we did not 1111 see any. 0PM has consistently asserted that no PII data was taken in the 2014 b1 each but as US- CERT stated additional documents may;1 have been ex?ltrated prior to March 2014, but there' 1s no way to determine with exact certaintyiir 15'At a minimum sensitive data was in fact exfiltrated by the hackers, as evidenced by the items listed 1n Appendix D- The Appendix data es?ltrated provided clues as the data targeted and the tactics, techniques and procedures of the attackers 0PM monitored in 2014 provided hints about the data breach 0PM later discovered in 2015. The attackers discovered in 2014 used Tactics, Techniques 85 Procedures (TTPs) "such as the type of malware and the attackers? ability to move throughout network?hinted at the targets of the attack 0PM discovered in 21115. These TTPs also indicate the persistence, scope, and sophistication of attacks on network. Those key pieces of information, however, were not enough for 0PM to stop the far more serious attack discovered in 2015. A public report by a threat analysis group has said the attackers discovered in 2014 used a speci?c and uncommon toolkit?or malware?designed for late-stage persistence and data es?ltrationdm The malware used by the attackers discovered in 2014 was identi?ed as two variants of Hil?t malware, referred to as HiKit A and HiKit 131.2El Notably, an October 2014 FBI Cyber Flash Alert said HiKit malware should be ?given the highest p1 io1ity for enhanced mitigation, and it ?uses rootkit functionality to sit between the network interface ca1d and the ope1at1ng system enabling the malware to sniffall trai?c toffrom the compromised ltost. ?2 2111 Wagner Tr. at 3? June 21.114 (111111 1111111211 111-111-111 at 11013111111111 -11111235. ?30 Novetta, Operation SMN. 14:1 10111 Threat Actor Group Rape? at 6 111 11, 1111111 21114 111111111 1111111111111 1111;111:111 11311111111111 1: at 110131111131 3- e01144- 124s '3 Cyber Div, F.ed Eilreau oFInvestigation, MW, FBI FinshAfeH (Get. 15,3114}, http: slideshare netfraaebeasu'ini'ragard? hikit?ash. 67 The use of HiKit malware is evidence of a sophisticated attacker that had achieved persistence on the IT enviromnent, and was capable ofperforming a variety of functions (including data ex?ltration) within network. In the 2014 incident Report, US-CERT described llikit as an ?extremely stealthy form of malware designed to hide its malicious processes and programs from detection of commodity intrusion detection and anti?virus products."233 Saulsbury described how the HiKit malware was used by the attackers discovered in 2014. He testi?ed: So the fact that it is still beaconing means that an attacker could use it to still obtain entry into network. Itjust means that they could get onto that command and control server and start issuing commands to that infected machine. So C2 means command and control. As far as it being an IP rather a domain, that?s not a significant issue. Basically, the way that their malware worked was there is a con?guration file that tells the malware where to beacon out to. And instead of it having a domain that they created, they just put the IF directly in there, so instead of doing DNS resolution it just goes directly out, so it is just a quirk.134 Wagner described Hikit as a ?form of a remote access tool, or RAG. It?s a, basically, a back?door command tool," with ?multiple functionalitics. Most malware these days are kind of a Swiss Army knife type effect. You don?t necessarily have a functionality like key logger. It usually utilizes multiple modules that allow various activities.?235 Wagner also said the Hikit malware was mostly used for persistence, or maintaining a presence at 0PM, though keylogging activity was also observed.2315 Effectively, the malware was used so the hackers could ?still use it to obtain entry into networki?m 3? June 2014 0PM Incident Report at Hoonos 13 cause. 334' Saulsbury Tr. at 1349. Wagner Tr. at 31. ass Wagner at 13. no Saulsbury Tr. at 18. 68 tampon-had Callback? Wilt-1.0311. . or", 3? 0 $1 '9 '9 ACDNFERENCE2D13 GGQQO Multiple Stages: The New Attack Life Cycle Exploitation of system First Callback for malwa no download Malware executable download Data ex?ltratlon Malware Spreads laterallyr FireEye From a presentation by Ashar Aziz, Vice-Chairman and CK), FireEye, Inc. at RSA Conference USA 2GB {Feb 23, 2Ul3) In other words, the Hikit malware is a rootkit?or a set of software tools that allow an unauthorized user to gain control of a computer system, escalate access, and persist in presence on the network without being detected. LIB-CERT explained that Hikit allowed the hackers to gain root level or administrator access to network and: [A]llow[ed] the attackers to create a reverse shell from their C2 [conunand and control] servers into the infected systems in network from a remote location anywhere in the world. The C2 servers are used to proxy the attackers? connections from their actual location on the interact in order to keep their real identities and locations hidden. Hikit allows the attacker to run commands and perform functions from a remote location as if they had the equivalent of a monitor and keyboard connected to the compromised 0PM system.233 The presence of Hikit on the UPM network was evidence of the adversary?s presence and capabilities, but it did not reveal the initial point of entry. However, the use of a rootkit means the attackers had to have high level access to network. US-CERT said, the attacker was able to acquire high level credentials by exploit a vulnerability and likely obtained access to network using social engineering methods, such a pl?shing attackm Outside threat analysis experts have described Ilikit as a ?late-stage persistence and data exfiltration tool? that 3? June and oral incident Report at aocacsrs some. 339 [ti indicates the final phases ofthe threat actor?s operational lifeeycle.m The use of? Hikit is evidence of a multistage operational lit?ecycle that would require the adversary to not only be well resourced, but also well organized?? The attack discovered in 2015 had similar characteristics. The I-likit malware allowed the attackers to remain on systems?to maintain persistence?but in order to move throughout network undetected, the attackers used Server Message Block (3MB) protocols.292 Hikit and SMBmetocols are TTPs that tend to suggest ?advanced penetration? and a sophisticated actor. 29 With respect to the use of the SMB protocols, US-CERT said, ?the malicious actors were connecting into the? server between the hours of 10pm and lilam EST with a compromised Windows domain administrator credential to search for related ?les on network ?le servers utilising 5MB conunands."294 Wagner described the attackers? use of 3MB protocols during the 2014 attack. He testi?ed: If you do some form of traversal or communications, you run over a normal communications protocol. it?s not uncommon to change the protocol language or change the protocol ports in which you do traf?c. And essentially, what they did is they tried to hide their activity and the things they were doing in a very highly utilized protocol port. So they basically hid their communications in the fuzz of the [network] traffic.295 Wagner acknowledged that the use of 5MB protocols, in addition to other were evidence of tlte threat actor?s sophistication and capabilities. Wagner testi?ed: Malware itself doesn?t indicate sophistication. The other tactics and techniques that they utilized, or other things that they did, such as hiding their commands through, 3MB, shows an advanced penetration. It?s not a simple attackzg? The use of the Hikit malware and SME- protocols by the attackers discovered in 2014 show the attackers had a well-developed foothold in environment and maintained a presence and persistence that indicated an advanced penetration that 0PM was facing in 2014. NIST described the challenge of a persistent late stage penetration: threats and identifying modern attacks in their early stages is key to preventing subsequent compromises . . . preventing problems is often less costly and more effective than reacting to them after they occur. Thus, incident prevention is an important complement to an Novetta, Operation Sii?t?' Avian: Tin-cot Actor Group Report at 15. Id. ?3 June 2cm orM Incident Report at tiocacsis -co1231. :93 Wagner Tr. at 33. June and ears incident Roport at aoeacsis amass. an Wagner Tr. at Id. 39" Wagner Tr. at 31. TU incident response capability. If security controls are insuf?cient, high volumes of incidents may occur. ability to determine the ?how? and ?how long" of the attackers discovered in 2014 was limited by signi?cant gaps in their capability to create, collect, and review audit logs of their network. Consequently, the answers to these questions remain unclear. Audit logs are collections of events that take place on information technology systems and networks.29 [n the course of a forensic investigation, a variety of sources produce reviewable log information, including: antivirus sofhvare, ?rewalls, and intrusion detection and prevention These sources can help investigators piece together how the attacker gained access, where the attacker has been, how long theJy have been there, and, most importantly, give clues as to what the attackers are after. to identi?ed numerous gaps in the centralized logging of security events at 0PM during the investigation of the attackers discovered in 2014 stating: ?Currently, 0PM utilizes Arcsight as their SIEM [security information and event management] solution of choice, but there are numerous gaps in auditable events being forwarded to Aresight for analysis, correlation, and retention.?3m Gaps in audit logging capability likely limited ability to answer important forensic and threat assessment questions related to the incident discovered in 2014. This limited capability also undermined ability to timely detect the data breaches that were eventually announced in June and July ZUI 5.3?2 If IT security teams can track the attackers? movements back to the point of entry, they can patch the system vulnerabilities that allowed the penetration in the ?rst place. The 0PM team did not, at the time of the incident discovered in 2014, have a robust logging capability that would have allowed them to determine the initial point of entry. Wagner acknowledged the audit legging gap and how that impacted their ability to identify the initial an Paul Cicltouski et. al., Natl] Inst. of Standards Sr Tech., Spec, l?ub, SUB-?lmy. 2, Secmv'rp Incident Hmtdit'rtg Guide: oftlre National! Institute and Technor'agy 2 (Aug, httm?nvlouh s. nistgovfnistoubsf?ceci alPublic atio nary rindi?. etc See generalist Karen Kent St Murugiah Souppaya, Hat?] Inst. of Standards and Tech, Sp. Pub. SKID-512, Guide to Conipurcr Scotti-Err? Lo Management (sees). :99 see cit-o Saulsbury Tr. at IS (testifying that ?There are many different log sources that we look at during a forensic investigation"). 300 Wagner Tr. at lit-18; Saulsbury Tr. at if? June 2014 cam Incident Report at access 1 s-cm 23?. 3c: 1.1.5. Office of Pets. Mgmt., Press Release, 0PM to Notify Employees of?vherzrecm-ity Incident {June 4, EUIS), Sitltifoorn?to?noti Fv-emplovees-of?cybe rsecttri ty?incidentt'; ILLS. Of?ce oFPers. Press Release, 0PM Announcer Steps to Protect Peder-oi Workers and Utitersjiem Either Tin-ants [July 9, Elli 71 point of entry. He stated: don?t think we ever necessarily found initial point of presence or point of contact. Our last log entries at best, gave us the evidence of adversary presence, was November of 211] 33"}m Wagner also testi?ed: We did forensics to try to ?nd the initial point of infection, but because we didn?t have the full volume of logging that we have today throughout 2013 or 2D12, or prior to the 2014 breach, we just ran into a point where there wasn?t logs to give as suf?cient evidence or indication of the exact point of presence?? Saulsbury also acknowledged the limited logging capability. He stated: Q. Okay. And after all was said and done and you were looking back, when were the earliest actions taken by the hackers relating to the breach? And when did they take place?I And what were they? A. So we don?t know with 100 percent certainty what the initial entry point into the network was and when it was. So what we were able to do is look back through some of the logs that can?t remember at this point what the actual -- like our earliest log entry of activity was. I want to say that we had stat}; activity at least back in 2013 that was observed, but I can?t recali at this point what the ?rst evidence that we have is.?5 The gaps in audit logs not only make it dif?cult to determine how the attackers perpetrated their hack of 0PM, but also to determine with any degree of certainty how long the attackers were in the OPM network and any data err?itrated. LIE-CERT said of the attackers discovered in 2014: it should be noted that the attackers had access to network since July 2012 and the documents below were eit?ltratcd during the time period of March 2014 and May 2014 when 0PM CIRT started their advanced monitoring of the infected systems. Additional documents may have been ex?ltrated prior to March 2014, but there is no way to determine with exact certainty?m? 0PM also could not accurately assess the risks to their IT environment because the agency lacked the necessary logging information and centralization practices to generate a full picture of how the hackers established and then maintained persistence on 0PM ?s systems. Threat and vulnerability infonnation are the I'oLrndational step in implementing risk based approach.3m ?3 Wagner it. at 12?13. as 1Wagner Tr. at 22. its Saulsbury Tr. at 14-15. ?5 Tune rem orM Incident Report at Hooaes rm Comput. Sec. Div.. Nat?l Inst. of Standards and Tech. Risk Managemem romewonli' Overview [last updated Ir?tpr. I1 2014), 7'2 The agency's inability to determine what other documents were extiltrated prior to March 20, 2014 revealed tvvo ?aws in network monitoring practices. First, from March 2014 forward, and 0PM were installing the monitoring equipment, including additional logging capabilities, to determine what was being exfiltrated going forward. This left the agency with limited ability to look backwards. Second, the gaps in monitoring practices prevented 0PM from determining what exactly was leaving the network and what data had been taken in the nearly two years the attackers had access to network. After investigating the attackers discovered in 2014, USE-CERT recommended 0PM implement a robust system audit log data practice and: Require program of?ces to send critical system audit log data to Arcsight. During the system development life cycle, security related information and auditing requirements should be identi?ed in accordance with 0PM IT Security Policy and NIST recommended guidelines and configured to be sent to Arcsight for analysis, correlation, and retention. The following log sources were identi?ed by Network Security as a high priority: Linux Secure Logs, HRTI Active Directory Logs, RACF authentication logs, and PIPS access logs. Aggregation of audit log data to centralized location such as Arcsight allows for proactive security monitoring and quicker time for triaging and remediating security incidents. (Low level of effort to implementimg Wagner testified that 0PM now (as of February 2016) has 100 percent visibility over their systems, but it is not clear when 0PM gained this increased visibility. He stated: Q. Did you have total visibility over environment during the 2014 incident? A. I would not say 100 percent. We had a great deal of visibility. Actually, at the time, we had full visibility on the perimeter. internal visibility, is where we had some gaps. Q. Why is that? A. As I said, it was an issue in which there was a longstanding project to have long entries loaded into the logger. Post the 2014 incident, that became a major priority, and we now have 100 percent visibility.? It is notable that as Mr. Wagner admits they may have had significant visibility on the perimeter of the 0PM network, but the gaps were more pronounced once the attacker was already inside the perimeter. Thus, an attacker already inside seemed to have the ability to move ?3 June 2014 one Incident Report at scenes I a not 237. Wagner Tr. at 33. 7?3 undetected across network. In a zero trust environment, an attacker?s ability move once inside a network environment would be limited by a segmented environment and strong access controls. As noted earlier, the attacker later discovered in 2015. had already established a foothold inside the 0PM network as of early llvlajg,r 2014. 74 Chapter 3: 0PM Attempts to Mitigate the Security Gaps Identified in 2014 While Iron Man and Captain America Go to Work (May 2014 April 2015) After the ?Big Bang" effort on May 2014, there were a number of events that inform the story of the data breaches announced in 2015. These events are also relevant to April 15, 20 5?when 0PM first identi?ed an unknown SSL certificate3 used to communicate with, an at the time, unknown domain: ?opmsecurityorgf?: 1' ??Opmsecurityorg" was later found to be registered to Steve Rogue?Captain America?s alter ego. 0PM subsequently identi?ed another domain, ?opmlearningorg,? which was registered to Tony Stark?Iron Man?s alter ego. These domains were part of an advanced and sophisticated attack infrastructure used to extiltrate data from 0PM in the summer oi'2014. As 0PM and a multi?ageney team began to investigate the scope and method of the attack, 0PM enlisted the assistance of two contractors, Cylance and CyTech. The multi?agency team and contractors eventually made ?ndings that caused 0PM to announce in June and July 2015 that the personnel records for over 4 million individuals and background investigation data for over 20 million individuals had been compromised.3 '2 To fully appreciate the May 2014 through April 2015 period, it is useful to establish posture with respect to mitigating the threat of the cyber incident that was identi?ed in March 2014. 0PM's IT Security Feature and Mitigation Efforts After the May 2014 ?Big Hang" On June 22, 2014, issued an incident Report to 0PM with fourteen observations and recommendations to address the security gaps identi?ed in the aftermath of the 2014 cyber incident. The observations and recommendations in this Report highlighted the poor state of IT security at 0PM and the failure to implement basic eybcr hygiene practices. The Incident Report directed 0PM to ?redesign their network architecture to incorporate security best practices." 3 '1 Brendan Sauisbury, an 0PM contractor who participated in OPM's 2014 and 2015 incident response efforts testi?ed that US-CERT deemed network ?very insecure, insecurely urehiteeted? and found there was ?lots of legacy infrastructure.?m ?a An SSL is a security sockets layer and is standard security technology used to establish an link between a server and a website. 3" June 9, 2015 DMAR at ?3 Of?ce of Pets. Padgett, Press Release, 0PM to Not-fie of?vbersecnrity Incident {June 4, US. Of?ce of Pete. Mgmi., Press Release. 0PM Announces Steps to Protect Fedora! Wot-karat rmd' Either: From vaer Threats {July 9, 201 5), othersafrom-cy lune 2e14 Incident Report at i?toenest 3:4 Saulsbury Tr. at ld-i'i'. 75 Saulsbury said this ultimately led to decision to ?create basically a brand new hardened networ they called ?the shell.?3 '5 According to Saulsbury, 0PM intended to eventually move legacy applications to the new shell.? 2014 Incident Report identified several Speci?c technical recommendations to improve network security in the legacy envirotnnent, including buying security tools and reorganizing the The Incident Report included the level of effort required from 0PM to implement each rcconunendation, from low to high. Three recommendations were considered ?low? effort, four ?moderate,? and two ?high.?313 The US-CERT Incident Report found 0PM did not have the capability to centrally manage and audit ?rewall access control lists and rules. Consequently, DHS recommended short and long term actions to combine manual auditing and scanning tools and then buy a network equipment solution to centrally manage con?guration settings while also auditing these settings against best practices. This recormnendation was considered ?high level of effort?? The Report also found {)Plvl?s network was ?extremely flat? and had ?little to no segmentation.?32H Thus, LIE-CERT recommended a redesign of network architecture with security best practices incorporated, including enforcing no direct user access to servers and requiring PW credentials for access in order to ?limit an attacker?s ability to move laterally across the network once initial access is ohtained."32' This was a ?high level of effort" recommendation. The recommendations that required a low level of effort to implement were related to logging, security awareness training1 and a redesign of Incident Response Plan. In rcconunendations related to the OCIO, found ?there is a gap in information technology leadership across 0PM as an agency? and that ?it is not uncommon for existing policies to be circumvented in order to achieve business functions while exposing the entire agency to unnecessary In response, recommended 0PM undertake a policy review and gap analysis to determine the need for additional policies to manage IT security and business functions and noted a ?cultural change will need to occur to ensure policies are never circumvented unless absolutely required.?31' DHS also recommended 3'5 Saulsbury Tr. at an M. ?i June 2014 one Incident Report at Hoonosis-on 1235. See nan orin Cybersecurity Events ?I'imcline. The Cybersccurity Events Timeline states that the UPM Security Opelations Center (SOC) began unof?cially reporting to the UPM CID in April 2014, and officially began reporting to the 0PM CID in March 2015 after the union approved the reorganization. As of March 22, 2G1 5, the relevant onions at 0PM formally approved the DCIU reorganisation- 31s June 2014 0PM Incident Report at 3? June and 0PM Incident Report at itoonosis-ooizss. id. 311 M. June sent 0PM Incident Report at noenos 1 8-?01233. 3 Id. 76 reorganizing the 0010.324 Among other things, the reorganization shifted the Director of Security Operations to report to the {310.325 Documents and testimony show OPM began to implement the DHS recommendations in or around May or early June of 2014. The effort continued through early 2016. Based on testimony front two witnesses involved in reaponding to the 2014 incident, it appears OPM tried to implement [ll-[SE recommendations, but the agency was hindered by the fact that it started with a woefully unsecure network. Throughout this phase, the attackers involved in the data breaches announced in 2015 had already established a foothold on the OPM network.316 Key 2014 US-CERT Recommendations Highlighted 0PM IT Security Vulnerabilities One of key recommendations was to ensure all OPM users were required to use cards for access to the OPM network?? in a 2015 OMB Report on IT security, OPM was identi?ed at the end of?sca] year 2014 as one ofsevera] agencies with the ?weakest authentication a majority of the agency?s unprivilcged users logged on only with a user TO and password, making an unauthorized access more likelym The OMB Report also stated that at OPM, only one percent of user accounts required PW cards for accessm Wagner, Director of IT Security Operations stated PW card enforcement did not ?ilty roll out until September 2014, and was being implemented through early 2015.330 He added the FIS [Federal Investigative Services] contractors (who did the background investigations) were the last group required to have PIV cards for access.? Had OPM leaders fully implemented the PW card requirement or two-factor authentication security controls when they first learned hackers were targeting background investigation data, they could have signi?cantly delayed or mitigated the data breach discovered in 2015. The agency ?rst learned attackers were targeting background investigation data on 3? June sent one! Incident Report at 325 OPM Cyberseeurity Events 'I'irneline- ?5 Wagner Tr. at 75-?3 {discussing implementation status of two recommendations); Sauisbury Tr. at 31-34 (discussing implementation status of six recommendations and noting logging capability gaps remain due to technical diffitmlties applying the logging function to mainframes}; June 9, 2015 OMAR at HOGR0724-001154. 32': In August 2004, the federal government initiated several initiatives to enhance cybersecurity across the federal government, including Homeland Security Presidential Directive 12 established a mandatory govemment-wide standard for seoure and reliable identi?cation for access to government 11' systems and facilities that was further de?ned as a requirement for personal identity veri?cation (PW) credentials. Then OMB directed federal agencies to issue and use Pl?v' cards to control access. OM13 reported that as of the end of ?scal year 2014, only 41 percent of all agency user accounts at the CFO Act agencies required PW cards to access agency IT systems. Critter ?eece and Date Breaches Illustrate Need for Stronger Curitrtils Across Federal Agencies: Hearing Before Subs-0mm. wt Research is Teclt. and SitlJ-tl?umm. mt Oversight ofthe ll. Comm. on Science, Space :0 Tech" 114th Cong. [July 3, 2015} {testimony Gregory C. Wilshuscn, Dir. oflnfo. Sec. Issues Gov?t Accountability Of?ce}. 32s Of?ce of dc Budget, Exec. Of?ce of the President, 20114 Annual Report tn Congress: Federal Security Management Act at 23 (Feb. 2015} available at: docst?nal fv14 ?sma report 02__2?i' 201543;?. 33" at. at an. Wagner Tr. at 33, '15. 3? Wagner Tr. at "l5. Mei-en 20, 2am.?2 vet the first data major ca?ltration involving 21.5 million individuals? background investigation files did not occur until early July 2014, giving the agency over three months to implement security controls to protect those datam Testimony from the Department of Homeland Security revealed that implementation of twoefactor authentication for remote logons in January, 2015 which was already uircd of federal agencies ?stopped the adversary from taking further signi?cant action.?3 4 If 0PM leadership had implemented two factor authentication even earlier, for example in April or May of 2014, the agency might have locked out attackers before they had a chance to commit the most signi?cant digital violation of national security faced to date. In July 2015, 0MB launched a to require all agencies to expedite implementation of cyberseeurity measures, including enforcement of PW card access, within 30 days. According to 0PM, 100 percent of their privileged users were required to use PW cards as of April 2015, but only 41 percent of their users were required to use PW cards. The agency improved its PIV card compliance?by July, percent of unprivileged users were required to use PIV cards?? In August 2015, 0PM updated its PW card implementation status in response a request from the Committee. The agency reported ?approximately 99 percent of 0PM users are required to use a PW card (or equivalent) to access 0PM workstations with two-factor authentication?a? The agency also told the Committee that 0PM bought 5,000 ActivClient licenses in 2005;j to enable the use of PW card credentials to access 0PM workstations and further clari?ed that currently 8,400 such licenses ?are activated, current, and operational.?m The agency?s response raised questions as to the status of the 5,000 licenses purchased in 2009 and why PW card enforcement was not a priority earlier, particularly given that OMB had identi?ed 0PM as an agency with one of the ?weakest authentication pro The use of basic cyher hygiene practices, such as full implementation and enforcement of card access, would have limited the damage incurred during the 2015 data loreach incidents. Den": of Homeland SecuritleS?CERT and card, can: Cybersecurity Events Timeline tang. as, 2st 5) {one Production: May 13, 2015). 333 lift. 334 Under/titted? Federal and the 0PM Data Breach: Hemfng Before tlte Comm. on Homeland Sec. ti Gm-?etvrmenml A?atts, 1 Hill] Cong. {2015) {statement of Andy Clement, Assistant Secretary lb!" Cybersecurity 3.: Communications, Department of Ilomeland Security} [adversary activity June 2014 to January "2015, stopped by security control rolled out January 2015); see Dep't ofHomeland SecuritleS-CERT and 0PM, 0PM Cybersccurity Events Timeline (Aug. 26, 5-1015] Production: May 13, 2016] {security control rolled out January 2015 was two factor authentication for remote access). 335 Of?ce of Mgmt. Budget, L-?aee. {1f [ice of the President, Results {July 31, 2015} [Un file with the Committee]. are letter from Jason Levine, Dir. Congressional, Legislative ll: Intergovernmental A?airs, US. Of?ce of Pcrs. to the Hon. Jason Chairman, 11. Comm. on Oversight 3: Gov't Reform {Aug 23, 2015). ?3 of?ce of a Budget, Exec. oraee of the President, Frrateatninnt Report to Cortgrers: Federal fryer-arteries Security Management Act 23 (Felt. 2015} available at: docsl?nal fvl-fl fisma report 02 2015.odf.. T3 0PM Efforts to Buy Security Tools to Secure the Lettjacyr Network and Rebuild OPl'ltl?s ?Very Insecure, Insecurely Architected Network? In response to US-CERT observations and recommendations in the 2014 Incident Report, 0PM launched a multi-phasc 1T improvement project to (I) buy security tools to secure their legacy network and (2) create an entirely new network environment. Former 0PM C10 Donna Seymour testi?ed to the Committee this project began alter the March 2014 cyber incident?? In May 2014, Seymour contacted Imperatis, an IT schrity contractor, to discuss the project. In an email to former colleagues at Imperatis, Seymour wrote: ?[Djo you recall all the work we did at MARAD Maritime Administration] to straighten out a very messy network with poor security? Well . . . I'm looking for an expert consultant who can guide me and my team through the onset same thing.?m Seymour and two Imperatis employees worked together at MARAD.341 Ultimately, these discussions led to a sole source contract award to Imperatis for the multi-phased IT Improvement project, in June 2014.343 The project included four phases: Tactical (securing the legacy IT environment). (2) Shell (creating a new data center and IT architecture). Migration (migrating all legacy IT to the new architecture). (4) Cleanup (decorrunissioning legacy hardware and systems). Phase I, or the Tactical phase, supported effort to buy security tools to secure the agency?s legacy IT environment immediately following the 2014 incident. The Tactical phase of the project began in June 2014 and was completed in September 2015.343 efforts to buy security tools involved interactions with a number of contractors, including Cylance and CyTech which would later provide cybersocurity and forensic solutions to 339 0PM Doro Breach: Hearing Before tire H. Comm. 0n ?ver-sight and Gnv?r Re?n-m, [?lth Cong. {June to, 2015} gtestimony ofDomta Seymour, Chief Info. Of?cer, US. Office olPers. 4? Email from Donna Seymour, Chief Info Of?cer, US. Of?ce of Pers. Mgmt., to Patrick Mulvaney and- Imperatis (May IO, Ell-4, 9:46 am}, Attach. 12 at (101463 {Imperatis Production: Sept- 1, 2m 5). Id; Imperatis Proposal Volume ll Staf?ng and Management, Attach. 5a at 262-264, (Apps. A: Key Personnel Resumes}, {Imperatis Production: Sept. 1, 2015}. Letter Contract {June 15, Attach. at scones (Imperatis Production: Sept. 1,2915). The DIG raised concerns about the sole source nature of this contract but did acknowledge given the urgency need to secure the UPM legacy network making a sole source award for purposes of buying security tools (Tactical phase] was reasonable. [15. Of?ce of Report No. 41 Hash [13. G?ice of Personnel Improvement Project 5 {June 2D 5) [hereinafter DIG Flash Attdit Alert glunc 2015)]. 43 Letter From imperatis to H. Comm. on Oversight 3c Gov?t Reform Majority StafF(Fcb. IE, 2t} 6} [on ?le with the CommittEe]. 79 Documents and testimony show Cylance began conversations with 0PM about their products through a reseller, artd Cy?t?ech was introduced to 0PM through Imperatis. The Committee obtained documents that show 0PM was buying and deploying at least ten security tools to the legacy IT environment. Websense is one such tool. in 2014, Websense had limited functionalit and simply filtered users? web traf?c to prevent access to certain sites (like gambling sites}?4 The agency had to upgrade Websense because, according to Saulsbury, the old version ?wasn?t performing? and did not include the ?advanced capabilities? such as web ?ltering, email and data security ?rnctionalitym Saulsbury also testi?ed that in 2014, the Websense server was not the primary target. Saulsbury believed the Personnel Investigations Processing System (PlPs) was the target. 43 The Websense upgrade was identi?ed as a Priority 1 task and 0PM quickly made a purchase in June 2014, but the phased deployment of this tool was not completed until September 2015.349 As of February 2015, there were continuing challenges with the Websense pilot and as of April 2015 the project status for Websense was only at about 60 percent complete?? Saulsbury testi?ed one ofthe deployment challenges was balancing ?usability and security,? but, after the 2014 incident, there was less resistance from users and security became the higher priority.351 In April 2015, according to 0PM, the ?rst indicators of compromise were detected (including the unknown SSL certificate that was beaconing to the domain ?opmsecurity.org") during the roll out ofthe upgraded version of Websense.3? The agency purchased another tool to improve network access control: 353 The agency purchased? on July 23, 2014, and deployed it from September 2014 September 2015.354 Documents show the_ deployment was delayed at least in part by required noti?cations to relevant unions. In August 2015, art lrnperatis Weekly Report stated that ?project sponsor [for is irt noti?cation stage with the Union" and the proposed mitigation strategy to ?prepare updated project tinteline, plan at memo to pilot? to non- Union Agency users.35 in the aftermath of the 2014 incident, 0PM attempted to implement recommendations, including buying new security tools and building a new IT environment, but ?4 See Infra Chapters 4, The Role of Cylance and Chapter 5, The Cy'l'eeh Story. ?5 Saulsbury Tr. at I'll-13. 3.45 Saulabury Tr. at 49. Saulsbury Tr. at 1113. 3-43 In, ?9 0PM Tactical Toolset: Purchase, Kick?cit and Completion Timcl'rarnes (Oct. 21, 2015}{1mperatis Production: (Jet. 21, 2015}; Saulshury Tr. at 50. 35" Imperatis Weekly Report (Apr. 13, 2015-Apr. it, acts}, Attach. a at 00073? {Imperatis Production: Sept. 1, 2015); Imperatis Weekly Report (Apr. 20, 2015?Apr. 24, 2015}, Attach. ti at 000?53 Production: Sept. I, 2015} 35: Saulshury Tr. at 53. 35? Saulsbury Tr. at 58-59. ?1 lmperatis Program Review {July-Aug. 2014), Attach. a at 0009?} [Imperatis Production: Sept. 1, 2015}. ?4 DPM Tactical Toolset: Purchase, Kick-oil" and Completion Time?'amcs (Oct. 11, 2015} [lrnperatis Production: Dct.21, 2015). its Imperatis Weekly Report (Aug. 3, T, 2015], Attach. {3 at 000942 {Imperatis Production: Sept. 1, 2015]. 80 because of the state of IT security at 0PM was so poor, there was much to do. The agency, however, missed opportunities to prioritise the purchase and deployment of certain cutting edge tools that, as Cyiancc CEO Smart McClure testi?ed, ?would have prevented this attack.?3m Meanwhile, as 0PM worked to deploy badly needed security tools, Captain America and Iron Man were ex?ltrating sensitive data from OPM's unsecure IT environment in the summer of 21]] 4. 0PM Missed Key Developments The Committee obtained evidence that shows 0PM was working to respond to the attackers discovered in the spring through the summer of 2014, while the attacker groups who ultimately stole background investigation and personnel records data were moving through the agency?s network. 0PM did not discover the attackers responsible for the background investigation data breach until April 2015 when it was too late. These attackers had already established a foothold in network as of early May 2014 and began to ex?ltrate this data in early July 2014. Meanwhile, 0PM continued its mitigation efforts in response to the attackers discovered in 2014. Documents and testimony show a timeline of key events that provide context for data breach discoveries made beginning in April 2015: I July 2012 Attackers had access to OPle networkm I November 2013 The ?rst known adversarial activitly begins in network that led to the breach identi?ed by LIB-CERT in March 2014. 53 I December 2013 Adversarial activity to harvest credentials from 0PM contractors begins by the attackers later identi?ed in April 2015. I March 21], 2014 USHCERT noti?ed 0PM of malicious activity and 0PM initiates investigation and monitoring of adversary. I March 21114 to May 2014 0PM (under guidance} investigated 2014 incident and monitored attackers. I April 25, 21114 The domain ?Opmsecurityorg? is registered to Steve Rogers (aka. Captain America}.3551 This domain was later used to ex?ltratc data from network. I May 7, 2014 The attacker poses as a background investigations contractor employee (KeyPoint), used an 0PM credential, remotely accessed network and installed malware to create a backdoor. The agency?s forensic logs show ?infected machines? were accessed through a connection, which was how background 35? McClure Tr. at 18. June 9, 2015 mean at 154. .138 Hearing rm 0PM Dolor Breach: Fru'l {Statemerll UfD?nna Seymour, Chief Info. Of?cer, US. Of?ce or Mgmt.}. 35a Saulsbury Tn, Ex. 4. 81 investigation contractors accessed network. At the time, 0PM gave contractors a username and password and investigators would log in with this 0PM eredential.360 I May 27, 2014 OPM initiates ?Big Bang? to eliminate attackers and complete remediation. This decision was made after 0PM observed the attackers ?load a key logger onto . . . several database administrators? workstations? and they got ?too close to getting access to the Meanwhile, the attacker that established a foothold on May T, 2014 remained in the OPM network. I June 5, 2014 Malwarc is installed.3?52 This malware installation appears to have been facilitated through the backdoor established on May 2014.3? I one 2014 contractor USIS self-detects a oyber?attack on its IT system and noti?ed USIS investigates and blocks and contains the attacker by early July, and invites us?cenr to osrs facilities to investigate by late July 2am.365 I June 20, 2014 Attackers conduct a remote desktop protocol (RDP) session indicating the attackers had escalated their access and began moving deeper into the network, contacting ?important and sensitive servers supporting . . . background investigation processes.? This RDP session was not discovered until 2015.? I one 23, 201d First known adversary access to mainframe, according to US- cents? I July to August 2014 Attackers successfully ex?ltrate 0PM background investigation data. contractor Brendan Saulsbury testi?ed that forensic lo 5 showed ?they are sort oftouching or accessing the data during the summer of 2i] 53 sea Wagner Tr. at Iii?128; Saulsbury Tr. at OPM Cybersecurity Events Timeline; Brie?ng by LIE-CERT to H. Oomrn. on Oversight dc Gov?t Reform Staff (Feb. I9, 2t] KeyPoint CEO testi?ed that ?there was an individual who had an 0PM account that happened to be a KeyPoint crnpioyee and the credentials of that individual were compromised to gain access to Hearing on 0PM Dore Brooch: Pro-r [statement of Eric Hess, KeyPoint The OPM Director of IT Security Operations [Wagner] said multiple credentials were compromised during the E?l? incident, but a KeyPoint credential was likely used for the initial attack vector. [Wagner] added ?the adversary, utilizing a hosting server in California, created their own FIS investigator laptop virtually. They built a virtual machine on the hosting server that mimicked and looked like a IS investigator?s laptop. . .and they utilized a compromise KeyPoint user credential to enter the network through the FIS contractor VPN portal." Wagner Tr. at 35. 35' Saulsbury at 25-25, at 25-25. 3'53 Letter from KeyPoint Govemment Solutions to the Hon. Elijah E. Cummings, Ranking Member, H. Comm. on Oversight 8e Gov?t Reform {July 2, res Brie?ng by to H. Comm- on Oversight Oov't Reform Staff {Feb l'ii, 2D lo]. 3? Hearing on 0.9M Dara Breach: Pair [statement of Robert Giannetta, Chief In fo. Officer, U.S. Investigations Serv?s, Despite a contractual obligation to notify contractors immediately of a ?new or unanticipated threat or hazard?, OPM did not notify their contractors {KeyPoint and U513) of the March 21314 incident. Id. 36? Hearing on OFM Doro Breach: Part II (statement of Robert Giannetta, Chief info. Of?cer, LLS. Investigations Serv's, LLC). 1? Coultcr Tr., Es. IE. 36? OPM Cybersecurity Events Timeline. 32 I uly 239;. 2014 The domain ?Opmulcarningorg? is registered to Tonyr Stark Iron Man}. ?3 I August 2014 Following public reports of a data security breach at another contractor, 0PM requested access to KeyPoint facilities and KevPoint agreed.3m I August 16, 2014 The malware installed on June 5, 2014 appears to cease operational - - - 3n capabilities. I October 2014 Attackers move through the OPM environment to the Department of Interior data center where 0PM personnel records are storedm I December 2014 Attackers es?ltrate 4.2 million personnel records?? I March 3, 2015 - is registered by attackers. Attackers would use this domain for C2 and data ex?ltration in the ?nal stage of the intrusion?"M I March 9, 2015 Last beaconing activity to the unknown domain ?opmsecurityorg? registered to Captain America, attackers switched their attack infrastructure to ?wdc- ?newstosteom? as their primary C2 domain for the remainder of the intrusion. ?5 I April to 2015 Primary,I incident reSponse and investigation period. The timeline outlined above sets the stage for the incident response and forensic investigation that took place in the spring of 2015. In April 2015, 0PM Realized They Were Under Attack Again On April 15, 2015, 0PM sent an email to reporting the presence of four malicious binaries, and what would later turn out to he the first indicators that systems had been compromised in the largest data breach in the historyr of the federal government?? so: Saulsburv Tr. at 70. the 0PM Director of IT Security Operations admitted 0PM did not have a ?fully logged? environment in the summer of 2014, but theyr were working toward that end during the summer and through the fail of 2014. Wagner Tr. at no Saulsbur}r TL, Ex. 4. Hearing on 0PM Darn Breech: For! (statement of Erie Hess, Chief Exec, Of?cer, KeyPoint Gov?t Solutions}. .rn Letter from KevPoint Government Solutions to the Hon. Elijah E. Cummings, Ranking Member, 11. Comm. on Oversight Gov't Reform (Julyr 2, 2015} [citing Report (Aug. 30, 2015}. OFM Cybersecuritv Events Timeline. an id. 3? moment nmc-newsrosreom. 'l'ItnEnTCnonmoao {let visited Eune as, sole). ?5 Sauisburv n. at 59; see not: DOMAIN nmc-newsrosreom, TI-tnEATCnowoono, available at: ln'catcrowd .orgfdomai .php?idomain-wdc-news?po st.com. 3? us. Dep't ofHoIneland Preliminary Digital Media Analysis-465355 (May 4, sets) (orM Production: Oct. 23, 2016]; Brie?ng by U.S. Office ofP'ers. Mgmt. to H. Comm. on ?versigltt tic. Gov?t Reform Staff (rapt. 13, 2016}. 83 Documents and testimony show the initial discovery of the indicators of compromise (IDCs) involved a number of parties, including US-CERT, the FBI, 0PM contractors, the OPM and several private companies. In April 2t] 1 5, 0PM discovered and began investigating the ?rst indicator that its systems had been compromised.m Director of IT Security Operations Jeff Wagner testi?ed that the ?rst indicator of compromise was an unknown SSL ccrti?catef? and was discovered during the rollout of a new version of the security application ?Websensef?w A Secure Socket Layer (BEL) certi?cate is used to establish a secure channel between an individual?s browser and a website. In this case, an 0PM computer had been communicating with an unknown website, or domain: ?opmsecurityerg.? The Committee obtained documents that show the unknown domain opmsecurityorg was initially brought to the attention of OPM by a contractor, Assurance Data, during the roll out of a new functionality for Websense technology.?m Assurance Data identi?ed opmsecurityorg in an email with the subject 0PM Daily Health? on April 14, 2015.33' 0PM was adding groups of users to Websense, as they were transitioning towards ?ltering all outbound traf?c through Websense.m During the course of this rollout, Assurance Data observed ?a certi?cate error for the domain called opmsecurityergf?m The next day, April 15, 0PM responded to Data Assurance. In an email, an 0PM tr. . ?334 - in employee descnbed the domaut as sketchy at best. the agency looked up the domain details and observed that it was what appeared to be a spoof domain?? or a domain that was purposely named to emulate legitimate looking websites belonging to or af?liated with 0PM- There were clues that ?opmseourityerg? was a spoof domain: ?it was a randomized email addressim and it was registered to Steve Rogers, aka. Captain America. 0PM provided to the Committee a document entitled Timeline? that provided more information about their ?ndings on April 15 and 16 related to the unknown SST. certi?cate. 3? June a, 2?15 DMAR at Hosanna-om 154; see also Saulsbury n. at 51-53. Wagner Tr. at 3D. in Saulsbury Tr. at 53. 330 M. ?1 Email from? Chief See. it: Strategy Df?cer, Assurance Data, Inc. to et. al., US. Of?ce ot'Pers. Mgmt. (Apr. 14, 2:36 at 133? Production: Apr. 29, it] 3? Saulsbury Tr. at 53. 33-3- If! Email from? Of?ce of Pers. Mgmt. to? Chief See. a Strategy Of?cer, Assurance Data, Inc., and et al, US. Of?ce of Pcrs. Mgmt. (Apr. 15, 211115, 9:51] am.) at 6? 1886 (0PM Production: Apr. 29, 33? Saulsbury Tr. at 59. 3? ThreatConneet Research Team, 0PM Breach Anoivsis, {June 5, 2615), available at: 84 According to this document, the unknown SSL certi?cate as identi?ed and attached to domain ?oapmsecurityerg? and ?six machines [were] identi?ed as communicating with this domain."3 The AAR Timeline also reported that the domain ?opmsecurirtyerg? was registered to ?a fake email address? under the name ?Steve Rogers.?3mg Further, the AAR Timelinc, noted that an ?alert? related to this unknown SSL certi?cate was initially discovered on February 24, 2015 and the original beaconing traffic to this domain began in December 2014.339 The MR Timeline also indicated 0PM had identi?ed three work stations and three servers on the OPM network that communicated with the suspicious domain ?opmsecurityorg."35m The investigation revealed that these machines had also contacted another potentially malicious domain - which was registered to Tony Stark, a.k.a. Iron Man and ?wdc?news?postcorn.? Two of the three suspicious LP addresses?each registered to 3 Marvel comic book character?was ?a really big red flag? for security team?? After running forensic scans 0PM was able to determine the suspicious 1P address registered to Tony Stark was in fact communicating with malwarc that was trying to under the radar as if it was a McAt?ee antivirus executable."392 This was noteworthy because 0PM did not use Beginning in 2005, US-CERT had issued alerts that APT attacks often used tnalware speci?cally designed to elude anti?virus software and ?rewalls and mentioned the use of McAfee and Symantec names in connection with these attacks.394 After identifying the false IP addresses and the malware, 0PM alerted At 6:53 p.m. on April 15, 2015, Computer Incident Readiness Team ?led a report, INC428069, identifying four malicious binaries ?les that 0PM considered to potentially be malwarc or other malicious code. Three of the four malicious binaries reported to US CERT on April 15, 2(115 were identi?ed as having the ?potential for a breach or a compromise passed a malware infection?gl? Wagner, Director of IT Security Operations, also contacted the FBI's CYWATCH to report that the 11? addresses and domains associated with the incident as potential C2 servers?the infrastructure necessary for an adversary to conduct an attackm The first evidence of the attackers' presence comes on May 2014, when the attackers dropped malwarc {Plng} onto an 0PM server that was one hop away from a machine with 3? ans. Timeline? Unknown SSL Certificate (April 15, 21115) at Hooaczcs I o? 1922 (oat-a Production: Apr. 29, 2015). ass Id. 139 Saulsbury Tr. at 59. Saulsbury Tr. a so. 192 vii lrl. 39'! Technical Cyber Security Alert Targeted Iinjnn Email Attacks (July 39? Saulsbury Tr. at ass Coulter at 14-15. ?7 Email from nsoacrso, Fed. Bucrau ofinvestigation Cyber Div to Jeff Wagner, Dir. Info. Tech. Security Operations, LLS. Office of Pers. {Ape Ill, 2015, 2:19l am.) at 1910 Production: Apr. 29, tom), see also AAR Timeline Unknown SSL Certi?cate [April 15, 2'1115} at HOGRG20316- 1922 Production: Apr. 29, 2Gl?}. 35 direct access to the background investigations and ?nger print databasem Ultimately, these attackers were able to access Local Area Network (LAM?the foundational component of internet infrastructure?and drop malwarc.39431 The P1ng malware, which is a sophisticated piece of malware, allowed the attackers to maintain a presence on system and network as of May 2015, and it also provided the attackers with other functionality. This maiware has an estimated 19,000 lines of code and comes with 13 default, modular plugins.m It provides an attacker with a ?range of functionality? including the ability to log keystrokes; modify and copy files; capture screenshots or video of user activity; and perform administrative tasks such as terminating processes, logging off users, and rebooting victim machines.4m has the ability to give attackers ?complete control over the [infected] The 1?1ng malware, which was the primary piece of malware used in the 2015 data breach, was engineered to covertly beacon back to the ?host?s network resources [and] establishing a SSL connection to malicious domains (opmsecuritymorg and wdc?news? and setting the state of a TCP connection."4m In effect, an SSL connection establishes a secure, or link between a server and a website which in this case was established between the malware and the malicious domains (?opmsecurity.org? and also found these attackers used ?opmsecurityorg?, primarily associated with the 1P address as part of their attack infrastructure?the intemet components necessary For the attackers to corrununicate with their 13?1ng malware throughout the life-cycle of the intrusion.?M Further, found [based on domain ?rewall logs) that the compromised machines on network connected with ?known malicious 1P on January 12 and January 20, 2015.4?5 Other variations of were found to have been active within the 0PM environment throughout the 201432015 intrusion. The attacker placed additional, modified versions of Plugxwdubbed by investigators as the ??rst? and ?second? variations?on victim machines on October I0, 2014 and January 31, 2015, respectively.4?? These versions of 131ng were installed months after the key objectives of the intrusion were already achieved. This shows the attacker was continuously modifying and customizing in order to better customize the malware to (JPl?vt?s network environment, maintain access, and conceal maliciorrs activities. June 9, 2015 mass at ?9 Cybersecurity Events Timeline. ?on Roman Vasilcnko :52 Kyle Creyts, An Analysis LASTLINE Lacs (Dec. 17, 2013}, Ryan Angelo Ceneza, Pulling the Plug on Plugx, 4, 2012}, vin foll usr?threat-enc yclop ecliat'web-attacl-o'r Zipul Iing-the-plug-on-plugx. 4433 Id. June 9. 2015 name at nocaorea?oer 154. June a. 2015 nan-m at 4115 In: 4? June 9, sets DMAR at Hosanna-oer 154. 86 On a related matter, the security research firm published a February 2015 analysis of the Anthem breach announced on February 4, 2015 that mentioned the ?upm? learningorg? domain?? Anthem is a health insurance company that held data on as many as 80 million Americans?current and former members of Anthem health plans, and some nonmembers.? ThreatConnect attributed the Anthem hack to a threat actor group, variously described as ?Deep Panda.?4m in February 2015 {over one month before orsrs April 2015 discovery), ThreatConnect found that this group may have also registered the domain opm~ learningorg as part of an intrusion campaign, and noted had been compromised by a likely state~sponsored Chinese actor in mid-March of [203413410 ThreatConnect warned that because the domain was registered alter the breach occurred on July 29, 2014, could be an ongoing direct target of Chinese state-sponsored cyber espionage activity?? In March 2015, it appears that the attackers changed their attack infrastructure. The attackers switched their command and control servers, installing a new, updated version of malware on infected systemsm Consequently, on March 2015, the attackers registered the domain wdc?news-posteom, resolving to the IP address? 413 The domain would switch [We to? on May 1 l, 2015, after the intrusion was already discovered.? The switch from to wdc?news?post.com was accompanied by a new version of 1?1ng nralware, dubbed the ?third version? by which would be programed to call-back to the newly-created ?wde-news-postcom? domain.?5 The March 2015 change in the attack infrastructure could [rave been prompted by a number of factors. First, it is not uncommon for attackers to use different infrastructure during different stages of the intrusion life-cycle. It is possible largceseale data had been completed by spring 2015 and the attackers were moving to a new infrastructure wholly unconnected front that used to effect the initial entry into network. In the event this intrusion and theft of data was discovered, the infrastructure used would be compromised. Second, changing the infrastructure would allow the attackers to maintain access to the network should their previous infrastructure be discovered. it is possible open-source threat researchers were dangerously close to independently discovering infrastructure used in the 0PM intrusion. 4m Threalcunnec! Research 'l'eam, Hie Anthem Hock: AH Ronnie Lead to China, (Feb. 2G l-roads-lead-to-chinaf. on Michael Hiltzik, Anthem is ?boring Consumers About its Huge Doro Brooch. More ?s o' Translation. LA. TIMI-15., Mar. 1915, m- is-warni ng?consumers?Z?l 503 'l'lireatconnoct Research Team, The Anthem Hock: AH Hoods Land to Cirirro, THREATCOMNEFT (Feb. 2&15), .na Id. or M. "3 June 9,2(315 paras at Hookers-4on1 isr. ?3 DOMAIN :e WDC-NEWS-PUST.COM, Tussarflsowuono [last visited June as, sore), lu?eatcrowd .orgt'tlornain. php?ido main=wdc-news- posln om. 1; June DMAR at 15?. Id. 8? The version of used in the intrusion had a suite oi? capabilities that were likley customized for the OPM enviroruncnt. In describing the malware, delineated the capabilities at the particular version of used in the 2014:2015 intrusion?? [T]his version of also is capable of remote access control, enumeration, ?let?directory creation, process creation, enumerating the host's network resources, establishing a SSL connection to malicious domains {opmsecurityworg and and setting the state of a connection. The ability to establish an connection to malicious domains? would become a critical component in the hacker's ability to execute command and control, maintain access, and ea?ltrate data out of network. Hackers used the Ping to create fake SSL certi?cates that would allow host machines to connect to the malicious domains ?opm- and The use of these SSL certi?cates eventually led to the discovery of the intrusion. In April 2015, 0PM security personnel began installing Websense, which gave 0PM an enhanced ability to ?lter SSL certificatesm During the Websense roll?out, the newly installed system was able to flag fake SSL certi?cates to and other malicious domains. it is not entirely known how, or even when, the attackers gained access to an 0PM network credential held by contractor Keyi?oint, but the attackers were able to use that credential to gain initial access into network, using a virtual private network. login to access an 0PM SQL server. The attackers also setup remote desktop protocol (EDP) sessions from the SQL server to move laterally, infected additional systems and gained additional footholds until ?nally connecting to their primary target, the background investigation and ?ngerprint databases. The KeyPoint credential was ?utilized for the initial vector of infection.?m but a number of compmmised credentials were used over the course of the data hreach.42' The credential that was used at the initial vector of infection, the point at which the adversary drop ed malware to obtain persistent presence, was being used by a KeyPoint employee?s account.? But that KeyPoint employee did not have administrator credentials, which are necessary to conduct higher?order functions on IT environment. JeffWagncr testi?ed: So the adversary utilised tactics in order to gain domain administrator credentials. Exactly how they obtained the credentials, we don't have forensic evidence for, but they needed to gain another set of credentials to do operations. We not the only set of credentials they utilized to perform operations. So there are multiple stages where various June 9,2015 omen at not: acres cc1154. 4? June a, 2015 DMAR at Hooaovze not 154. Saulsbury Tr. at 53-59. 4:9 Saulsbury Tr. at 53-59. Wagner Tr. at 85. ?m Wagner Tr. at 36. 42: Wagner Tr. at lid. 88 credentials were used, and though us enforcing PW killed the capability of them utilizing the KeyPoint credential, they still had persistence from the malware. So they were able to get into the environment through another method to maintain persistence and then utilize domain?a After gaining access to the SQL server, the attacker opened a RDP and dropped malwarc to maintain a presence on the SQL server. The SQL server itself is significant for its use as the ?back and storage" for various 0PM applications, including a Jumpboa server used by the administrators that had access to background investigation data. Saulsbury testi?ed ?this jumpbox had access into the environments, into the network segments that contained the background investigation systems."424 The attackers used an RDP to enter the jumpbox and use it ?as a pivot point to access all of the systems that were firewallcd off from [the] normal network??125 The move from the SQL server to the jumpbos was a "lateral movement? by the hackers, and it demonstrates their ability to maintain a presence on systems, and also to gain the necessary administrator credentials necessary to move from system to system, from computer to computer. Using the jumpbox as a ?pivot point,? the attackers were able to access the PIPS mainframe, which stored the background investigation data, and ?all the FTS boxes? which ?are related to the ?ngerprint transmission system,? and ?nally the human resources department?s systems with personnel records stored on systems hosted by the Department of the Interior-.425 These lateral movements, as evidenced by RDP sessions and the timestamps on the variants, continued from May into June of 2614.42? With access to mainframe as early as June 23, 2014 {and less than one month alter the May 2014 ?Big Bang?), the attacker would have had access to mainframe applications such as the background investigation data stored on the system?? By early July 2014, the attackers began to ex?ltrate the background investigation data. Evidence of data ex?ltration would appear to 0PM and US- CERT in the form of RAE archives??crashes" of stolen datam The attackers continued to estiltrate the background investigation data through August of 2tl14,4m but the ?ngerprint transaction system data was not taken until March 26, 2015.43: 4:3 Wagner Tr. at 36. 43': Saulsbury Tr- at 75 41: I'd. "35 Saulsbury Tr. at or Coultcr Tr., Ex. 13. ?3 0PM lCybersecurity Events Timeline. Coultcr Tr. at 25-35. Mr. Coultcr would go on to describe the attackers? use ot'R?R files to ex?ltrate data Saying, ?so as is common in a lot cases, or actually a lot of breaches, iftlieir end goal is to collect data, then they're going to search for it and bring it back to a central point for aggregation. A lot of times data, like this email, if you were to compress it, it would be, you know, potentially one?l ??th ofthe size. So EAR, which is a compression Format, is used to shrink data. You can also then apply a password cases, where there is data ea?ltration or a con?rmed breach, it?s very common to find these compressed, stashes of whatever bad guys were after.? See also June 9, 2015 DMAR at 156. 0PM Cybersccurity Events Timeline. June 9, acts paras at Hooacrza-oct 15a 39 The time period from early July 2014, when the attackers begin to exfiltrate the background investigation data, to April 24, 2015, when 0PM ?successfully eliminates [the] adversary from their systems" represents the data breach end-stagefli?2 In this ?nal phase, where the attacker achieves their primary objective whether it is accessing and eitfiltrating data or some other malicious activity it is important to note this end-stage would have been preceded by an initial penetration through 0PM's defenses, an intelligence gathering phase to learn about network, systems, and security measures. Then after all of this activity the attacker would ?nally drop the maiware and set up the domains necessary to collect and extract data. The details of the initial phases of the attack and how the 2015 attackers penetrated defenses and gained sufficient knowledge of systems so as to quickly begin cx?ltrating data, likely will never be known. What is known is how 0PM discovered the data breaches announced in June and July of 2015 and how 0PM, their interageney partners, goveniinent contractors, and private sector incident responders took 0PM from the initial indicators of' compromise discovered on April 15, 2015 to remediation of the incident in June 2015. Between the ?rst si of the attackers? foothold on May 2, 21114,?3 to the ?rst esfiltration of data in early July 2014} 4 0PM would complete the ?Big Bang"435 to expel from their network the attackers discovered in 2014. From perspective by the end of May 2014, the 2014 incident was over little did 0PM know that the 2015 data breach operation was underway. The following chapter provides additional details on 2015 discovery and incident response effon's that ultimately led to the discovery of background investigation and personnel records that were exfiltrated from the perspective of an 0PM contractor called Cylance, which was brought in to assist 0PM in April 2015. 0PM Cybersecurity Events Timelinc. 43.3 0PM Cybersccurity Events Timeline. m- GPM Events Timeline. 43? Email from Press Secretary, U.S. Df?cc of Pers. Mgmt., to Jeff Wagner, Dir. of IT Sec. ?perations, UJS. Of?ce of Pers. Mgmt. (June 18, 2015, 8111 pm.) at HUGH (0PM Production: Feb. 16, 2616}. 432 90 Chapter 4: The Role of Cylance Inc. Cylance Inc. ?s information security tools detected critical malicious code and other threats to network in April 2015. While Cylance tools were available to 0PM as early as June 2014, 0PM did not deploy its preventative technology until after the agency was severely compromised and the nation?s most sensitive information was lost. OPl'vl's IT security operations recommended deploying Cylancc?s preventative technology, CylanceProteet (Protect), to insulate enterprise from additional attacks after it became aware in March 2014 of a data breach whereby sophisticated adversaries targeted background investigation The Committee obtained documents and testimony that show internal bureaucracy and agency politics trumped security decisions, and that swifter action by 0PM to harden the defenses of its enterprise architecture by deploying Protect would have prevented or mitigated the damage that systems incurred. to .1 one 2014, 0PM began evaluating numenous products, including two Cylanee products, for possible use in its legacy environment?? The agency?s consideration of these tools occurred at a time when the agency was aware its existing environment had been compromised and vulnerabilities had been exploited by a sophisticated adversary. On March 20, 2014, US-CERT noti?ed 0PM that data had been exfiltrated from system.433 Agency of?cials later testi?ed this data breach resulted in the less of security documents and manuals about high~valued systems and applications on its enterprise architecture, but downplayed the significance of these documents June 2014 0PM Incident Report highlighted the sophistication of the attackers, which used ?an extremely stealthy form of malware [a Hikil rootkil] designed to hide its malicious processes and programs from the detection of commodity intrusion detection and anti?virus products."440 A rootkit is malicious piece of software that uses administrator or "root? access to modify system settings to hide malware and malicious code at lower layers of an operating system, rendering itself and adversary activity almost undetectable by common anti?malware software.? From March 20, 2014 to Mayr 2014, 0PM and US-CERT observed the attackers to learn more about their tactics, techniques, procedures and objectives including the ex?ltration of date??- In the ?nal US-CERT June 2014 0PM Incident Report, stated: Wagner Tr. at ?92. 43? McClure Tr. at 14. "3'3 June ems orrvt Incident Report at Hooaosts-oo 1 233. Hearing on 0PM Dore Breech: For: (exchange between Chairman Jason Chaffetz and UPM Dir. Katherine Archuleta and 0PM 011'. Donna Seymour]. June 2914 0PM Incident Reporl at see supra Chapter 2 The First Alarm Bell Attackers Discovered in 2?14 Target Background Information Data and Ex?ltrate System-related data 441 P?zer is a Remit-ft, AVG available at: rootkit. ?3 June sore orlvr Incident Report at scenes 1 s?oo1233. 91 [T]he attackers primarily focused on utilizing [Server Message Block] commands to map network ?le shares of 0PM users who had administrator access or were knowledgeable of [Personnel Investigations Processing System] system. The attackers would create a ?shopping list? of the available documents contained on the network ?le shares. After reviewing the ?shopping list? of available documents, the attackers would return to copy, compress, and ex?ltrate the documents of inter?gt from a compromised 0PM system to a [Command and Control} SBWEF. The discovery of a successful intrusion and data breach in the spring of 2014 put 0PM on notice. Sophisticated attackers defeated their information security measures and practices, and remained unnoticed as far back as July 2312.?4 The attackers had a clear objective: the background investigation material contained in PIPE. In other words, 0PM had every incentive to take swift, decisive action to immediately fortify its legacy systems against a persistent threat that already had secured an advanced understanding of environment, including its highest valued targets. The agency purchased select tools from various vendors in June 2014,445 but declined at this juncture to purchase a key preventative tool recommended by the OPM Director of IT Security Operations called CylanceProtectm' and only bought its more limited tool, CylanceV.447 The agency?s security personnel remained interested in Protect, and Cylance arranged an extended demonstration in early 2015.443 When 0PM identi?ed an indicator of compromise on April 15, 2015, the agency turned to Cylance for assistancem As soon as 0PM began using the Cylance tools in April 2015, it immediately began ?nding the most critical samples of malicious code on its networkm Cylance tools identi?ed a signi?cant amount of malwarc on network within 48 hours,451 and Cylance personnel quickly recognized the agency?s cyber situation was dire.452 4ylanee personnel even con?ded to each other internally over e-mail: ?They are fucked btw." 53 By April 2015, it was too late to undo the damage. Following the May 2014 Big Bang, 0PM decided not to purchase and deploy Protect as a result of internal bureaucratic ?3 June aura om Incident Report at Hooaos t?noI234-35. June 2014 0PM Incident Report at Hoosost-ootzss. ?5 0PM Tactical TooIset Purchase, Kick-off and Completion Timeframcs (Dot. 21315) (Imperatis Supplemental Document Production: Oct. 21, 2015) {on ?le with the Committee). Wagner Tr. at 91-92; see also McClure Tr. at 35-36. McClure Tr. at Ill-2'0. 413 Id 449 Coulter Tr., Ex. 2; lj-mail from Matthew Morrison, Assurance Data, Inc., to Jeff Wagner, Dir. Info. Tech. Security Operations, 11.5. Of?ce ofPers. Mgmt. (Apr. 15, 2015, 10:43 pm.) at (0PM Production: Apr. 29, Z?lti}. no Coulter Tr., Ex. Saulsbury Tr. at 72; Email from to Brendan Saulshury, Senior Cyber See. Engineer, ERA (Apr. 5:19 pm.) at 75 Production: Dec. 22, 451 ISouller Tr., Ex. 3; Saulshury Tr. at 7'2. "53 McClure Tr., Ex. 9; Coulter Tr., lint. 5. 45.1. M. .92 hurdles and ?political challenges on the The Big Bang remediation proved unsuccessful; the malicious actor linked to the theft of personnel records, background investigation data, and ?ngerprint eit?ltration had already gained a foothold in system by May 3314.455 The malicious actor downloaded 1?1ng malware on May 7, 2014 on a key Microsoft SQL serverm' at 0PM, and had moved laterally across the network to access the mainframe (which holds background investigation data) on or about one 23, Mil/4.45? The attackers ultimately es?ltrated background investigation data from early uly through August 2014, agrd then exliltrated personnel records in December 2014 and ?ngerprint data in March 2015.? Overview of the Gylanee Cyber Tools In June 2014, Cylance and 0PM personnel began conversations about the potential use of Cylance?s products in the agency?s legacy (existing) information technology environment.?LJI At this time, Cylance offered two products to the marketplace. CylanccV (V) is a detection product used on end-point devices desktop computers, laptops, etc.). First available to the marketplace in October 2013, software scans endpoints to determine ?whether or not something is malicious on a computeri?w Deployment of?v? is limited to one endpoint at a time. The product is focused on detection?rather than prevention? ofa cyber threat. Cylancc CEO Stuart McClure testi?ed that ?will find where an infection might already be or exist, and that will help IT operations to go into the computer, clean whatever they want to that system. But is not preventive. It just is after the fact [it] will catch somethingm?' Protect, on the other hand, is designed to prevent malicious activity. It is distributed throughout an enterprise where it utilizes mathematics and algorithms to determine ?good" from ?bad." That is, it seeks to identify and address items that do not belong within an enterprise that could be a threat. The agency?s tlu'cat detection and initial response efforts in the wake of the March discovery revolve, in part, around the two modes available tlu'ough Protect: ?Alert" and ?Auto Quarantine." In Alert mode, Protect places the onus on the administrator running the tool to determine whether or not Protect has identi?ed a malicious computer process that should be quarantined, or if it should be ?white listed? and remain operating on the environment. When ?4 McClure Tr., Ex. 4; McClure Tr. at 44-45. ?5 0PM Cybersecurity Events Tirneline. "55 June 2014 0PM Incident Report at 154; 0PM Cybersecurity Events Timeline. .rsr Coulter'fr. at 79-32, Ea. it! {Email from Christopher Coulter to Jonathon Tends]; 0PM Cybersecnrity Events Timeline. ?3 Cybersecurity Events Titneline; Brie?ng by us-csnr to a. Comm. on Oversight a Gov?t Reform stair (Feb. I9, June 9, 2315 DMAR at 153. 4? McClure Tr. at 14 (The Cylancc sales team was introduced to IT security personnel at 0PM through Assurance Data. L'ylance?s sales staff, Nicholas Warner, was introduced to IT security personnel through Mathew Morrison at Assurance Data}; McClure Tr. at 124 3 {Assurance Data maintained a re-scller arrangement with Dylan-cc}. ?0 McClure Ea. 1; McClure Tr. at E. 4? McClure Tr. at E. 93 Protect is operating in ?Auto Quarantine" mode, it automatically removes and quarantines threats, thereby requiring no intermediary action. McClure testi?ed: ?[Protect] sits on a computer in real time and watches everything that happens on a computer. And every single element of the computer determines whether it?s good or bad, whether it's safe or unsafe, malicious or not. And ifit?s malicious, it stops it. it blocks it. It doesn?t even allow it to start. So true true prevention."4m According to McClure, V: [R]equires a user to actually hit a button that says point to this drive or point to this computer or this share, whatever, new hit scan. It requires a physical body to do something like that. Whereas, CylanceProtect, the agent, can be completely hands-free. . . . If you just set it into auto quarantine mode, just forget it. If you have an alert mode, of course, then you have to review the alerts hopefully and then try and quarantine whatever things you ?nd that are bad in therefi63 On April 15, 2015, 0PM reported to LIE-CERT the ?rst indicator of compromised? This led to June and July 2015 armouneements regarding the loss of 4.2 million personnel records, 21.5 million background investigation, and 5.6 million ?ngerprints. At this time, 0PM owned V, but had not yet purchased Protectims 0PM Director of 11" Security Operations Jeff Wagner described how malware was discovered in 2015. Wagner testi?ed that an indicator was found, then it was followed back to an infected server, and then the search began the the malware on the infected servanm Wagner testified: [Tlhe initial malware discovery on an infected machine is normally not done by, say, a tool. It?s done once you find an indicator and that indicator points back. Then you use a tool such as Mandiant or Carbon Black or Cylance or various tools to do an overall search, because once you find one piece and you at additional indications, you can then look for other indications as Wagner testi?ed that the unknown SSL certificate was ?discovered by Websense" and that ?Cylance would have found the speci?c malware on the machine. And then one engineers would have reverse engineered the malware to find it written within the malware.? 53 ?53 McClure Tr. at 8-9. 463 McClure at 46-41 June a, 2015 DMAR at Hosanna-eel 154. ?5 McClure Tr. at 2D. 4543 Wagner r. at 54. 4s: 1 liv'agner at 54-55. an: Wagner Tr. at 3t]. 94 On une W, 2014, the agency purchased an upgraded version of ?ti?tiebsense?hi'q to replace an older Websense to ?enhance the capability to include protection of remote users while attached to foreign networks.?m Documents show the upgrade started on September 9, 2014 and was completed by September 2015.471 By April 2015, DPM's IT Security Operations began to deploy the upgraded version of Websense and during this deployment process identi?ed an initial indicator of compromisem Saulsbury testi?ed: We originally detected [a problem] during the course of the Websense rollout as we were sending groups of users, adding more and more groups of users to the pilot group, to have all of their outbound traf?c being ?ltered through Websense. One of the things that we were doing was SSL Because that is such an intrusive method of inspection, we were monitoring for errors with $31. certi?cates that were potentially breaking access to applications, updates, and things like that.?1 3 Saulsbury continued to describe the ?ndings while rolling out Websense saying: [W]e also looked at the 1P [sic] domain resolved to and put it into NetWitness. We were able to see that going back we had these three machines that were going through Websense, but we also had tlu'ee servers that had been contacting this IP address. It looked very strange because there wasn?t any business connection between these users' work stations and these three different servers. So that is when the red ?ag started to go up as this could potentially be malicious activity.??1 At 6:53 pm. on April 15, 2015, Computer Incident Readiness 'l?eam (OPM-CIRT) ?led a report, Newscast, with LIE-CERT, and it was assigned incident number l?NCth?ttDO459698.4i5 ?9 is New Forcapoiut. FDRC EPDINT, available at: (?On January 14, 201d, Raytheon announced that it was rebranding the product FonsepointTM as part of a new venture between Raytheon and 1v?ista Equity Partners"}. List of Tactical Security Products Production: Clot. 2] HHS). Id. Sauishury Tr. at as. 473 M. 4m Saulsbury Tr. at 59. E-mail from to can" (earn) (Apr. 15, 2015, as: pm.) a {own Production: Dec. 22, 2.015]. 95 From: Sent; Wednesday April 15. 201R ti '14 PM To: Subject: follow Up on lririrlent rel- r'umnei W1 le4?Fl(lfi?l hm. wieiveil your report NEHSUEEI and has assigned Incident number for ltilure reference. incident Submit Date: 411512015 6:53:13 PM Thank you, US-Cllt'l Operations Center As 0PM began to grapple with the developing cyher incident, the agency also discussed the possibility of using Cylanee tools to step the malware from functioningm The documents show there was already a high degree of familiarity with the Cir/lance products and their capability, but that 0PM did net have full access to the tools.?1 Message From: Matthew Morrisrm Sent: MISHOIS 10.4313 PM To Wanner, Jeffrey csoim - JPann er] Subject: Eviance I also have Cvlancc on ready to deploy protect to the veneer-?rs desktop and servers. it WILL step malwaie from ru q? - matt 063% As of the evening of April 15, 2015, 0PM owned but did not have the latest version of nor did 0PM have access to Protect, the preventative tool.?3 The next morning (April 16) Cylanee offered assistance to 0PM as the agency was attempting to point at endpoints, and soon thereafter provided technical support to 0PM via conference call to help 0PM overcome ?incompatibility? issuesm Chris Coulter, Cylance?s Managing Director of Incident Response and Forensics, testi?ed that was] trying to use against a forensic image, and the methods to do so aren't we E?mail from Matthew Morrison, Assurance Data, Inc., to Jeff Wagner, Dir. Info. Tech. Sec. Operations, U-S. git-LDC ofPers.Mgn1t. (Apr. 15. 21315. 10:48 pm], at (0PM Production; Apr. 29, Ellie). I . 4n Coultcr Tr., Ex. 2 [In this email, Matthew Morrison {with Assurance Data) wrote to Grant Moerschel {Cylance Sales Engineer), seeking the latest Cylance versions, copying Nicholas Warner [Cylanee sales director), OFM Emanuel and DPM contractors, including Jeffrey Wagner Director of IT Security Operations)). Coullcr Tn, Ex. 2; McClure Tr. at 65. 96 clearly documented because it's more of a trade craft to know how to do that."43? Coulter offered to be onsite at 0PM the following morning if the incompatibility issue with was not resolved.?' Jonathan Tonda (then an 0PM contractor in IT Security Operations} replied: ?We were able to resolve the issue and obtain results from Cylanee. Thanks for your help!"432 Hi am Confidential >3 -. From: Tonda. Jonathan D. [na?i?ttm? Ir Sent: Thursday. .?pr?n 1.6. 2015 4:15 PM a To: Chris (cutter-I Jr Hag Cc: Sau?lsbury, Brendan B. on subject: RE: cylaoce versions as H1 Chris, is we were able to resolve the issue and obtain resu1t5 from Cylance. Thanks for your help! .. "Jon At 3 :56 p.1n., Saulsbury sent Wagner a list of Four malicious eaccutables identi?ed by that were residing on 0PM servers, and each malicious executable was assigned a score under the Cylanee rating system.?3 McClure described this rating system in his testimony to the Committee. He stated: So we rank and score ?les and exceutional elements in a spectrum from positive 1 to negative 1. Anything from a positive 1 to a zero is considered safe mathematically. Anything from zero to negative .8 is considered abnormal. And then From negative .3 to negative 1 is considered unsafe.MM Three of the four malicious executables found by on April to, 2015 were rated -1 and the fourth was rated ?.93 on the Cylance seale.435 Coulter testi?ed that the tiles showed ?That there?s a potential for a breach or a compromise [past] a malware infection.?436 One of the four ?les included a Windows Credentials Editor (WCE). Coulter described the signi?cance of the ?nding: So malware, while, as nasty as it can be, is fairly common, at least in a broad sense. Somebody actually has to use that malware for it to be malicious, most of the time. When you see something like a con?rmed Windows Credentials Editor of other types of credential dumping tools, that?s usually a sign of an overt act, so something that somebody with ill intent actually was trying to achieve versus just a presence of a ?35 Coulter Tr. at mil 1. an Coulter Tr., Ex. 2. 432M- 433 Coulter Tr., Ex. 3. McClure Tr., Ea. sass. ass Coulter Tr., Ex. 3. I{joulter Tr. at 97 malicious file, which may or may not have been used. A WCE 64 doesn?t just appear for -- just to have it there. It usually is used?? LIB-CERT would later confine WCE as a ?hack tool.?433 On April 15, 0PM found another suspicious file?a McAt'ee dynamic link library (DLL) called ?macutildll? that Saulshury recalled in testimony as being integral to the attacks: So we took Cylance and put it on the known infected machine with the McAfee macutildll malware -- so the machine with the mcutildll maiware and then we ran Cylance on it to scan the machine for malicious artifacts. And what it came up with is it successfully identi?ed that mcutildll ?le as malware.439 The McAfec file was highly suspicious because 0PM did not use McAfee in its systems. Saulshury stated: ?It was basically trying to fly under the radar as if it was a MeAfee antivirus executable. The problem is that 0PM doesn?t use McAfee, so that stood out right there to us that, at that point, I was 100 percent certain that this is malware that is heaconing The next day, continued the malicious nature of this file. On Friday, April 17, 2015 at 1 1:39 am, Saul shury processed a new rnalware submission to US-CERT for its review that included the tiles he shared with 1Wagner the night before.? At 5:19 reported to 0PM its iairiai analysis of the executable US-CERT reported that the mcUtill.dll was a ?loader??an operating system component that copies programs to memory. When executed by a seemingly innocuous executable mcutill.dll decompresscs, and loads a third ?le into memory This file is the primary - or payload for a remote access tool (RAT) called Plugx. Each ofthese tiles was contained within a ?McAfee?VC? folder, which also contained an output file for the keyiogger used the malicious domain ?wdc- newsporteom? for conunand and control.493 In other words, the four tiles contained in the folder, which resided within a directory called worked in concert to harm 0PM, and did so in a way that was hard to detect. Each of the four files had a speci?c function: 4s: Tr. at 16. are U.S. Dep'l of Homeland SecuritnyS-CERT, Malwarc Analysis Report-46635? [April 17, 21315} at HOGRUD92 50PM Production: Dec. 22, ENE). so Saulsbury at as. Saulshury Tr. at email From? to Brendan Saulsbury, Contractor 0PM Security Operations (Apr. 2015, pm.) at 7'5 (DPM Production: Dec. 22, 2015). 4" Email am? it] Brendan Saulshury, Contractor orwi rr Security Operations (Apr. ices, 5:19 pm.) at ts (0PM Production: 2 Id. Iri; June a, 2015 cams at 157. 98 I is an .dll ?le and Plug}: malware considered malicious. After analysis of the Master File Table USE-CERT found that the ?le was time- stamped. Documents show the creation date was March El, 2U15 at 6:13:?l am. I is a binary itself and is innocuous; however, it is used to load the [?1ng malware through McUtildll- Analysis of the MFT shows the ?le was time-stamped. Documents show the creation date was March 9, 2015 at 15:13:01 am. I is a binary that has been identi?ed as a 131ng loader. It attempts to connect to the malicious domain which resolves to US-CBRT found the attacker time-stamped the ?le. Documents show the creation date was March 9. 2015 at 6:13:01 am. - was found to be the output ?le created to store the key strokes recorded by in addition to key-logging, this version of is capable of remote access control, enumeration, ?lefdircetory creation, process creation, enumerating the host?s network resources, and establishing a SSL connection to malicious domains.W4 reported 1?1ng was located in two 0PM directories: a McAfee folder and a dimmrl? 611?? (i?"l495 From: Sent: Frlnav. ?015 5 as PM To: 5- mma?, Paw-dw- Le: Sui-lien: ll? H-?n 5 lhlt?iib (lit-Fi?! Itch-ct. Update: untiware valthin theI tone on car Marvel} very tn lint- malware Iatml with another Mhl'l ruleasvdvotl the holders contained two loads-rt. named Mrallil rill "was small Inarlou are written ll'1 Msemhls language and are um ln damp? and as the loaders withm the other Mle We loader: themselves are loaded with the volrtl Mchfoe tool :tl-us tool :3 not malwarc Itsettl. they in turn load and decode the file: [found ?It the folders}. Thr- files will In turn launch the conlamerl within the [dc In this the: utilized for command and control Mill the. PLUGK RATS is as follows: ?Hollows-[min June 9, 2015 LINEAR a 154. a veneer Digital Media amass Report pmvidcs detailed analysis and insight into the speci?c tactics, techniques, and procedures observed on the media submitted for June a, lots DMAR at 155, 99 On April 2015, arrived at 0PM's headquarters in Washington, D.C., to provide on-the?ground assistancem That day, 0PM decided to deploy Protect, but only in ?Alert? mode (not in auto-quarantine mode).4W Since 0PM had been familiar with the product since June 2014, but still did not execute a purchase, Cylanee staff was skeptical about whether this time the agency was truly moving to purchase and deploy Protect. Cylanee sales engineer Grant Moersehel emailed Coulter: "Is this a [Proof Concept] in their mind or the statt of a real deployntent?"4qa Coulter replied: ?Not entirely sure what the back stories are, all Iknow is they want this on all systems by the end of today?? Director of Sales Nick Wamer replied: ?It?s go tirnel"m To Nicholas ?Hunter 2 inbjeer RE: 0PM Protect Access From: Ntcholtts Warner Sent: 201.5 AM To: Stun-rt McClure. Subject Fwd: 0PM Protect Access It's go time! NW Begin forwarded messag- an1: Date:Apn .- . tll' Te: Chris E'o-nlter - o: ?mas Warner -. Gram Meen'eltel . Subject: Re: 0PM Pretect Access {Ill-t. Keep Support. _nd I the loop. We will do what we can to help. glnut [In Apt 2015. :11 3'13 AM, Chris Caulk-1' wrote: Not entirely sure what the back stories are. all I know is they want this on all systems by the end of today. Sent from my iPhone On Apr 1015. at 10:11 AM. 'wl'ole? Unis 7? - a Director of IT Security Operations, Jeff Wagner, testified that ?we initially started using Cylanee for rnalware analysis. Within a day or two, we obtained the Protect. It was part Coulter TL, Ex. 2; see m?sa UPM 1 it"isitor Log Washington, DC. [April 1, 2015 to July It), EDIE) at (0PM Production: Feb. 16, Coulter Tr., Ex. 17?. "95 McClure Tr., Es. a. Id. at 100 of our license, lbelieve.?SDI As oprril 2015, 0PM had not purchased a Protect license and did not purchase such as license until June 30, 2015.?32 Nonetheless, Cylance provided 0PM full access to Protect in mid?April 2015 on a demonstration basis and without purchasing a license because as Cylance testi?ed it was evident 0PM was under attack and they deemed it the appropriate course of action. McClure testified: A. Yes. So typically, like we say, an evaluation of this sort would be a small evaluation. However, when it?s under these kind of incident response emergency situations, we allow them to install on as many boxes as they want. Because we just want to help them, provide them the support, get them to be able to identify the problems and then prevent them, clean it as quickly as humanely possible, get the bad actors out of the company, organization- So we allowed them to install on all of them, as many systems as they had a little unusual for an evaluation but not completely unusual, especially under these circumstances. Q. Those circumstances being? A. That they were under severe attack and had been for quite some time. Q. And you just described incident response efforts going on. Are you aware of the sense of urgency in how 0PM was responding to what they found and flagged for your attention the day before? A. Once we were engaged on April [6th, l'r'th, it was very much a ?re drill, every 24 hours. And they were taking it very, very seriously from all of our observations, and reacting as quickly as possible, and getting as much help as they could, and engaging with us, and getting the technology out there, and trying to quarantine as quickly as possible. It?s actually one of the poster-child examples of how to do it properly in an investigation, just as soon as you humanely possibly know that you?ve been breached, to try and roll out this new tech. 1 think they did an admirable johm With respect to why 0PM utilized Cylance tools in April 2015, Wagner testi?ed: We were uncomfortable with just trusting that we knew all the indicators of compromise. And so we obtained the Cylance endpoint client and 5m Wagner Tr. at 95. McClure Ex. see arise Cylancc Purchase Ii'lrrler from Assurance Data, Inc. [June 3G, at CYLANCE DDGDIE [Cylattcc Production: Dec. IT, 2015}. McClure Tr. at 53-59. 101 d?pl?y?d it, and then a Cylance engineer helped make sure we got it configured Wrist-112v? in get proper information out of it?s? Wagner also testi?ed that Cylance was able to ?nd things other tools could not ?because of the unique way that Cylance functions and operates. It doesn't utilize a standard signature of heuristics or indicators, like normal signatures in the past have been done, it utilizes a unique proprietary methodf??j On April 13, 2015, one day after deploying Protect, 0PM rapidly escalated its use throughout the enterprise. McClure wrote: checked in on the deployment and we are at 2226 devices at last count. Tons of ?ndings. Chris is working through them already quarantining. It is juicy??l McClure testi?ed: ?[Wle were ?nding a ton of malicious attacks on -- on the boxes that we were getting deployed to??-Jr On April 13, however, 0PM was not yet utilising Protect?s full capability- The agency was using the product in ?alert" mode and not ?auto quarantine" mode?ug Agency personnel therefore had to determine what should be stopped from operating in environment after reviewing alerts. When McClure stated in the April [Sm email that ?Chris is working through this statement describes the steps that must be taken to evaluate each item 0PM was alerted to before agency personnel could then consciously address them extracted from the environment, white listed, etc). McClure testi?ed that only about ten percent of Cylanee?s customers use the alert-only mode and in alert?only mode, the product ?will alert only when an attack is present or happening in the systemi?m Wagner testi?ed that 0PM was running Protect in ?passive mode, because we didn?t want the tool to automatically end up deleting forensic evidence that we needed.?m That is not how Protect works. McClure testi?ed: ?[W1hen we quarantine a ?le, we don?t actually delete it yet. The rationale is, if we quarantine something by mistake, that?s a false positive. In that rare instance, the customer would want to unquarantine it to put it back in production. So we keep it in a secure, untamperable space on disk that allows us to perform that unquarantining. Unfortunately, that does take up Space as part of the quarantine area??1 Protect identi?ed 39 ?Trojans" on various parts of network that were rated a negative one on the Cylance rating scale?the worst possible rating?and Cylance staff recommended quarantining these items.5 [2 The ?nding of 39 Trojans was signi?cant because as Coulter testi?ed the ?Trojan?s? functionality allows the attacker to ?bypass to some degree 5? Wagner Tr. at 32-33. 5'35 Wagner Tr. at in}. Ex. 3. if?? McClure Tr. at 25. ?m McClure Tr., Ex. 3. McClure at tel 1. 5m Wagner Tr. at 94. 5? McClure Tr. at 71. Coulter TL, Ex. 4. 102 security controls and allow a bad actor, in some cases, unrestricted access to a Coulter stated: ?Any one Trojan could have that capability."m in fact, when reviewing the work ticket that identi?ed these 39 Trojans, Coulter testi?ed: ?To say it bluntly, [Protect] lit up like a Christmas tree??5 According to lCoulter, Cylance?s team concluded these were downloader ?les, which are typically associated with malware and multiple Troj ans.516 When asked these results caused concern, Coultcr stated: ?Having gone through security clearance process many times, I know what 0PM does. And dealing with APT almost on a daily basis, you put two and two together. You can just assume the risk that, you know, what could unfold or what could he there."5? It quickly became clear to Cylance that the l?l? security situation at 0PM was dire. 5 '3 By April 19, 2015 malicious items continued to he found in enterprise. From: Chris Coultei Sent: Sunday. April 19. 1015 [13:49 Alt-I To: Stuart Subject: 0PM They are ?icked hrw. .. their threusic guys through some analysis and I pointed them to an rar archive of some bad stuff. Stu can we use Brians rig to crack thenl'.? Not seeing the contract: that would give us the password easily Chris Conlter I Consulting Dircctor . - In an April 19 email, Coulter reported to McClure that he had identi?ed ?an rar archive of some had stuf McClure told the Committee a ?le is ?a compressed archive of other ?les? that he recalled ?seeing evidence of an attack that had already been there, been successful, and it was nasty" and that ?[t]here were signs of exr?ltration of data, yes."519 In order to address the rar archive" ?nding, Coultcr asked for assistance with another tool to help break the McClure testi?ed: [W]hen forensic folks like us get on-site and take a look at these things, we can?t easily open them and see what they?ve been able to steal and push out of the environment without using something like a GPU [Graphics Processing Unit] password?cracking rig, which is what?s ?3 Coulter Tr. at so. 5? Coulter Tr. at so. 5? Coulter Tr. at sit?21. lCoultcr Tr. at 26-21. 5? Coulter Tr. at 2] . f? McClure Tr_, Ex. 9; Coultcr TL, Ex. 5. ?9 McClure at 103 referenced here. . . So he?s saying, you know, I?m not seeing the common BAT or VHS files that would give us the passwords easily. Sn typically, BAT is short for batch files, and they are Windows batch ?les. And VBS is short for visual basic scripting or script, both of which help automate certain commands that are run on a computer system. And oftentimes, because hackers are lazy, they?ll put into the batch or the VBS scripts, the actual hard-putted password of the RAE, so that they can help automate hoth and of it in their tasks.m On April 19, the signs of a signi?cant compromise at 0PM were clear. Coulter testi?ed: They?re in a severe situation. . . . It?s an incident now. It?s much more than just a malware incident. So when I was talking earlier about, you know, credential dumping tools and overt actions, this is again another overt action. If you don?t usually -- if you can?t explain why you have a large RAR archive in a location that most administrators would recognize, there?s -- it?s likely a stash of somethingcommon in a lot of APT cases, or actually a lot of breaches, if their end goal is to collect data, then they?re going to search for it and bring it back to a central point for aggregation. A lot of times data, like this email, it? you were to compress it, it would be, you know, potentially one-100111 of the size. So RAR, which is a compression format, is used to shrink data. You can also then apply a password cases, Where there is data ex?ltration or a breach, it?s very common to find tgese compressed, stashes of whatever bad guys were 5 after. Like McClure, Coulter also testified that, as of April 19, 2015, a signi?cant chance existed that data from 0PM had been eafiltratcd.523 analysis validated their concerns. According to Analysis of the image revealed that several variants of Plugx once resided on the victim machine, with the last variant from downloaded folder RAF. SFX2 still residing. Several password protected EAR files were found on the victim machine which have been identi?ed by the customer as ex?lh'ated data.524 5.2a McClure Tr. at 2123. 53' Ccullcr Tr. at 25-25. 513 {Toulter at 2154?. 523 Coulter at 2T. 51" June a, set 5 Baths at nocaarm-ooi 1515. 104 The RAE ?les that had been identi?ed were notable because these ?les were ultimately linked to the data exfiltration of the background investigation and ?ngerprint data and personnel records. For example, RAR SFXE appears to contain FTS data held on the attackers? primary foothold? WDC?new-postcom?? Another, RAF. SFXZ, when downloaded created the ?MoAfeeS?fo? folder in a deems located on a key Wcmw? SQL em.? and its duplicate server This location gave attackers access to a key jump box that facilitated access to other segments of environmentmsegments that house sensitive information.m found the attacker was active on that server stating: ?the first appearance by the actor that was observed on the victim images was on 5!??!2014 at from a SQL Server."513 analysis of this string of malicious .. . activity would later point out the liability to the country: ?It is interesting to note the machine had an [remote desktop protocol] session with [United States Govemmen! system_ on mammar? In other words, was pointing out a remote desictOp session that occurred in October 2014 on the system that led to a tunnel (Interior Business Center) at the Department . of Interior (D01) and to the federal employee personnel . records that were stolen. US-CERT and 0PM would later af?rm that the attacker pivoted to the data center at DUI in October 2014, with the personnel records subsequently being exfiltrated in December 2014.530 In an exchange with Rep. Robin Keily (IL), DOl?s CID, Sylvia Burns would later testify before the Committee about how the attacker traversed onto network and stole the personnel records: Ms. KELLY. Thank you, Mr. Chairman. Ms. Burns, the two data breaches 0PM recently reported have been particularly concerning to us because of the national security risk involved. According to testimony you ?35 June a, 2015 BM at Hooaocousz-ss. ?5 us. Dep?t of Homeland Security?JS-CERT, Digital Media Analysis Report-465355 {June 9, Elli-15} at 000090 Production: Dec. 11, 2015]. 5" Saul sbury Tr. at ?3 June 9, acts DMAF. at 154. us. Dap't of Homeland SecuritnyS-CERT, Digital Media Analysis Report-465355 {June a, sols) at accuse S-CERT Production: Dec. 11, 2015}. an 0PM Cybersecurity Events Timelinc. 105 gave at a recent hearing on the OPM data breaches, the OPM personnel records that were compromised in one of those bleaches were hosted in the data center maintained by the Department of Interior. Did the cyber attackers who gained access to those records also gain access to the Interior Department data center? Ms. BURNS. So the adversary had access to our data center. It was exposed. There was no evidence based on the investigation that was led by DHS, and the FBI, there was no evidence that the adversary had compromised any other data aside from the OPM data. Ms. KELLY. Okay, so the same cyber intmder who breached personal data, which the Department of Interior hosted on its servers, also breached the defense?s ot? the Interior Department data center? Ms. BURNS. So this, the intrusion that you?re refen'ing to, was a sophisticated breach. And my understanding, based on assessment, was that the adversary exploited, compromised credentials on side to move laterally and gain access to the Department of Interior?s data center through a trusted connection between the two organizations- Ms- KELLY. So the cyber intruder, did they gain access it to data center through 0PM or was it the other way around? Ms. BURNS. The adversary gained access to [1301?s infrastructure through 0PM, as far as I understand. based on investigation. Ms. KELLY- In addition to hosting OPlvl?s personnel records, the Department hosts data from other agencies in its data center. is that correct? And, if so, which agencies? Ms. BURNS. Yes. Actually, the Department is a?tbe data center in question, the biggest customer of the data center is actually Interior. So it?s the Interior Business Center, what we call IBC- They?re a shared service provider, and they are the majority user of the data center. And we also host some applications for the Of?ce of the Secretary in the data center.53 I The same day RAE ?les were being discovered {April 19, 2015), Protect also identi?ed ?command shells."532 Command shells are signi?cant because they provide a means for the attacker to remotely control a victim machine. 011 April 19, 2015, McClure wrote to Coulter: 53' Cyberseeurr?ge The Department ofthe Interior: Hearing Be?n-s the Sabeomm. on burnt-marina Tech. and Sirbcomm. on Interior ofthe H. Comm. on Oversight d?c Gov'r Reform, 114th Cong. 21-22 (July IS, EDI ?2 McClure Tr. at 31; Email from Stuart McClure, Chief" Exec. Uf?eer, Cylance to Chris Couller, Managing Din, Cylanee (Apr. 15}, EDIE, 9d}! at CYLANCILUUEI 12 (Cylancc Production: Jan. l?id]. 106 ?They quarantined one of the medexe ?les but I found two more. Might want to recommend they quarantine those too.?533 McClure explained the signi?cance of finding ?med.exe files:" A. Sure. So XCMD -- so CMD stands for command, and they usually stand for conunand shells. And what that allows you to do is actually have remote access of their computer on your own computer. So when you start XCMD on the victim has, it will then create a shell to you on your remote computer, wherever you are in the world, and you can then type commands as if you are sitting right there on the computer. Q. And why did you recommend quarantining another two mentioned in the message? A. Because that?s that?s as nasty as you can get. I mean, they can do anything that they want with that access??34 Cylancc and 0PM made additional ?ndings about the breach on April 19, 20] 5.535 Then on April 20, 2015, a Cylancc expert contacted Coulter about 0PM data collected and a ?backdoor." Thus, began a chain of events eventually leading to the discovery background investigation data had been stolen. Speci?cally, the Cylance expert wrote to Coulter: Give me a call when you have some time. l?rn going through the data now. Wanted to ask some questions about the system WCE was sitting on and a few others. You may want to have them get an image of is a backdoor that looks like the [command and control server] was active around $2014 corresponding to when they came out and said they had a problem. Callback was to resolved to if they have any kind of network or DNS logs going back that fans? This communication in particular would start the process of revealing how the background investigation materials were compromised. More evidence would unfold and become clear in the coming days. 5? McClure Tr. at 29; Email from reituart McClure, Chief Exec. Of?cer, Cylance to Chris Coulier, Managing Dir. of Incident, Cylancc (Apr- 19, 5, 9:9] at {Cylance Production: Jan. 27, Ellie). McClure Tr. at 29?30. 535 The same day that Cylance identi?ed RAR ?les and was working to decode the passwords, Protect found ?a fraudulent attempt at making this look like a BitliI signed binary. See the signed by ?Bit?? And [website 1 ii'irns Total] ealls it quite evil.? McClure 'l'ranscrihed interview, Ex. 1'v?irns'l'otal, a subsidiary of?ooglc, is a free online service that analyzes ?les and URLs enabling the identi?cation of viruses, wonns, Trojans and other kinds of malicious content detected by antivirus engines and website scanners. About Virus Total, available at: ?36 Cottiter Lia. e. 10? The agency continued to expand its use of Protect through April 21, 2015. The tool was on 6,225 hosts and it was expected to roll out to 10,000 hosts soon thereafteer On April 21, Cylance also identi?ed two Trojans sitting on key servers?? Fran: Chris Coulter Sent: ruesda rt] 3] 2015 12:51 AM To: c: Subject: 10:5 for 0PM gen Gross flagged these. please make sure they are tagged correctly as malware Trojan: callback to Team? - At thatpoint, 0PM also began utilizing more outside help. CyTeeh's Enter rise was installed on the servers where Coulter had identi?ed new pieces of Trojan malware.53 then imaged malware and artifacts residing on these servers that were subsequently supplied to Those ?ndings were covered in US-C May 4, 2015 Digital Media Analysis Report? and June 9, 2015 ?Digital Media Analysis Report.? Cylance also discovered remnants of malware used by adversaries in the 2014 intrusion against 0PM. CylanceProtect found ?dormant? variants of Hikit, which was the primary malware used by the attackers discovered in 2014, on systems during the discovery phase ofthc 2015 investigation. Jeff Wagner, Director of IT Security Operations, stated Cylanee. ?In doing a full analysis of the entire ?nd an older version oinkit. It also found library fragment ?les of malware":5 41 Wagner testi?ed regarding the Hikit malware found by Cylance and its relevance to the 2015 intrusion: A. So the llikit variant discovered in 2015 was not an active piece of malware, it was a dormant piece of malware. That because Cylanee was utilized to analyse the entire environment, we discovered the malware was dormant within one of the servers. It was believed to have been an abandoned piece of malware that was previously installed at some other time. Q. Was it related to the incident in 2015? 5'37 McClure Ex. 1 l. Coulter Tr., Ex. 539 Brie?ng by 1.1.3. Df?ce ofi?crs. Mgmt. to 11. Comm. on Oversight St Gov?t Reform 13, 2016}. ?u 1.1.5. Dep?t of Homeland Security?lS?CERT, Preliminary Digital Media Analysis Report - 1NC405355-A {May 4, 2015). at Production: Doe. I 1, 2015], Brie?ng by 1.1.3. Of?ce ofPers. Mgmt. to H. Comm. on Oversight r'l?r Gov?l Reform Staff[Apr. ill, 2016}. 54' Wagner Tr. at 120 108 A. We don?t have direct evidence it was necessarily related to the 2015 incident. it was discovered in the 2015 incident. sisal-e Q. Sorry. So did you have any indirect evidence that the [Hikit] Found referenced in the 2015 DMAR was at all involved in the 2014 breach? A No. We don?t don't remember the exact, quote, ?born on date? of the malware, which shows the initial point of infection, but it was not during the HHS timefrarnc of adversary activity. So we really didn?t have a recognized idea as to when it showed up. It was one of those pieces of malware, as well as additional fragments of former malwarc that Cylancc identi?ed, and we proceeded to eliminate along with everything else.542 One of the two Trojans found on April 2] contained what called a ?unique?s'13 ?le named winrsvesdll, with a compile time of 5:34:4e EST on March 18, This file was a malicious Windows Dynamic Link Libraries ?le designed to run as a service. When running, the DLL allows a hacker to pass and execute cxecutahles and DLLs to a victim system at willm This ?rst "unique? Trojan file (winrsvesdll) contained a "plugin? framework that allowed it to import and load DLL ?les. described the file as follows: ?The DLL [which is identi?ed as a Hikit Remote Access Tool is unpacked and loaded into memory, while never being written to disk. During execution, this ELL will attempt to read a con?guration file in the same folder in which it was executed. This con?guration is expected to have the same name as the originally executed ?le, but with a .conf extension. In this case, the expected con?guration ?le is winrsvesconf. If this ?le is not found, the malwarc will create a con?guration file which contains its default con?guration?m The Cylance found on April 19 would reveal that the con?guration ?le contains the command and control location The con?guration ?le contains the configuration string ?5?13 54: Wagner Tr. at l34?135 us. Dep't of Homeland SecuritnySaCERT, Malware Analysis Report-4603513 {corrected} {April 24,2c15) at EUGENE-001065 (UPM Production: Dec. 22, 2015]. 54+ LLS. Dep?t of Homeland Securilyi'US?CERT, Preliminary Digital Media Analysis Report - mosesass?a {May 4, 2315}, at Production: Dec. 11, 2015}. LLS. Dcp?t of Homeland Malware Analysis Report?do?352?E {corrected} [April 24, 21315) at (0PM Production: Dec. 22, 2?15]. 5'15 U3. Dep?t of l-lorneland Securityi?US-CERT, Malwarc Analysis Report-460352-A (April 24, 2015] at QUE-CERT Production: Dec. 1 I, 2til5). U3. Dcp?t of Homeland Securityi'US-CERT, Malwarc Analysis {April 24, 2?15] al 12530] Elli-91 Production: Dec. 11, 2015). :43 June 2015 DMAR at [54 [This particular HiKit uses the some strong?" in the output con?guration ?le as found in DMAR 355 TB). 109 1* The second Trojan was located on a server? and was called According to this was a Dropper. Generic? TIC Hikit found to have resided on the victim machine since September 15, 2012 at This binary also pointed to the malicious domain?"550 The cybersccurity event that was developing at 0PM was serious. It was not until April 22, 2015, however, that the agency noti?ed the Of?ce of the Inspector General that it was dealing with a breach.55 in fact, the notification occurred entirely by accidentm And while the Protect deployment was successfully identifying critical malicious items, the product was still being introduced into system conservatively. Protect was in Alert mode meaning threats were not automatically In addition, Protect was not yet on all 0PM hosts. (in April 23, 2015, Coultcr emailed an 0PM of?cial: ?Just letting you know we do not have Protect on the following key hosts On April 24, 2015, 0PM upgraded Protect to autouquarantine mode. At 4:1 1 pm. on April 24, Coulter emailed several colleagues to announce the upgrade. He wrote: Guys 0PM hit critical mass today and is burning the house - literallyl They just hit ?glohal-quarantine? for every threat! I think it was around 1180 threats in the queue. This was done per senior orders. They are also pulling the power on every device starting Saturday at 9am Sunday at 5pm. I need everyone?s help to make sure what they quarantined will not he mission critical ?les. I have been up for 24 hours so 1 really do need help.555 on Apr 21315. at 4:11 PM. C?lu'is E'oulter wrote: 3? Guys - 0PM hit critical mass todayr and is binning the house - literally! 9? They just hit "global?quarantine? for every threat! 1 think it was around 1133 tlu'eats in the queue. This was done per senior orders. They are also pulling the power on every device starting Saturday at 9am - Stuiday at 5pm. 3? I need everyone's help to make sure what they quarantined will not be mission critical ?les. I have been up for 2-1 hours so I really do need help. l'n- June 9, acts DMAR at :13. It: 55' DIG Mellie, Serious Concerns. 1? See Tri?e, Chapter Ophelia C10 and its Federal Watchdog. ?3 McClure It. at 33. 55" Coulter Tr., Ex. 3. 555 McClure Tr., Ex. 12. Hill Prior to April 24, 0PM manually considered whether each item that Protect ?agged should he removed from the system. McClure testified: My recollection was was] processing all the alerts themselves, along with the help of us at Cylance, our alert management team, as well as Chris Coulter, myself and others, to help them triage and process the alerts to make sure that they are malicious and not safe, and just trying to empower 0PM themselves to make the judgment call on whether to quarantine those files and move them out of Thus, while Protect was operating in alert mode, the burden was on 0PM staff to determine what ?les should be quarantined, or he allowed to remain operational in environment. ure testi?ed: Q. Can you de?ne, when you said that 0PM was processing things on their owa, can you de?ne A. Yes. They were in our management console looking at each alert trying to understand if they should actually quarantine it, delete it, or just allow it to continue to he on the system and study it for whatever purpose. Q. So 0PM was making the decision on what to delete out of the items identi?ed prior to April 24th, 2015?? A. Correct. All customers manage their own quarantinessi?l Saulsbury, who was on site at 0PM on April 24, 2015, provided similar testimony: So after we observed that Cylance was able to detect the APT rnalware, in this case it was, in the 2015 incident it as a malwarc family called Plng. And once we were able to determine that was able to detect Plng, at some point there was a decision made to deploy the Protect agent to all of machines. So that was done with the assistance of the vendor of lCylance. And so the guy that I am emailing on that is Chris Couitcr. So Chris was really good about helping us getting Protect deployed throughout the environment and then also analyzing all the findings that it is coming back with. So Cylancc is detecting not just the APT malwarc, but every type of malicious, like, adware toolbar that somebody downloads and things like that, as well as the false positives here and there. ?ll McClure Tr. at 34-35. 5? McClure Tr. at 35-35. 111 So Chris was really good about helping us triage through that list and separate what we want to quarantine versus what is false positive and whitelisted. So at a certain point we were con?dent enough that we had identi?ed all of the malware and had whitelisted the business critical applications that needed to be whitelisted. And so .IelT instructed us to quarantine all of the identi?ed ?ndings. What that quarantine means is, so when Cylanee detects something, we just had it in alert mode. So it would see it and say, hey, this is bad, but it is just alerting us on it, it is not actually doing anything about it. So what we essentially did on April 24th was press a button in the Cylance console and says everything that you?ve seen that is bad, take that and quarantine it so it is not operable on the machine.? Wagner also continued that 0PM quarantined all the identi?ed malware on or about April 24, 2015. With respect to why the quarantine did not happen before Apri124, 2015, Wagner stated: So once you identify maiware functionality or adversary activity, you try to get a sense of the adversary?s intention, activities, and exposure. You look to see how deep they are in the environment. So once you discover something on the l5l we didn?t want tojust start shutting things off. We didn?t understand the depth in which the adversary had been in the environment- With the deployment of the Cylanee tool, a full accountability of all binaries, we had discovered, identi?ed, and all the malware was placed into the uarantinc queue by I think it was the 19th of April . . . . And by the 241 we had a full understanding that it had discovered everything that was to he discovered, and we no longer necessarily needed the adversary to have an active presence within the environment. So we ordered Cylance to destroy the malware.559 The auto-quarantine did not apply to all of systems, however. For certain systems, 0PM made a value udgment as to whether they should be included in the auto-quarantine, or remain subject to the human command quarantine in auto-alert mode. Coulter provided guidance to his colleagues at Cylance on April 24, 2015 regarding what ?les to quarantine. He wrote: I would say anything on desktops are ok to quarantine. Servers should be the only thing questioned at this point. If they can live without it keep it blocked. They are sening up some help desk protocols to identify issues that come out of this. Mission critical items that Iknovv of: 551-: Saulsbuly Tr. at T233. 559 Wagner Tr. at 121-122. 112 USA JOBS related apps - they said if we bring that down senators will come for us LAN Desk at SCCM SQUOracte components and connectors to mainframes Past that they can live without for a few weeks. This is a desperate move, tomorrow is even more desperate by unplugging every device and moving over to new networks. They will blame any issues on the power outage 3.550 McClure testi?ed that in auto-quarantine mode, mission?critical items may stay in ?alert? mode so as not to unden'nine the system in the event of a false positive?? McClure also testi?ed that 0PM should have considered shutting down mission-critical items given the severity of what Cylance was ?nding. He testified, ?Yes, they should he.?5m Documents and testimony show 0PM used Protect as its quarantine tool and that Protect was not put into auto-quarantine mode until April 2.4, 2015. Documents and testimony also show some 0PM systems were not placed into auto?quarantine mode at all. Contrary to this evidence, leadership testi?ed before the Committee in June 2015 that the quarantine was hilly in place by an earlier date, and stated that the malware was ?latent" and merely being ohservedfll33 The term ?latent" means the malware is not active on the environment?it is frozen or othenvise not running on active computer processes. The quarantine status was not activated until April 24, 2015 when 0PM gave Cylance the authority to place Protect into auto-quarantine mode.5 Unless Protect is in ?auto-quarantine" mode, malicious items are not latent?wan action is required to stop malicious items ?om functioning in the environment??j According to Wagner, in the days that followed the deployment of Proteet?s auto? quarantine function, 0PM had ?discovered everything that was to be but significant discoveries continued. The new discoveries were noteworthy because they provided evidence related to the loss of background investigation materials. On April 26, 2015, Coulter and Jonathan 'l?onda (an contractor at the time in 0PM IT Security Operations) engaged in an entail exchange about a segment of the OPM networks? This was the same segment that a Cylance expert asked Coulter to image on April 20 writing: ?Give me a call when you have some time. l?rn going through the data now. Wanted to ask some 55? Eculter TL, Ex. 55' McClure at {?53 McClure '11. at as. 3? Hearing on 0PM Dem Brooch: For: at 69; see In??o, Chapter 5: The CyTeeh Story for more on quarantine statements by 0PM of?cials before the Committee. 5? McClure TL, Ea. 12; Conller Tr. at Til?7'5. 5? McClure Tr. at 34-36; Coulter Tr. at 34?36. 5515 Wagner Tr, at 121?122. 5? Coulter Tr. Ex. 13. H3 questions about the system WCE was sitting on and a few others. You may want to have them get an image of is a backdoor that looks like the [command and control server] was active around @2014 corresponding to when they came out and said they had a problem. Callback was to resolved to if they have any kind of network or [Domain Name System] logs going back that farfd?? In this April 26 email exchange between Coulter and Tonda, Coulter was investigating a Remote Desktop Protocol session that dated back to June 20. 2014 and accessed a particular segment environment. Conlter asked Tonda what was hosted on the segment Couiter was investigating?? Tonda responded the segment Cylance identified was where . . lot of im ortant and sensitive servers supporting our background investigation processes are located." 0 This was an important development because this server provided access to the PIPS mainframe where background investigation data was stored. 71 CERTIOPM would later confirm the ??rst known adversarial access to mainframe? as occurring June 23, 2014.in 5? Coulter Tr., Ex. 5. sea Coulter Tr. Ex. 13. M. 5? Conlter explained in the email that the segment he had identi?ed was a key ?jump box? at 0PM identi?ed as -- means a server that manages access between two different network sections of the larger information technology environment [Saulsbuiy Tr. at ?ta-T6]. r'u 0PM, this particularjumpbox enabled access to various parts of the 0PM environment {Saulshury Tr. at T4?Tti} and Cylance's Coulter was letting 0PM know on April 26 that the jumpbos had a Remote Desktop Protocol session to a signi?cant server that gave access to the portion of network where background investigations are stored [Conlter Tn, Ex. 13). Brie?ng by LIE-CERT to H. Comm. on lEnter-sight d: Gov?t Reform Stafftlieb. 19, 0PM Cybersecurity Evans Ti meline. 114 To: Ehr1s