U.S. Department of Justice United States Attorney Southern District of New York 86 Chambers Street New York, New York 10007 August 11, 2015 By Electronic Mail David E. McCraw, Esq. Jeremy A. Kutner, Esq. The New York Times Company 620 Eighth Avenue New York, NY 10018 E-mail: mccrad@nytimes.com jeremy.kutner@nytimes.com Re: The New York Times Co. and Charlie Savage v. National Security Agency, 15 Civ. 2383 (KBF) Dear David and Jeremy: This Office represents the National Security Agency (“NSA”), the defendant in the above-referenced matter. Pursuant to the Scheduling Order, dated May 15, 2015, NSA has completed its review and processing of the attached documents. NSA is releasing 16 documents with redactions. Information has been redacted from these documents pursuant to 5 U.S.C. §§ 552(b)(1), (b)(3), and (b)(6). Each redacted document being released has been marked with the applicable FOIA exemption or exemptions. If you have any questions, please do not hesitate to contact us. Sincerely, PREET BHARARA United States Attorney for the Southern District of New York By: Enclosures /s/ John Clopper JOHN D. CLOPPER ANDREW E. KRAUSE Assistant United States Attorneys Telephone: (212) 637-2716/2769 Facsimile: (212) 637-0033 E-mail: john.clopper@usdoj.gov andrew.krause@usdoj.gov DOCID: REF ID: A417724:9 4230.241 TOP SECi:tETtlCOldlNT-STE.LLARlVIN'DllORC0.7N~1'lOJ!OR1V!fld.R !Furl her dissemination of tbis i'eport outside the Office of the Inspector General, NSA is PIWHIBITED without the approval of the Inspector Ge11ernl. (TS//Sl//NF) REPORT ON THE ASSESSJ\1ENT C~F MANAGEIVJENT CONTROliS FOR IMPLEMENTING~ 'THE FOREIGN INTELLIGENCE SURVEILLANCE COURT ORDER: TELEPl-IONY BUSINESS lffiCORDS ST~06-0018 5 SEPTEMBER 2006 DERIVED fi'ROl'vl!: NSAJCSSM 1-5:2 DA'fED: 20041123 DECLASSIFY ON: MR ~pproved for Release by NSA on 08-06-2015. FOIA Case# 80120 (litigation) TOP BECR.E'fYiCOMINT~S'FELL...4RWll\lDfl-ORCOh~,l1tlOFO.RN1/AfR REF ID:A4177249 4230241 DOC ID nn ffl·~~1-1·<,'"iY•{l hv ]"ho Pirpt>t·c11· '\. '- J ... .1. i."-• ._ _.,_'- ,,... -.!.A'l..• .!....>'J ....... "-'. ""':; -!·l' l'-"-.": Of-1''ic·p rrf' t'Lw (-'~"''"'e"l'~t·l ((Hn) i.. ~.• ·' .._, 1-,.., 111.::1v:;..-•t<•1·· - . ~l....·li.Jl ... ~ '\.. J.\ •-r.; l\J'~,.~ f''..~;:Q l t. :·:t.,if'hic:f ... ~,"-"-.ti .. ~, i.. ..1 •••\ •• \ , ".:'l-a·· 1.. ".,...,"t.1··~·~""..;:t·'.t•:>b'"',,,+-1' ')'""' <.:! c .IL-. '':1'!1.. 1r']·[·t···t-, .. 1" 'l·~·"'·~ar·tl ,...>:. ..... S "'UHi.:;.::i..~ G_.v -~! ...... u::;. H>'.'l}'t<,..,1.,i_t,, -:, ~11.~,, «H 0 ·a·h·· .•i'\17s '' ;.r..:i .,, ,::·1·0I' . ·'.1· ::i •) ' 0 :''-"'.., ••.•• l<'\....... t.:1 !~1 ~ ··11ra .1..l'"\·e 1. ·o-f.~c~lr1·1"r\1 ... <..· L·.c . . J....... ~!' "··, • • ... , • •' ernc.1ency, ancr.J e. ff'_ecttveness ot 1-~c:;PJ..1 1.....0:-3 operations; to provwe mtcc1}'1ge:nce overs1gnt; to prntect against fraud, 'iYHste., and m.ismanagement of resnun::;e3; m~d to ensure that NSA/CSS activities are cow..ir~cted in compliance ·with the Constil:ntion, lrrws, e·,;·ecutive orclers, regulatj~~11s, a.!J.tl o•.+-1'~!f'.nC·>i•·· ~ ••\.\::;•'l... L ' ~ -·- .,... ,~· r''J"(j ..ll.~. ·- ..... , .... _. \..• ~ ~..... r .... ;>r1··t~1J·.;,-,,~ .,> ~ .J.\...•,, t.:~. il\.. 0 1 '1 ....r·~,:-q1J:::i1"1·,·,1"J>:" ,,,l,.•r111· ~v;t'1 .. , ;n) ::::<:.;:;.oq<:'·1-.·ir-:n+· -la;,rn~ <:~ 1 • .!. ..... -i...~. 4,.1:.u .... .._._..,_, ~- ........ t. nf .,_ (•f'••r1n}1·,..,.,...,1f'P ..... J"". ·t.t. (Li~_ .... , '\!Vl,·l-}·; .• .v~. '~1I1 . \.. r.:i.., .. 1.-, _ .,, •• _,_.,, "L·'hc.1 .tJ.,_. t:J..J.'l_,r recommendations for corrections or Jmprove1nents are subject to follcnvup. 1 ~fJ'.·'I)Pc··t;PT. J. ·~' •., .t ··' ,.J, u··fl:;r:.::.v •••• - ·.. ::.r1...... '15~(1 • .•. na1·t•1orq r- ... .#. t. .• c. .•. urit1·1 t't'Je Tl'1"pecto":.:.? t"V .a._.1::J • ·1. ~ (''"'11"1"'1] u'l,..._ c_ c_ ,...,f v._ t'l-1n S""l"';1'(':1'". v v. '-"""" J! •• • . .. -.L '-' The 1"J"lff}i'n10 2',f' '\...J ·J .t ...... 1.vt':.>t-···-· Elements to conduct joint inspections of th~:; corrnolidated cryptologic facilities. t;) provide an indspend~nt ,:issessrnerri: uf i--rnoTa'.1-)1'' ~·1Jc1.. C••'t)'::~n izat1'q·r1.c• ~>'['"('·'" 1p_.._(.·1t•-' 1·h·~ ,;;~';«.! 1'J...1'.r-,:~11cv of' •. -r.J ''-'· . JLf':J'.-.1.I....L 'U!'. ·"-'" l\:.vfnT''"..., .-. ...... J. ..,..._, ..... ll.J.•..!. \ .• ,_ ;;n1dit~ .. •. e·v·" V.• •,. C·r.rf1"'1..'Y1-l"V \.,. _.1,.; L.l . ·"-... \-i..l ,\.. ,..,•..... ·'-' (U) Tlie internal audit furn::.tirn.1 is designed J:!··'->C"'• ,,..~. an entity or program, a.s weU ns ~- V•'hr.~ther .~• .t. ) •- r. program ohjcetives are ·being rnet nnd 1 'pl.l··r-t·1~J-.,...•ri ""~'Y"" Lt • .. l "''ill}').l1"-l'l..:-1·",... ·i; ·1~.:..1, r··•i"'···ti,. ....~.l' ,-1c• fJ". r>·-'l(.'l.'.J ~l'':IJ't· 1 ,:l.-~t·)-''1'}1~1'j .. 1'.h•"'- •" """''ll'J~.- "'"_T (.•_-'-'U:L ..)11., ,,,.r_: '.-'J '. «l.i_.•='. "'" L•'- · "-·i~t:.~aL~ ..r ,~ . .i:kil6.Lrl 1-n; .L<.i:i.(.t.l.L!.t-· •t.~ .(.'\. lur.,·~·~i Li.J '·· l~_l. . ',=',i , •. ,,~... '<..ll AJJ audits are c.ond.ur.ted in accon.lo.nce 1.vith -L·h:.:>. 'r t)l'.'· (!·".:. Q-L· ":1·Lp..;: t(.... c·'C•"1·"rl·,.,.,il;:.,.I· ,,,J..1 ·!...... ~<(..•J.J: •..,. ("..c.ne· --~\.. .. 1·;.:1 ..... ... _J.~ iT•1~ted .,.\. J,. ~'\. i..1 t.. ~ ..... ,,, 1 ·~,-1 {'l'~ ..~ :::l-,·r ·."'..tS'.·=:.!.~. .••i.:.-1•1 •C4=..· 1 r)0·~ 1 ···-1 'J"r... '1.,...'..r1r::-_1• (._·,i1--r._.,: •.!)f1.·t·,,_'.1.·~r1J'•.·-~~.-f..,·.-.'.·1·.•.,~ ··:i "'°f.1i.::•·'1·P.1~...1.1 "1i"n.~~1·n,-.P~ . . ,1tti.J' (1 •,.. u i•_;o>.-•. ;. .... _ _ ,.,, ..:.l.'<«••·l:'J t. ..,tU .- '···lJ.fli.-' -;:. "··•t •.. 1.1. -i··,..,.l'J ,.,. u,;,.,(,; l .• __ . "' . ~ - ..,..... 'r..,. t ·-1~ -i:-·tf' (_1i.l• { ; ..... ,-.h··c~ t,.... r· .. J•.< i..uJ,S \.-U.<-!I_~"!<:l. l! .•:, r, '•J '11·· "l1°'".:""\ -·11 .... ,j[J. \. !J.L!.1U co'.> f·'. ...- c .,) 1.1p,•. ·t· 'l J.n111t· .• t •. 1f-'~~"·11 ..<.:., ;....i,, .,;'\·(••'.'L,, '7•:l C..t.1..l <., I11vesti~gat~£;ns ar~d ".':'1-·~('3 'I'~ 1'., .... ,. r . .....,\., ·~·-·1·~. · r"i <..ti:;'.• lr.. S1;;ccia1 I1!c111irics .n1;~y~ he u.Ddertr:d\en as a res-ult \Jf s11cl1 reccuests or ;::ompiaint.s; at the r~~quest of n1anageu11..:mt; a.s the resnh d b--regnlarihes that Stt:rface 1h1)"i"l'J ::it ·ll10 nf·th'" ~-··· Tn<::n~>•'l'PT r:;.f"-f"""l.• ~ ~.a. t..t·J ·~·• t_. i11q-·· --...... µet··'·1(·,·n .. t. . . _.\;..n~· ""'lf1"'1, ~:t'-· "" ., r·i· ~"" ...... ·- _...frt1"ri<~fr'.P - ..... .t..V-...... ·.t-""·"""·-J- \._..,.11_~1,,...i ........ cnEATIVE •• IMAGiNG-5~198:31 i.: .... I iOHJ340 i_ 1.'\ ....... Ji. • \.,.. ... DOCID: OFFICE OF THE INSPECTOR GENERAL NATIONAL SECURITY AGENCY CENTRAL SECURITY SEHVICE 5 September 2006 IG- J. 0693-06 TO: DISTRlBlITION SUBJECT: (TS/ /SI//UF') Report on the Assessment of Management Controls for Implementing the Foreign Intemgence Surveillance Cour'£ (FISC} Order: Telephony Business Records (ST-06-0018)-ACTION MEMORAl\TDTJJ\1 1. {TS//Si//NF) 'This report summarizes fue results of our assessment of l\/Ianagemeni Controls for !n1plen1entLng the F'ISC Order: Telephony Business Records. The report incorporates management's response to the draft renort. ~- 2. (U/ /FOUG) .A..s required by NSA/CSS Policy 1··60, NSA/CSS OJ.,Tice of the Inspector General. actions on OIG audit recom.m.endatlons are suI~ject to monitming and fo11owup u.ntH completion. Consequently, we ask that you provide a wiitten status report concerning each planned corrective action categorized as "OPEN." The status report sh.ould provide sufficient information to shm:v tl1.at corrective actions have been completed. If a planned. action V\Till not be con1pleted by the 01iginal target completion date, please state the reason for the delay and give a revised target completion date. Status reports should be sent tol IAsS.~~!..ant Inspector General, at OPS 2B. Suite 6247, within 15 calendar days aftet·eachJarget completion date. · · .. .....:.:::::>-(b )(3)-P. L. 86-36 3. (U/ /FOUO) \Ve app:reci.ate the courtesy and. GoofJ~ration extended to the auditors thr01.1ghout the review. If vou ne ri danflcation or addition.al infonnation, please contact · A.ssistan.t Inspector General, on 963-2988 or via e--rnail at ,__ _____ Derived From: NSA./CSSM l-52 Dated: 20041 J.23 Dedassify On: M.R ' ....... .t: __., v; : }.. - . • DIS'TRIBU110N: DIR D/DiR SIGINT Dil:ector SID Program Manager for CT Special Projects, S Chief, SID O&C SSGl,I ······· ................... ········ ····· I········· SID Deputy Director for Custom.er Relationships SID Deputy Director for Analysis an.d Production Chief, S2l5 SID Deputy Director for Data Acquisition Chief. 8332 GC AGC(O) ij .(b)(3)-P.L. 86-: ST'..06-0018 #:XI:!'!! Ii /C'J;i' f .fil"';.&ll1;7!r. Ill ~il1.':~rl1"'~~1l1$.l~ll!..~'Tr {Tl$/ t .,;,1,lt j h4Fi "} &'11.~.;;;z;t1':~:Jl1i""~w;;n !ii CONTROtS fOR, Pd. ir•.H "'~~El l'l!!.!:::lli.Q~· . iir-- r.rr1l.t'.ianAu~~Vil~!i~ Ii I~QliPlEMEll\1TING 'il!'~MiTtr.'~ 1i1 "ili''l1'"1re!!r0.llif"'!lt' .~fr~ o-~ !!i&il i£'."m "l}."lll\\f.fr.::wrn Ill &, !full•""'!!!: lbt~ik.1¥.~!fr.;;;~\;lr;i.,.~ ~1Uhr\l, -w·rr.;;.;11:i,,.~n11l,..;lk 1"ElEPHOt~Y- THE FOREIGN ro~ H})T 'W'"""""') f'il!l'%f,"t,~trJi ~ Uri\.~ ~~~. · (,.p·!Ji~,u ..... !t'!!.~ B.USl]\HESS RE(;ORDScon(f.j (.."O/.!ect'f!:d. t..'fnlf,, if'it/!if?!#tl~,tet'b1~nii'Jl c0tlfea)f:d,.. wlli be .'$wlftly expa!t?ged and 1u;t :,Wad~'"' #iVaflai.Me· ftN' iJJ!!"Biil'jlSi:!~• .~1t.."fparate tbe- lJJMtl'Gotiiy to iilpprove l'luff: ta!tiatf.i t;MtNrie~~ ft'tt/.ii!F'lt 1 1.~he c@pa/Jiflty to CtfitndM'lt: querietS tJ'f n~etadata u11der f:!Iti! Order& 1 DOCID: JREF ID: A41 77249 4230.241 ST-06~0018 (3) condut..1: periodic .reconci!.iatit.m ofappn:ntetl ·;;elephone numbe;r,1'P B-With t:l'#e fogs or qaeriel!:I nunilberrs to verify ittuat only authorized q11eries have been .made under the On/er~ !l5//~l//f!:i'fCN//OC,li.Pj Thr:; On:!et·. The Order authorizes NSA. to collect and retain telephony metadata to protect against intem.ationa( ten-01is1n and to iroces!" 2r · ......· (b)(1) (b)(3)-P.L. 86-36 (b)(3)-50 USC 3024(i) ~·<:.sF. l" ·~ · To protectJJ.S, privacy rJ.ghts, fae Order states specifi.c terrr.u~~ ._3_1_1..,L_,. ._J-_.... ,StrieltufiiS° f' reganilng the Collection, prOCE'SSL."'1g, retention, l dfssemjnation, data seculity, and oversight of telephony metado.ta a...'1.d U.S. person inforr.nation obtained under the Order. To ensu.r{:: compliance 1.vith these terms and restrictions, the Order a.lso mandates ,L\.ge:n(y rnanagement to hnplement a se1ies of procedures to control the access to and use of the <..rchived data collected pursuanl to the Order. 1hese conh*ol procedures are dearly stated in the Order. Appendix B inch1des a sumn1ary of the key tenus of the Order and the related mandated control procedures. fD1 St:andards o~· 6ntemai Cc1nrl:rol. Internal control. or 1rn.magement control, comprises the plans, methods, and procedures used to meet mi.sslons, goals, Rnd olajectives. It provides :reasonable assur;;m.ce that an entity is eftectlve and efficient in its operations, reliable in its reporting, and compltant v;ifh applicable laws and regulati.ons. TI1e General Accounting Office's Stc:nd.a.rrLs for Inf"em.o.l Con.iro1. in the Federal Govemment November 1999 (t.1--ie Standards), presents the sfandaTds that defL.1e the rr.li:ntmum level of quality acceptable for mai.-i.agement control in government. NSA/CSS Polic;y 7-3. lntemoI Control Pro~;JroJn., advises that evaluaiions of internal control should consider the requiren1ents outlined by th.e Standards. The OlG uses th::: Standards as the basis agamst which management control is evsluated. (TSf /Sl{/it'!F) 0!l)CUirnented PrntllJ.::edmres are Needed to (iOil.fferr~ the .if:o~~e~J-~ut_ \l~.T~~e~!!~~~l-~-~-?~d~!:a ·-----··--··---·--···--------··----·-- nr. l'nr·,·;'rn"I ')!'OC'e"''1·r·"f" ...... ; i ; '':)( ' . 1,' 1n'IFJ J.4 -·\. . .,..t_._,,.~ .t -~ ..... 1r... c,_, r"c,r v~ f'Oll"ctlnrs ~~ G .0 L'..:•l·"pr1riP~• '-· v _ .... -.i.....,Y •Y1rt··.;,,··1·-_,-J··.-, .......... 1.:A."-'-:it.L-V. under the Order -;,vere not forrnal\y designed and are not dearly documented. .As a result. management controls do not provide reason.able assurance that NS.A will comply with the fol1o1.v:tr1g terms of the Order: ~'@/.'EHrWe did not assess the control:; over retention at this time as the Oder. a.Hows data i:o be retH!ned fm five years. 2 :DOCID: ST-06-0018 NSA may obtain telephony metadata, wb5ch includes comprehensive commm.1-ications, routing infonnati.on. including but not limited to session identifying :informaUon. tnmk identifier, and time and duration of a f'.ali. Telephony rnetadata does not :lndud.e the substai-i.tive content of any communications. or the name. address, or financial. information of a subscriber or cu.stomei-. (TS//3f//T enmn-iate unauthorized data from the incoming data stream. f,FG//SI//S11Nl/ /NF} With these proposed control procedures in place, the risk that .Agency personnel will mistakenly collect types of data that ar.e not auU101ized under the Order \Vill be minimized. P.Jthou§f).1 llie primary and secondary orders prohibit tbe providers from passing speci.fi.c types of data to NSA rrlistakes are possible. For exa.TI1ple. in responding to our request for inforrnation, Agency management discovered that NSA w-as obta.JruJ1g two cypes of data that may have been :in violation of the Order: a 16-digit credjt card number and name/partial name in. the record of Operat.or-asststed ml1s. (It should be noted that the name/partial name was not foe narrie of the subscriber from the provider's records: rather, a telephone operator entered naxne at the time of an Operator-assisted call) (Tei/SI//fITI.JV//T'JE} Jn the c..qse of the credit card number, OGC advised thRt. ln i.ts opinion, collecti.ng this dat'.1 ts not what the Court sought to protJ.bit in the Order; but recominended that H st..tll be suppressed on the incoming data t1ow if not needed for contact chaining purposes. In the case of the name or partial name, OG·C advised lliat, while not what it believed the Comi. was concerned about when it issued the Order, collecting this informatlon was not in keeping ·wtfu the Order's specific terms m1d that it should also be suppressed from the incoming dat.a flow. OGC indic..ated that H 1.vil.1 report these issues to the Court when it seeks renewal of the authorization. Agency management noted that these dat.a type..'".: were 3 4 DOCI:O: 4230.2~ ·' ",·:;,·--~·; ~'"',.'.', · ·:".;J.. '.~;r.,":"~,~;.~~~l;·~,,~?!·~, \:.-.,<~J.~ :V,"}.ffr")T"j'~.'/t;,n.;· ,.J.Jfl l.i.\. .,,• ..._..#.}:.. t._..;.;"\ •.--J.~ ...···"· ,• / V .•J,f-.... ~.11.J L .. ..t1••.....;J.• .t,!J.....:...;.,,r.•.i.-V' .;..,_ •-I.<:.'\ ..,.·..,,./..i.\J.1•"-'l\....'- ~••.r.P."\. ... ~//_! .. ST-06-0018 blocked from. foe analysts' view. Management also statefl that it '\Vill take inimediate steps to suppress the data from foe incoming data flow . These steps should be con1pleted by .July 31, 2006. ~,¥1Si;~~{,~~;~:£~!lfi;:::,·.··,,~]:1~,i~:!I~''.~"~~"·· -;,~~I ' (TSNS!) Oe$ig~~ arr.d dtJcurnu~i'lli pro©edtn'es to provide a highew ~eve! of assurance theit r~oil"M~omp!iant data ·wm not be collected and, if ~ru:l!dv~wte~Uy collected, wm be swUUy expm1gsd ~nd not made avaUab~e for an~iys~$, II ! ! L__________________ (A.CTION~ ·~---------·~:_::-=~ (b)(;)-P.L. 86-36 CONCUR. eIS//::I//G11~W/NF} Management concurred vrith the finding and recommendation and has already partially implemented the .recommended proceduxes to block the questionable data from the providers' incoming data.flow. A final system upgrade to block '(b)'(3)"'PJ,,~::~!?.~~~ the uesti.onable data from one remaining provider is scheduled for ....... ::::::::::: .. Testing is currently ongoing. Qi·~· " ·:· ...... •.:n.ci.tus, OPEN ...... Target Completim-l'fJ'ate:a..1_______. (U) OIG Comment [t.Jl Planned action meets the intent of the recorn.1.-nendatton. ff!ll /Sl//Nr) Additior~a~ C@;nt~"t1l~!$ are Needed to Gov~rn the !.!~ces~ing of' r~leph(m~tt_ft!~~~d-~--·---·------·------·-..----·""·--·--fT8/i'EI//PJFJ Agency n:IBnagement designed, and 1n some ways exceeded, tl1e series of control procedures over the processing of telephony met.ada.ta that were: n1andated by the Order: ho'ivc~rer. there are cmTe:n.tly no means to pno;vent an indi\r!dua.l who ls author:l.zed access the telephony rnetadat.a from querying, either by error or intent. a telephone number thal is not compHant "Ni.th tht" Order. Therefore, additlonal controls as intended by the Order. 5 DOCID: 42302"4.l,"~'"'· ''".''·' ., ..,,,, .. ,~.! . .,:~P?. ?~~~,~,?_7__?.~.~.>. / .J. \..l.il ?.,]-...( ..._,"" }....... , j '"~-~~ ..... _J_l•.J\ ):.~'I;~ l'~l . ..:.. ;.i.L.Ji<~.f· ' ') y • ST-06-0018 Internal Control in the Federal Government require th;.;rt key du1ies and responsibilities be di.vided among cllfterent people to reduce the risk of error or fraud. J.n pm·tk'ti.1ar, responsibilit!es for authori.z".1.ng transacUons should be separate from p:rocesstng and reconHng them. 1hi.s lack of segregation of duties increases t.'1e risk that SbHt Coordh1ators and the Chief and Deputy Chief of AA.D will approve and query, eJ.ther by enor or intent, telephone numbers that do not meet the terrns of the Order·. H1~~H Sep.m·a~e the ai1..~t!~011·ity cap,abmty 1¢'.D ccmdl~c·~ to ~lpprove metadata quer;es from the queries of metadata ILmde~ the Orde~-. (ACT~ON: Chlf~f~ t\dvam~ed Am~~ysis Oivis~o~) IL ____ ·------·-------------····-----·-l COMCUR. ff8 //Eit.//STL\V/UF} Management conc1.m-r~d vliU1 the Bnding but stated that it could not implement the recommendati.on because of constraints ill manpower and analytic expertise. As an ~lternative, management reconunended that SlD Oversight & Compliance {O&C) rouHnely review the audit logs of the Chi.ef .::i..nd Deputy Chief of the Advanced A:n..aJ~sls Division &"1.d Shift Coordin.ators to verify that their queries comply 1;i;7:1.t11 "t11e Order. This a1tem.ative would be developed rr1 conjunction with actions taken to adclress Recommendation 3 and is contingent on the apprnval of a pending request to SID management to detail Dqmputer progran1mers to the team. Ma..""la.gemeni: is also negofiaqng vvith O&C to accept the responsibility for conducting the :reci:n:rlm\:nded reconciliations. Status: · .. OP~N Target Completion Date: l._______....luuummm u .. mum mu. uU (6)(3)-P.L. 86-36 (U) OIG Comment fY3,.'/5l//GTflN//NF) J.\lthough not ideal, management's a.tte:rnattve recommendation to mnnitor audit 1og;s to detect enors v:.UL at a m:Lnimum, mitigate t.he risk o:f querying telephone numbers that do not :rneet the terms of the Orts, Mot3t of tbese controls had been in place prior to and aside from the tssuance of the Order. On!y the requtremr::nt that OGC periodiC<3iliy monitor indivi,duals wlth sccess to the archivi:: was desig,ned in response to the Order. Combined, these procedures .are adequate to provide reasonable assurance that Agency :management complies with the foHmN.lng terms oft.he Order: DIRNSA. shall establish mandatory procedures stri.ctiy to c::mh'Ol access to and use Qf the arcb.Ived metadata collected pursuant to tlJj.s Order. 8 ! li ,! l , Ji1 :OOCID: 42 302Al-1 ,_,.,... \~J-~·' 1.-•.-r., (.-., ... ,~ r··-, .;~:g, .. l:Q.~-~~l.J,'f.2. 4,S',, · , ·n r·~rl., .. e·> } :, ./ k :..ii_,_:. 1 j / '<....· \~·~ L I . .: .1 • .r...~,: :; .. .J/ / \: ..,....·.~ \. l_.·\.../ :J- V '-.. ..·-.•!' 1~J'.i\...! ;' / i ). 1 J.·-. ·t ,,,\. ':I' .J.. '·f Li .;; .'-:.-..1.•• t "- t· J' ; .( 'If ;. .i \ r) ST-06··001.8 (TS//CV/HF) Additionally, O&C plans to reconcile tl1e li.st of approved analysts ·with a list of authorized users to ensure only approved analysts have access to the metadata. (TS/18:M'AfF) fil1aruigenuini· C(:tr(!;trols Governing the O'fle.rsigf'd of Activities Conducted Pursuant' to tbe Order are Adequate (TO//CI//MF) J1.s mandated by the Order, Agency management designed plans to provide general oversight of actlvmes conducted pursuant to the Order. The Order states that, The NSi\. Inspector General, the NSA General Counsel, and the Signals Intelligence Directorate Ove.-rsight and Compliance Office shall pe1iodically review this progra1n.. S ili'cal11y, _,Arr ,;:in,;:io' · d-' :d; y"'f' . .rn '•mnr' ..:.;rcfr/ r'86 .i'Wj " 1 ,_pee. , "fienc-y m~~emen1 csjo~· g..1.eu the follov.ling· plans that are adequate to ens.urn compliance ·with the Order. fr~' ,.:i ~· '> 0 (TS//91//IJF} 11:1e OGC will report on the operations of the prograrn for each renewal of the Order. (TG//Sf / /Mf?) O&C pl G'> verify that Agency man.-J.gement. has designed the control procedures mandated by t..~e Order. assess the adequacy of all management controls in accordance \vith the Stcmdards oflntemal Control in the Federal. Grmernmeni.. (U) Sccip® 3',Btnid ____ MethodoD@~y ,..;...,. . (U/ /t?CUGj-The audit was conducted. from May 24, 2006 to July 8, 2006. (U //FOUO} Vile interviewed Agency personnel and reviev.red documentation to satisfy th.e review objectives. (ffi//St} "\Tile did not conduct a full range of compliance and/or substan:tive tesurig that would allow us to draw conclusions on Ule efilcac-y of management controls. Our assessment was .lirx:dted to the overall adequacy ofmru.1agement controls, as directed by the Order. (T8//Cffl As footnoted, we did not assess contrnls related to the retention o:f telephony metadata pursuant to the Order. A.s the Order au.thorizes NSA to retain data. for up to five years, such controls would not be applicable at this time. (U) ~rrs@tr ~owera{l'l® -------------------------------- __ ____ __ ..., ,. ,. 1. (1J //f'OUO) Intmim Heport of the STELLA.t':\.WIND Program: Needjor Documentat:ion and Develc~pn-i.en.t qfKey Processes, 14 May 2004 2. Report of the STEU ARVVlND Program: Need Jar I11creasedAttention to Security-Related Aspects oft:he STELLARWIND Program, 13 September 2004· ~Interim 13 REF ID:A4177249 DOCID· -,.~ / / ~· ·i;•\;:";'1 ·:: •,,~,· .. .?.•. ::J. .i.'. \I .t ~-J:i .1. >· J i:"'4 ~· .::.:..1 .....;.i._d'"J_/f.>.. .1 ~J ~,.·~- M .,, ":""- .... f .i.• I . ' .... .._ .,..,. ,...,., ,,.~ ... "" /I ... ,I.. •. . J(./ \! ':t- lf ~ .....~.{'• ,)'Ji."... 'l;I / / .1.~:.~ f\. S'T-06-0018 This page intentionally 1eft blank l l ..... ~ ~· ... '· l ....... ..t - . ./ 14 ·"--~~ . ST-·06·-0018 (UllfGUO) Telephony B~1u~h1e$S Records F~SC OrdE~r .,, Mandated Terms and C@n~t"'om !Proce«h,1ire$ 15 DOC ID 4230;41, REF 1 my?. awqw-gr 1433,?, I u? . . x1itMI. Sup}. g; Lulu ?1 J. 2? Sr 1.1: v:er Mr. "um v; I page inte?tiuna?y lair. blank i ! ' DOCID: F ID:A4177249 ST-06-0018 (U) Business Records FISC Order (U) Mandated Terms and Control Procedures (TSHS f//~ ff) ...----·---..------------------· Control. Area Responsible ----+-NSA may obtain telephony metadata, which includes comprehensive communications routing infonna!ion, iuclnding but not limited to session identifying infonnation (e.g .. originating and terminating telephone number, communications device i.dentifier, etc.), tnmk identifier, and ti.me and duration of call. Telephony metadata does not include the substantive contt-nl of any communication, as defined by 18 USC 2510(8) or the name. address, or financial information of a __,_ subscriber or customer (pg. 2, para 2). Collection of Metadata L___ ----·-----------·--·----~ Terms of the Order Control Procedm·cs EnW:y OGC Al least twice every 90 days, OGC shall conduct random spot checks, consisting of an examination of a sample of call detail records obtained, lo ensure that NSA is receiving oniy data as authorized by the Comi and not receiving the rnbstantive cl1nt.ent oftl1c communications (pg. l 0, parH (4).1). J __ 17 DOCID: REF ID:A4177249 4230.241 : ' ·•• ' ..•• •, ! . ·, ! . " .. 'J. '·.fl\.'-.! ' ..:. '.~_' ST-06-0018 ! ' 1·-'1 l·+ ('fSNSL','fJF) -~--------------------------- Contro~ Terms of the Order Arca Processing (Search& Analysis, or Querying of Archived Metadata) Although data collected under this order will be broad, the use of that infommlion for analysis shall be suictly tailored lo identitying terrorist conununications and shall occur solely according to the procedLLrcs described in :he application (pg. 6, para (4)D). _ OGC shall review and approve proposed queries of archived metadata based on seed account numbers reasonably believed to be used by U.S. persons (pg. 6, para (4)C). Queries of archived data must be approved by one of seven persons: SID PM for CT Special Projects, the Chier or Deputy Chief-: Coumencmorism Advanced Analysis Division, or one of . th<:: four specially authorized CT Advnnct!d Analysis Shifi Coordinators in the Analysis and Production Directorate of SID {pg. 7, para (4)D). I ,....__ I ,. . . . · . . · I PM; Chief& I SID PM for CT Special Prnjects; Chief and Deputy ChieC CT D/C:hief of Advanced Analysis Division, and CT Advanced Analysis Shift A1\D, & Shift Coordinators shall establish appropriate management controls (e.g., records of all tasking decisions, audit and review Coordinators procedures) for a~ccss to the archived data (pg. S. para (4)G). I AAD Analysts Maintai11 a record ofjustifications because at least every ninety days, the Departmcnl of .Tllstice shall review a sample ofNSA' s justificarions fell' querying the archived data (pg. 8, para (4)E). (b (3)-P .L. 86-36 I A-telephone number believed to be used by a U.S> py1·son shall not be re rardecl as associated with ..__ _.,...,_____.·olely on the asis of activities that arc protected by the First Amemlrm:nt to the Constitution (pg. 5, para (4)A). When th<:: metadata archive is accessed, !ht: Ltser·s login, IP address, date and time, and retrieval request shall be automatically loggt:d for auditing capability (pg. 6, para (4 )C). and Technical Support OGC will monitor the fonctioning of this automatic logging capability(pg. 6, parn (4)C). OGC DIRNSA shall establish mandatory proceclLtres strictly to control nccess to and use of the archived dafa collected pursuant to this Order (pg. 5. para ~~----~---~~(4~)_A~)·-~- OGC D/Cbief of AAD, Shill Coordinators occur only after a particular known tele >hone number has been associal ct\ witl . ___,,.,.------' :v the standard NSA minimization procedures found in the Attorney General-approved guidelines (lJSSID 18) (pgs. 6-7, para (4)D) & pg. Chief of Infonuation Sharing Services in SID 8, para (4)G). Metadata Retention Metadatii collected under this Order may be kept online (that is, accessible for queries by cle.ared anal.y sls) for five years, at which time it shall be dt~SlrO yed (pg. 8, para (4)F). --------+·-Data Security ('J'Ji'/ OWrw+ Dl.RNSA shail establish mandatory proccf.thd dafabase, do prevent the data in question from ever being loaded into ....... ······the operational contact cbaiD.ing databases. Still, the data in question did exist in the ... ·· · / dataflow and should be suppressed on the provider-end as the OIG recommends. I (b)(3)-P.L 86-36 ·.. · ... ·· .. ·. I · - a. (T£//Sli4£STU.-Vh'l'ffi') Corrective Actions: Although already partia1iy imple:mente sound :md confom1s fully to the standards of internal controi in the Federal GovemmenL In practical terms, it is not something that can be easily implemented given the Derived From: NSA/CSSM J.• 52 Dated: 20041123 Declassify On: 2030 l l I 5 1'()11 8ECFtE1'//C()l\~.IlJT//STL?~1l//f'~OFORfc4'/'20JOl l :.s u (b)(3)-P.L. 8 IDOCID: I REF ID:A4177249 4.230.241 TOP SECRETh'COMIHT//3TL\iiirr'i9'sistanllnSpecfor lon·963-2988(s) or by. e..-mailat f:::,,,,. Generalf · . · · . (b)(3)-P.L. 86-36 4. (U) We appreciate the courtesy and cooperation extended to the audit team throughout the review. · Yr . eoJLcuA GEO~LARD Inspector General . . REF ID:A4197238 DOCID: 4230244 SECRET;)ccg},fINrI)~(''l0FOZl.7'l lOP DISTRIBUTION: ------------. D / GC(O) L...,.r-----------1 Chief, SV42,..___ ___.. Chief, S 12 .,___ _ _....__....___ __ Chief, S2I.,.:.4.....__ _ _ _ _ _ __...._;;.__ _ _ __ Chief/TD DoJ/NSDT--------:-~_-._-~'~\-------.:i, .....__ ...,...........-;;;..___. ___ ······· ········· ' (b)(6) cc: IG D/IG D13 D14 D 1 AIG for Follow-up Chief, SV Chief, Sl Chief, 821 Chief, 83 Chief 833 Chief, 8332 Chief, T12 Chief, T122 Chief, T1222 OGC IG POC ·. ' ' ' ~ ....... "\<:\:. // ··:xb)(3)-P.L. 86-36 ,/ /f ! i ' ' ' ' .. : ' ' / -----.J/ .-1 sm IG Pocl , f TDIGPOC~I-------------f1--~ TOF 1 SECRET1)'COMHff1) }~'0FORI'cl ii DOCID: JREF ID: A4197238 4230244 SECR£T//COivifl'\·, ?~\=<->· ·-::__-"-'> SUMMARY :·.,.·. :.:.:/:~ ·<::_;:.: \: ·. -. :.'.ff:S/5'$.b~'fJff:j'fhisY(epoitprovides the results and related findings }\\ ·. i;,fpl/q{te~t/hg•qtN$A controls to comply with the Foreign .· ;./:~;}. ~; •. ~Jbt~flig~i)q~:§iJl'y~i,ll~pce Qou(t (FISC) Order regarding Businf:jss ·.·.•o:.'~. ·'.·.B~go(q$'.·.(£3RJ;,.JJ!~$~resufts will be used to design test objectives· .~~I .i.• {g~'r[lpptf1JY't~$t!f}.g:frorrrJapuary to December 2010. NSA [/ ;;i¢6rfipl~e{\.'. '.appJiq~ti9b~?p~rt~t'outrnonthlytesting and make formal ··:;,,: ·°' j,t~r/c)lr.w.~~~Att0.f1$ ~$ necess[3ry. >·' · , . ·:- 0 _" •. O(~ i:. :~:::r.I ~'.\:::<·:;r :::/::::~- ·:.- .... (U) Background ('f8//5I//NF) The Business Records Order ('f'S//SI//NF) Pursuant to a series of Orders issued by the FISC beginning in May 2006, NSA has been receiving and analyzing certain call detail records or t~~~PP9PY metadatal....__ _ _ _ _ _ _ _ _ ___, ......""''·''"'''"' :•I l"NSAfefors. to the Orders collectively as the "BR Order" or "BR fb)t1}::::::·: :. FISA." (b)(~J~e:~~~··8.!;>-36 (b)(3k50 U~¢ 3024:(i) ········· ············ jTS/fsij/~fl?}TheBROrd~!' provides NSA access to bulk call detail . records that primarily include records ofteleph.qne calls between the Uh:ited States and abroad or wholl within the u:;:tl.ted States __ ..._ _, To access this data, NSA must conclude that, based on the factual and practical considerations of everyday life on which reasonable and prudent persons act, facts give rise to a reasonable articulable s t at n 'de ti er ·s This collection of information is not available to NSA through its other foreign intelligence information collection. It is valuable to NSA analysts tasked with identifying potential threats to the U.S. homeland and interest abroad by enhancing the analyst's ability to identify, prioritize, and track terrorist operatives and ,__--:'!'......,__,._,.,~-,....---.--.,..,.-,.,.--,! TOP SECRET//COMHIli') Authorized officials must revalidate RAS determinations of foreign and U.S. selectors within one year and 180 days, respectively; and (b)(3)-P.L. 86-36 hereC:]i11sfo;~~~~ ~f 1 We do not include non-compliance with dissemination rules that NSA had already reported in June 2009 and for which NSA had taken appropriate corrective action. TOP SECWT;) 1COM»rF;) ~VOHJRlJ 2 1 DOCID: REF ID: A4197.238 4230244 TOF SI3CRET;'/COJi.'iJJ:>.Jl;i;IJ'l0FOP.N • (TS//SI//'[}JF) Analysts who query time-restricted selectors must be made aware of the time period for which the RAS determination applies so that the information may be minimized. ('f'O//SI//WF) Specifically, the database is not designed to tag and track selectors associated with U.S. persons, revalidation dates, or timerestricted selectors as separate and distinct fields. Though analysts might include such information in the "comments" field, that field is not easily searchable or usable in designing controls. Separate fields are needed to integrate preventive control_i:;jntol and ensure the completeness of detective c9ntrols·;··such as weekly audits by the SIG INT Directorate (SID).Qffice-·of"Oversight and Compliance. As a result, NSA increa.i:;~s itsYiSk of non-compliance with these requirements, and the .-sc6f>"e of testing on selectors associated with U.S. persons and revalidation dates may be limited. 2 I ··················· f~l(3)•P-.L..§6-36 ITS//SI//nF) Management stated that the Homeland Security Analysis ·:::::::::...... .. ·· ... ··. · ·· ... _genter (HSAC / S214) and Technology Directorate representatives decided -. . ·'>'···· 1-i"ofto·corre..c::t control weaknesses because such modifications could take ·... "''=::,,, enough time s·o a:s··to.ov~r.:lap with the release of a new selector · · · · · . . ·.·.· .·.· ·::: management application lnow planned for May 2010. ··:-.... Initial demonstrations and requirements documents suggest that · . ··::::j lwill resolve these weaknesses. Not only will key data be ·. ·.. frapked to enable controls that will detect instances of non-compliance, ·... but coµtrols will also be added to prevent querying of selectors that have rtot beeit·woperly approved by OGC or revalidated within appropriate ·.. tim·ef{ames>l Ihas also been designed to track time-restricted selectcfri:;, and SIGINT managers are defining related requirements to 2-o,nfigurcH lwith appropriate preventive and detective controls.s called! ITS/J~_O//WF) Bec~use NSA recognize.s these weaknesses and plans to May 2010, we will not make formal release! recommendations to correct control deficiencies in the existing database. Time-restricted selectors are not currently a compliance risk because i:p,ap.J;i,ge.menLremovedallOtime-restricted selectors from querying as (bj(1")wmu . mm uumU they were determined to no longer have intelligence value. We suggest (b)(3)-P.L. 86-36 that management not reinstate time-restricted selectors until is in place anctJ can be copfi:g1;iE~d with appropriate preve.ntive and detective controls. · Im I I .I (b)(3)-P.L. 86-36 (TS//SI//WF) The HSAC/8214 is managing the risk associated with revalidations by implementing temporary manual processes to track revalidations and by using more stringent timeframes than those required by the Order. We will use the manual process to test revalidation dates as part of our monthly testing and will make formal recommendations as necessary. i(b)(3)-P.L. 86-36 '(TS/.'Slh'Mf) We did not incJ~1de time-restricted selectors in pilot testing because determining compliance with the requirement was too subjective to apply the continuous auditing methodology. 3 ('fSh'Sl/MF') I lls NSA's corporate contact chaining system used to store and analyze BR data. TOP 3 :OOCID: 4230244 T01 REF ID:A4197238 SECR:ET//COM1iv 17/NOFOitN (TSf/SI//PlF) ....1.\ft.~:r.. . NS.A.reportef;+,iIJ...,..,r+.:1i....,).:.P.~\.++1~04'1""*·0;+ffi"'"..ll..i"T_ This page intentionally left blank 'f(JP SECRET;)'COMLVIj)('>lOFOR1'l REF ID:A4197238 DOCID: 4230244 TOF SECR:ET//COMH'<11//1VOF011N (U) ABOUT THE AUDIT (U) Objectives ('fS//SI//NF) The objective of this audit is to test whether controls to ensure NSA compliance with key terms of the BR Order are operating as intended. Specifically, we tested the following objectives to determine NSA compliance and assess the feasibility and reasonableness of including these objectives in monthly testing: 1. (TSf/SI//l>TF) Only authorized individuals accessed BR data in December 2009. 2. (T8//8I/0IF) Selectors queried in December 2009 were documented as either approved to have met the RAS standard or were queried for data integrity purposes. 3. (T8//8I//W¥) Selectors queried in December 2009 that were associated with a U.S. person were documented as having been verified by OGC that RAS determinations were not based solely on activities protected by the First Amendment to the Constitution. 4. (T5//:SI//PrFt-Selectors queried in December 2009 were chained to no more than three hops. 5. (T£//8I//W¥) BR-related information disseminated outside NSA in serialized SIGINT reports during 2009 was approved by the Chief of Information Sharing Services or other authorized official. 6. (TS//SI//RF) Weekly Dissemination Reports issued in 2009 completely and accurately reported BR-related information that was disseminated in serialized SIG INT reports. (TS//SI//NF) Of the 58 NSA requirements in the BR Order signed on 16 December 2009, we decided to pilot test these six objectives because they were relatively stable, at risk for technical non-compliance or violation of privacy rights, and testable using the continuous auditing methodology. For a requirement to be testable, compliance must be clearly objective and verifiable by supporting data. (U) Scope (U//FOUO) We conducted pilot testing from January to March 2010. W.e.xe:v.iewed. the following-data: I Iaudit logs for December 2009, RAS approval dates from the BR FISA database, RAS-approved U.S. person selectors, and access lists maintained on the mh_fff)..// f?~h'N.F.) (b)(3)-P.L. 86-36 TOP SECRET//C~fFfff;'/fl DOCID: 4230244 TOF REF ID:A4197238 3ECRE 17/COM11\J 1//WOFO:RN This page intentionally left blank 'TOP SECttE'f;'/COMfllff'/JN fY!lOtt/'•l REF ID:A4197240 DOCID: 4230247 WP SECREWC02'~%'!-;F0f'ORN NATIONAL SECURITY AGENCY/CENTRAL SECURITY SERVICE INSPECTOR GENERAL REPORT (TSh'Slh'NF) Audit of NSA Controls to Com ply with the Foreign Intelligence Surveillance Court Order Regarding Business Records January to March 201 O Test Results (ST-10-0004A) 28 May 2010 ~pproved for Release by NSA on 08-06-2015. FOIA Case # 80120 (litigation) DOCID: 4230247 :JREF JI:D: A41:97240 'f'OP SECRErn'COMINTh'NOFOR:N (U) OFFICE OF THE INSPECTOR GENERAL (U) Chaitered by the Director, NSA/Chief, CSS, the Office of the Inspector General (OIG) conducts audits, investigations, and inspections. Its mission is to ensure the integrity, efficiency, and effectiveness ofNSA/CSS operations, provide intelligence oversight, protect against fraud, waste, and mismanagement of resources, and ensure that NSA/CSS activities are conducted in compliance with the law, executive orders, and regulations. The OIG also serves as ombudsman, assisting NSA/CSS employees, civilian and military. (U) AUDITS (U) The audit function provides independent assessments of programs and organizations. Perfonnance audits evaluate the effectiveness and efficiency of entities and programs and assesses whether program objectives are being met and whether operations comply with law and regulations. Financial audits determine the accuracy of an entity's financial statements. All audits are conducted in accordance with standards established by the Comptroller General of the United States. (U) INVESTIGATIONS AND SPECIAL INQUIRIES (U) The OIG administers a system for receiving and acting upon requests for assistance or complaints (including anonymous tips) about fraud, waste, and mismanagement. Investigations and Special Inquiries may be undertaken as a result of such requests, complaints, at the request of management, as the result of irregularities that surface during inspections and audits, or at the initiative of the Inspector General. (U) FIELD INSPECTIONS (U) The inspection function consists of organizational and functional reviews undertaken as part of the OIG 's annual plan or by management request. Inspections yield accurate, up-to-date information on the effectiveness and efficiency of entities and programs, along with an assessment of compliance with law and regulations. The Office of Field Inspections also partners with Inspectors General of the Service Cryptologic Elements to conduct joint inspections of consolidated cryptologic facilities. TOP SECRE1WCOA£R.J'I;$'.NOFOR..'\r DOCID: 4230247 REF ID:A4197240 'f'OP SECRE'f#COMThi7fi'NOf'ORN 'f'OP SECRE'l)J'COMIN't»'f\lOFORN REF ID:A4197240 DOCID: 4230247 TOP SECRET17'CO1vf1l,lT!75'VOf?ORltl OFFICE OF THE INSPECTOR GENERAL NATIONAL SECURITY AGENCY CENTRAL SECURITY SERVICE 01June2010 IG-11160-10 TO: DISTRIBUTION SUBJECT: (T£//SI//Rfr) Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records - January to March 2010 Test Results (ST-10-0004A) 1. (U / /POUO~ This report summarizes the results of our January, February, and March 2010 testing using the continuous auditing methodology. 2. (TI'//SI//NT') Audit Objectives We are conducting monthly testing of NSA controls to comply with the Foreign Intelligence Surveillance Court Business Records (BR) Order to determine whether these controls are operating as intended. 3. (TS//SI//HF) Pilot Testing The Pilot Test Report (IG-11154-10) was issued on 12 May 2010. It concluded that NSA had complied with the BR Order for the six pilot test objectives tested and within the time periods covered. Although no incidents of non-compliance were found, pilot testing disclosed weaknesses in controls over querying certain types of selectors, as well as concerns related to the dissemination of BR information. Weaknesses related to querying should be resolved with NSA's implementation of a new application to track BR selectors, which NSA now hopes to release in June 2010. We will monitor the situation as part of our monthly testing and make formal recommendations as needed. 4. (U / /FOUO) Monthly Test Objectives See Appendices A - C for details of January, February, and March 2010 monthly test results for the following objectives: ('fS//8f//NF) Access: Were all queries to the Business Records FISA BRF metadata made by authorized individuals (e.g., analysts and data integrity analysts)? (U / /FOUO} Reasonable Articulable Suspicion RAS Approval of Queried Selectors: Did all queries use RAS-approved seed selectors? TOP gJiCRETi!COM!l'lT:W\[OFOR.¥ 1 REF :ID:A4197240 DOCID: 4230247 TOF 8ECREIWCOlvf1N'f17'NO-FOR1{ ST-I 0-0004A • (U/ /FOUO? OGC Review of U.S. Person Selectors: Did OGC verify that RAS determinations of all queried seed selectors associated with U.S. persons had not been based solely on activities protected by the First Amendment to the Constitution? • (C//REL TO USA, FVEY) Oiaining: Were all queries chained to no more than three hops? (U //FOUO) Revalidation of Queried Selectors: Were all queried foreign and U.S.-person seed selectors revalidated within the Court's timeframes-one year and 180 days, respectively-and approved by an authorized Homeland Mission Coordinator (HMC)? (TS//51//Nli') Weekly Dissemination Reports: Did NSA accurately and completely report serialized dissemination of BRF metadata outside NSA? (TS//51//Jl>W) Dissemination of Serialized SIGINT Reports with BRF Metadata: Was all information disseminated through serialized SIGINT reports approved by the Chief of Information Sharing Services (S12) or one of the five other authorized individuals? 5. (U I fFOUO) Monthly Test Results We found no instanc:.es··o(g¢,h\b)(J)-P.L. compliance for the months of January and February 2010 ....Howe~er.~::::~tgNfic\;ant scope limitations remained for all months in testing.Office of Geµ¢f,:af:¢ouns~l (OGC) reviews of selectors associated with J,.LS:·j)ersons because·:()f..-66,titrol \ -'/e found\an weaknesses reported in the Pilot Te~t.Re···o~t. Also, expired selector marked as approved \1eried for foreign intelligence purposes:/ThelJ.S. person selector .1 I land. was due for revalidation ·based on the Court mandated 180-day requirement for revalidation of a.U.S.. person selector. This selector was then queried! .• / lafter·it had expired. The selector After being notified of this was changed to NOT APPROVED non-compliance, Special FISA, Oversight, Processing and Support (SV4) issued an incident report Tllis error w1derscores the control weakness in selector revalidations that we reported in the Pilot Test Report. It also raises a question regarding separation of duties when Data Integrity Analysts (DIAs) are able to query for data integrity and foreign intelligence purposes. We will make formal recommendations to address these findings in a separate report. and .. I I ./ I I TOP SECRET17'COlv11N'fl7"/tlOf1fYIVv' 2 86 36 - DOCID: 4'230247 REF ID:A4197240 ·'fOP s:Eci:E'Iwbo~·JiNtA#oFo:f?N ST-'l0-0004A 6. (U//f10U0) We appreciate the courtesy and cooperation extend~d to the auditors throu hout the review~ If you need clarification of additional information:, . please contact on.99.9..:~988(s) or vfa e~maitad o on952'.'.2171{~).?.:.. :.ia e.-mail'atl I< · .\ I ................................... :·... • ..... 1...... ············ ......... ;.,, . - - - - - - - - - - - - -....., Assistant Inspector General_ for Intelligence Oversight · ...• FOP SECR:ERS'CO:lrfl.i'611i~WOFO~-D.1'1 3 .. ···-·· . ............... . 1 ··. - ....... • . ·... .• • . \. ·;~f(~)-P.L. 86-36 REF ID:A4197240 DOCID: 4230247 TOP S£CRETWCOlvI1lfft7?vYJFORN DISTRIBUTION: D21 _____ '-r------------1 ...., Chief, Chief, Chief, '--------~--....;;....-S2I4~-------i..;.;.____;;_ _ _ __ Chief TD, ·······... ·· ... DoJ ··· .....::::···... ·.. .:· .:· .;:::· .::== .. .;:· T()P 8£CR£n?'COl'vf1Nn7'JvYJFOR1V 4 ST-10-0004A DOCID: 4230247 REF ID:A4197240 TO-P SECRETi?'CO-lJ1JtBS'NOF'ORN APPENDIX A (U) January 2010 Test Results DOCID: 4230247 REF ID:A4197240 TOP 8ECRE'fWCOlvfIlvTMvYJfi'tffi..N (U) This page intentionally left blank. TOP 8ECRETA'COltflJi.i'TiSWOf'0R}{ ST-10-0004A REF ID:A4197240 DOCID: 4230247 TOP SECRETtS'COM1lv71S'f)(1) (U) March 2010 Test Results _.......· (b)(3)-e.L 86 .35 ,, ,, . . ............ . (b)(.3)-SO·U.SC 3024(i) ('fi577SI77MF) Test results show that NSA complied w1t.h··:ri:J.ost requirements of the BR Order with noted sco e limitations for the time eriod 1-31 March 2010.' - ........................,.,.__,,.........,,.....,,.. ........................................................................... Append ix D of this report. Area -The rating definitions are included in Test Results 1. Access Scope Limitations Rating . None ············· ......... ........ ·. - . ·········... ··· ... . ·4.. _ch-~in1ng 5. Revalidati~ii of Queried Selectors None ·· ... of ine seed selectors ...q'""'u_e_n_e..,...,.o_r..,.oreign 1n elligence purposes were apprayed by authorized HM Cs within tt_itaurt's timeframes. The remaining seed selectors that were not RA .approved were queried for data iht~grity purposes. None 6. Weekly Dissemination Reports Weekly Dissemination Reports colp~etely and accurately reported the serialized SIGINT reports issue . · We decided to limit testing to serialized SIGINT reports because briefings, litigation, and other types of dissemination were not easily testable. 7. Dissemination of Serialized SIGINT Reports with BRF Metadata The Chief of Information Sharing Services approved t h e l l serialized SIGINT repo'irsis'Sued. Testing was limited to serialized SIGINT reports because briefings, litigation, and other types of dissemination were not easily testable. TOP 8ECRETWC°'-Y!LVTA'N· ......__ ..,._, OGCIGPOC,...._ . SIDIGPOC TDIGPoc·,..__ _.,..._____ _____ ____ "'-~ Dl D12.___ _ _ ~ D13 D14 TOP SECRETHCOMINT//~~OFORN" 3 DOCID: 4230248 TOfl REF ID:A4197245 1 SECi\:ET/1 COMIHT;'/i'q-Of oruq- (U) This page intentionally left blank TOP SECRET//COMINT//NOFORN 4 ST-10-0004 B REF ID:A4197245 DOCID: 4230248 5ECftEi17'COIY1IMi/JNOfOltN iOt' ST-10-0004B (U) April 2010 Test Results (TS// Oil/NF) Test results show that NSA complied with the requirements of the BR Order, with noted scope limitations, for the time period 1-30 April 2010. Area Scope Limitations Test Results Rating None 1. Access ··•''",., .. ...:z •. RP:$"'Ap,~rov(;ll of Queried········ ;;;;;:::::::;:;:;,,,;;:jd~§t~!'-t9r5 " "' ,,,, ;: :::::···· . "'' '_j"(b)(3)-P.L. 86-36 , ••• •• , •• •• '. _,f" .. None 4':..ChaiiiiQg 5. Reva.lioation of Queried Selectors 1 Th6' seed selectors queried for foreign intelligence·pµrposes were approved by authorized. IJ.M..Cs within the \Court's timeframes. TheLJseed s~lectors that were not RAS approved were queried for d~L-86~36 ···mm • (C//RBL 'PO USA, FVBY) A DIA was able to query an expired selector because controls were not in place to prevent such queries and the manual process that management had temporarily put in place did not identify the selector as needing revalidation. • (TS//SI//NF) DIAs can query BR metadata for both data integrity and foreign intelligence purposes, increasing the risk for non-compliance with the Order. (TS// SI/ (PJF) Management concurred with the recommendations in our audit report and completed one. Specifically, management released ........ , June 2010 and has verified that controls are now in place to address selector revalidations and the two remaining control weaknesses that we reported in the Pilot Test Report. Im TOP SECRET1/tCOMI1VT;)'NOFORH v DOCID: 4230249 REF ID:A4197247 1 TOf' SECI\ET;/ 1CClrfillT;/ 1"lOTGRll ST-10-0004C (U) This page intentionally left blank. 'f'O? SECR:ET//C(Xl!ff1\T//NOFOR.Jo{ vi DOCID: 4230249 REF ID:A4197247 TOP SECREii)''CO:tW:iNT/;/1VOTORi ef ST-10-0004C II. (U) BACKGROUND (TSf,)(J)-P.L. _ 86 36 (G//F"{gLTO U&.1., FVKY) Because! ltei~~~e date k~pt slipping, the risk for non-compliance remained for requirements relate(!. to U.S. persons, selector revalidations, and time-restricted selectors. However, Agency management reported on 28 June 2010 thatl lhad been released on 25 June 2010 and was operational. (U) RECOMMENDATION 1 {6)(3)~P~L .. 86•36··· ........ ff_§.1'(§._l(J'P.~..~) 1i:nmediately verifJ that con~rol~ in the newly released ve·rsmn·ofl _are function mg to: a. prevent querying selectors associated with U.S. persons without a documented Office of General Counsel review for First Amendment considerations; b. prevent querying selectors not revalidated within BRmandated limits (180 days and one year for U.S. and foreign selectors, respectively); i:md c. tag, track, and identify time-restricted selectors. (U) (ACTION: Homeland Security Analysis Center [S214] with SV42) (U) Management Response (b)(3)-P .L. 86-36 :findin, (U / /~OUGj-CONCUR. Management concurred with the and recommendation and has taken appropriate action. I was implemented on 25 June 2010, and the Director of Compliance, Office of General Counsel, SID Oversight and Compliance, and DoJ representatives were provided demonstrations and expressed their approval. TOP SECLlET;S'CO}lfl,\7;'/NOJVR,'f)l 4 DOCID: 4230249 ST-10-0004C (U) O/G Comment (U / /FOUO) Management has taken corrective action that meets the :intent of the recommendation. (U/,q::ouot Analysts' Duties Are Not Clearly Defined and Separated (C//RfflL "PO USA, FVf¥t1 The March 2010 query of an expired selector revealed another weakness: DIAs can query selectors for data :integrity and foreign :intelligence purposes. The Standards for Internal Control in the Federal Government state that key duties and responsibilities should be divided among different people to reduce the risk for error and fraud. No one :individual should control all key aspects of transactions or events. Although DIAs do not conduct target analysis or report on targets, they might help a foreign :intelligence analyst with a question on a target. In those cases, the DIA is query:ing for foreign :intelligence purposes, not data :integrity, and must use the same rules as foreign :intelligence analysts. These procedures require that DIAs and foreign :intelligence analysts use a standard login that :invokes controls over query:ing, such as preventing the query:ing of selectors with a status of "not approved." However, DIAs also use special logins that bypass such controls and allow them, for example, to query selectors that are not approved, which is permitted for data :integrity analysis but puts DIAs at risk for query:ing for foreign :intelligence purposes without controls. (C//R±SL 'FO U&h, FVESY) The 'March 2010 :incident revealed that the functions of DIAs are not clearly defined and communicated. It is unclear whether the DIA's query was for data :integrity or foreign :intelligence purposes. The standards for :internal control require that key areas of authority and responsibility be defined and communicated throughout the organization. The standards also call for managers to document clearly such :internal control mechanisms :in management directives, administrative policies, or operating manuals that are readily available . . (TS//SI//~W) Although 8214 management.stated that they discussed with DoJ the appropriate functions of DIAs, personnel did not have a common understandihg of the types of queries appropriate for foreign :intelligence and data integrity purposes. Furthermore, existing guidance did not clearly link the types of queries with the purpose of query:ing, and supplementary guidance was still :in draft. For example, after we identified that an expired selector had been queried :in March 2010, it was unclear whether the query had violated the FISC BR Order. Specifically, personnel had differences of opinion as to whether the query had been for foreign intelligence purposes and, therefore, a violation or for data :integrity purposes, which is not a violation. (T£f/~l/f1\T¥) Without clearly defined roles, a distinct separation of duties, and well-understood policies that differentiate queries for foreign :intelligence and data integrity purposes, DIAs are vulnerable to errors TOP SECRBT; 1/CO;AfE"lT;'/'1"lOPORi\T 5 DOCID: 423024:9 REF ID:A4197247 rOP SECRET//COMllvT;/NOFOR.N ST-10-0004C and violations of the FI8C BR Order. In particular, DIAs might mistakenly query selectors for foreign intelligence purposes while using the special login that bypasses key controls. (U) RECOMMENDATION 2 (TS//Gh'/NF) Clearly define and separate the duties of DIAs and foreign intelligence analysts. Specifically, implement controls to prevent an individual from querying BR metadata for both data integrity and foreign intelligence purposes and issue formal guidance to differentiate such queries. (U) (ACTION: Exploitation Solutions Office [S313] and Structured Repositories [T132]) (U) Management Response (U / /FOUO) CONCUR. Management concurred with the finding and recommendation and provided target completion dates. Management plans to move data integrity functions out of 8214 and into 8313, and T132 and will develop appropriate procedures and job descriptions. (U) O/G Comment (U / /FOUO) Planned and ongoing actions meet the intent of our recommendation. TOP SECRET//CO}.fl{1{T//fllOI'OKN 6 DOCID: 4230249 REF ID:A4197247 TOf' SECREli/COi"vfl:N'l//NOFORJ.V ST-10-0004C IV.. (U) ACRONYMS AND ORGANIZATIONS (TS Tl I 'Sf I ''NF) BR Tl Business Records (U) DIA Data Integrity Analyst (U) DoJ Departm.entofJustice (U) FISA Foreign Intelligence Surveillance Act (U) FISC Foreign Intelligence Surveillance Court (U) RAS reasonable articulable suspicion (U) S2I4 Homeland Security Analysis Center (U) 8313 Exploitation Solutions Office (U) SV42 Special FISA Oversight and Processing (U) T132 Structured Repositories 'f'Oi' SECR£T//C01vff.NT/;/fv'OFORN 7 DOCID: 4.230249 REF ID:A4'197247 Te:P 5E€.n T'."T' 11,-,r-.1 KT1\ T'T';-11\ rr-.r.nn 1\ T 1\LI// CVlWlV 1//1 VOL Olli' ST-10-0004C (U) This page intentionally left blank. TOP SECRET;$LCOi''tffl>IT;$1.Aro_,_r:.oR1\T 8 DOCID: 4.230249 REF ID:A4197247 TOP SE€RET; /tCOP.fI1VT;'j(\'OFORl\l 1 ST-10-0004C (U) APPENDIX A · (U) Objective, Scope, and Methodology TOP SECR£'F//C01ifINT/;'NOflottN DOCID: 4230249 REF ID:A4"197247 TO" SEC.n r:rr / /r>r>"A rrr7'/11. rnr:nn 11. r i\L1// COlWl 4' 1// 1"01 Olli Y AT/\ .L ST-10-0004C (U) This page intentionally left blank. TOP SECR£T;';'C01v11i\ft'//Nfff0t?.N REF ID:A4197247 DOCID: 4230249 TOP SIJ..Cfili.T//CUMlL'lT//NOFORJV ST-10-0004C (U) ABOUT THE AUDIT (U) Objective, Scope, and Methodology (U) Objective ('FS//SI//N:F) The overall objective oftbis audit is to test whether controls to ensure NSA compliance with key terms of the Foreign Intelligence Surveillance Court (FISC) Order Regarding Business Records (BR) are operating as intended. During the pilot test phase of the audit, oµ.r objective was to determine NSA compliance and assess the feasibility and reasonableness of including in monthly testing six objectives related to querying and dissemination. For monthly testing, our objective is to test NSA's compliance with seven requirements of the BR Order and determine whether controls are operating as intended. (U) Scope and Methodology (U) We conducted pilot testing from January to March 2010; monthly testing of January through July 2010 data was conducted from March to August 2010. · (T£//SI//Pf.F) For both-pilot testinr and monthly testing, we compared all selectors that were do~1:1!r.l.~nted.-inl . laudit logs and had been ql]._~rje..d.eaoh-montl;fagainst access lists, reasonable articulable suspicion .,,,,,...... -··- ·-··· -··-·-······ -·-·-·approvals documented in the Foreign Intelligence Surveillance Act BR (b){~):P;b··86~36........ d.gtabase, and Office of General Counsel reviews documented in the ·······... Homeland·Requests...P.~~base. We also counted the number of hops ·. . chained for each selector iii"th~·................... laudit lo s. For monthl · · ... _ testing, we also applied these tests to quene·s.ofthe We researched any an'-0-m---.....e-s'""'t_o_m__,,....1 final determination of compliance. l I (U / /FOUO) We met with individuals from the Office of General Counsel (OGC), the SIGINT Directorate, and the Technology Directorate, including the SID Office of Oversight and Compliance, Information Sharing Services, Homeland Security Analysis Center, SID Issues Su,ort Staff,------·· Analytic Capabilities, Structured Repositories, andl -- · (b)(3)-P.L. 86-36 Operations. (U / fFOUO) Details on the scope and methodology used for pilot testing, including scope limitations, are included in our Pilot Test Report (IG11154-10). Details on monthly testing are included in the January to March 2010 Test Report (IG-11160-10), April 2010 Test Report (IG11163-10), May 2010 Test Report (IG-11174-10), June 2010 Test Report (IG-11179-10), and July 2010 Test Report (IG-11188-10). Appendix A Page 1 of 2 TOP SECRET;'/COAfINT//f.v'OFORJ-l DOCID: 4230249 REF ID:A4;~7247 TOP SECRET;/rCG}•Hi\JTyrOFO.R.l+· ST-10-0004C (U) We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our :findings and conclusions according to our audit objectives. We believe that the evidence obtained provides a reasonable basis for our :findings and conclusions according to our audit objectives. Appendix A Page 2 of 2 TOP ~RF:T//C()},fL''ff//f'VroFORN DOCID: 4230249 REF ID:A4197247 TOP·SECRET//CO:iYJ.7J·{T/;~VOr01uvT ST-10-0004C (U) APPENDIX B (U) Summary of Recommendations TO? SECRE'f'//C01dlN'f'//t•lO?ORl•ll DOCID: 4230249 REF ID:A41.q7247 TOP SEC.Tl r.rr I I r rn ,rn rrr7 /( r .n r..n n "!\T .L(L1/) COlilll Q 1// 1 4 U1 U1tI'i ST-10-0004C (U) This page intentionally left blank. TOt1 S'l!:Cll'f'/;'COlvtI1V'f'/;'f<;l(JffJttN REF J:D:A4197247 11 DOCID: 4230249 ce·w 'T TOP SECRET/11/ 1 ' .t1J " 'Ol // l, Oll", 114 ST-10-0004C (U) Summary of Recommendations Recommendation 1 (TSt/Slt/NFl Immediately verify that controls in the newly released ver.~.i~9. are in place and functioning to: · ofl ...... ··············· (b)(3)-P.L. 86-36 I a. prevent querying selectors associated with U.S. pe_rsons without a documented OGC review for First Amendment considerations; b. prevent querying selectors not revalidated within BR-mandated limits (180 days and one year for U.S. and foreign selectors, respe~tively); and c. tag, track, and identify time-restricted selectors. ' (U) Status: CLOSED Recommendation 2 (TSNSIHNF) Clearly define and separate the duties of data integrity analysts and foreign intelligence analysts. Specifically, implement controls to prevent an individual from querying BR metadata for data integrity and foreign intelligence purposes, and issue formal guidance to differentiate such queries (ACTION: Exploitation Solutions Office [S313] and T132). ~~~ ~!~~~~:C~~~~tion Dates: i....--------.~~~T~~3 · · · · · · · · · · · · · · Appendix B Page 1 of 1 (b)(3)-P.L. 86·36 DOCID: 4230249 ST-10-0004C (U) This page intentionally left blank. 'I'D? SECRETi/C01vJi1\lr/;/:!•lOFOffN DOCID: 4230249 REF ID:A4197247 TOP SECRET//COfofll'v'Ti/IVO:FOftiJ;:·· AT&T, the Operatmg ·s~p;§l;·•· 'ii .. Pminership d/b/a Verizmi~~:· Associated Terrorist Organiz, and Abr , fiiliated with al'' the 9:~Z~ f han and P~s ';~ed States and " ,~. ·st · ·--·----·--- m: _-.-J'7;-,,,·:· . lligence Surveil '"ie?·•::fcfw.iher advises '.J.m·Jg.~.~fl:t~81,1~1 Security Agency (NSA) advised the Department of Justice's NatiQ!lttl,;~~~P;R:~Y:.·R!Ji~j()p.''.%the compliance incident desclibed below: ~~~l~~~ll~JJf fcRET//COMJNT//NOFORN cl1i_§·stfi6d by: Davids. Kris, Assistant o/ Attorney General, NSD, DOJ Reason: l.4(c) Declassify on: 2 August 2035 DOCID: 4230249 REF TOP I On March 9, 2010, a DLA queried the BR metadata ill to a Federal Bureau of hwestigetion (FBI) request for certain infunnmion l'elafing to a United States telephone identifier refereueed in a previously issued NSA report. Specifically, the FBI inquired a The reasonable, alficulable suspicion (RAS) approyal'lfm ler expiremon before the query. (I111 dibea'n RAE-approved on -.) Still, the identifier was listed on the iqn Table hisloiically, list of identifiers that have undergone RAS dete lions as RAS--approved unh'l - at time its status was eheng to "not . . ,i The DIA used the identifier eeiidue/t shingle quei 0/1" the BR needed-in the Tiensaetion Database. Altireughthe prelimiuni ntiee eitliis inei'de tr 0 that the query was timaebouudcd/m/flie period of through" mie- query was not time-hounded 333151, the DIA ed reigew'of'the queryresultsg to the time period referenced in the 1731's request a Based on the query results, the DJA detemiined that no memiu'g date was available for fl-re identified, an NSA provided than l'ilfolmah'on to the FBI. didnog report based err th squery. 1 This int/gent was discuveseg by the staff of SA's Inspector General through their review of 0011 used te the Court's 0rd is in this matter; NSA dirihiins that it conducted no using the identifier after the DLA's quary describcd abuvei At the timsbfihis incident, NSA managed the RAs--eppreyel'steti'is ofiderltifiers\oll the Station Table threughk periodic, ihnnusl review of these identirie 5. NSA assesses that '3 eernplianee incident resulted from delays in the manual ieview process. NSA further as esses that a technical modifieetiehliltely will 131':va these of eomplisneeiireideut flaw/occurring "fit In June 2010, implemented a-new pregral'n to manage and haemequests te ectierr Chief, Ovelsl'ght - Natlonal Security Division us, Department of Justice TOP 2 DOCID: 4230249 REF ID:A4197247 'f'OT' SECR£T//COMI1VT //N0f'OR1'l ST-10-0004C (U) APPENDIX D (U) Full Text of Management Response DOCID: 4230249 REF ID:A4197247 1 TOP SECRETi~ COi'riH'llT/1%TJr0tm' ST-10-0004C (U) This page intentionally left blank. TOP Si!CltET;'/COlYJJN'T;'/NOFORJ.\l REF ID:A4197247 SECURITY CLASSIFICATION NSA STAFF PROCESSING FORM TO OIG THRU I EXREG CONTROL NUMBER I KCC CONTROL NUMBER 2010-4645 EXREG SUSPENSE ACTION ~~~~~~~~~~~~~~~~~~~~~~~~~~--! SUBJECT (TS//Sl//~JF) SID Response: Quick-Reaction Draft Audit Report ofNSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records - Control Weaknesses (ST-10-0004C) D D 18 Aug 2010 APPROVAL KCC SUSPENSE SIGNATURE ELEMENT SUSPENSE X INFORMATION 2 Aug 2010 DISTRIBUTION SID, 802, S2, SV, D4, Tl2, OGC SUMMARY PURPOSE. (T6//SI//P'¢F) To provide the SID Response to the subject DRAFT Report. BACKGROUND: (TSHSI:h~ff) In May 2010, the OIG issued the Pilot Test Report (IG-11154-10) as part of the ongoing audit ofNSA Controls to Comply with the Foreign Intelligence Surveillance Court (FISC) Order Regarding Business Records (BR) (ST-10-0004). The pilot testing identified three control weaknesses in querying BR metadata as well as concerns related to the dissemination of information. Because there was no evidence of non-compliance and the release of the new selector tracking application that would address the weakne~~~sl was imminent, the OIG didn't make formal recommendations opting to monitor the sitµation and make formal recommendations as necessary. . .... ··· · {b)(3)-P.L. 86-36 I ('f~//~I//Mfl) The continual slippage ofl lreleas~d~t:I· Jreleased June 25, 2010) coupled with the March 2010 non-compliance incident (which underscored one of the reported control weaknesses and identified an additional weakness) resulted in the OIG recommending Agency management take immediate action. The subject quick-reaction draft report is the result of the problem that warranted immediate attention by Agency Management. DISCUSSION. ff5//5I//Mfl) The SID Response·to the subject document has been coordinated with 52, SV, T12, D4 and OGC. It includes the response to the two Recommendations for SID Lead and NSA' s response to the DOJ' s notice of violation. Also included for your reference is the SV42 response to the March 2010 incident relative to the subject report. _//(IJ)(3)-P .L. 86-36 .·.. / co0-RDiNAJION/APPROVAL ··. ·.. OFFICE 1)4 S2 ORIGINATOR SID IG Liaison, FORM A6796 John DeLon //email//8/6/10 /email//8/9/10 963-3335 sv T12 NAME AND DATE 10 /email//8/6110 963-1705 963-0247 ORG. S023 PHONE (Secure) 966-5590 DATE PREPARED 11August2010 Derived From: NSA/CSSM 1-52 Dated: 20070108 Declassify On: 20320 I oRff'IEjfl'-§1.~~~etll:M:fH:f#i~~ffl'.~ SECURE PHONE 963-8309 DOCID: 4230249 REF ID:A.4197247 'l'OP 8ECRE'tYlCOlrflll117'f\'&FORN (TSHSJ:n'-NF) SID Response: Quick-Reaction Draft Audit Report of NSA Controls to Comply with the Foreign Intelligence Surveillance Court (FISC) Order Regarding Business Records - Control Weaknesses (ST-10-0004C) (TS//SI//HP) Introduction: The SID Response has been coordinated with the Deputy Directorate for Analysis and Production (S2), SID Oversight and Compliance (SV), and the Office of General Counsel (OGC) because the same issue is being addressed in parallel channels at the SID level and above. The Department of Justice (DOJ) filed a 10c notice of violation with the Foreign Intelligence Surveillance Court (FISC) to which NSA, through OGC, is providing a non-concurrence on describing this event as a violation. NSA' s response to DOJ is included in the Background and Context section of this document. It is being provided to ensure that NSA provides consistent responses and appropriate context to these parallel reporting actions. While NSA does not agree that this event was clearly an 'incident of non-compliance,' it does highlight deficiencies in the previous selector management applicationi nevertheless it falls short of a compliance violation. (ilH?J~f.>:.L.JJ.6-36 \" -:::::::: : ::::·::: . ::.::RE~?.MME~J.?:_'.TION.1: ffSftj~fl) Immediately v~rify that controls in the newly \ ·<. ·"·l'.~leased..v.~!s1on ofl _are m place and functiorung to: ·.. ""a). prevenlquezying selectors associated with U.S. persons without a documented . ".. "·ode.review for.FitstA.mendment considerations, . b)"t?l'.:?~ehtquerying sei~clors.Q~t revalidated within BR-mandated limits (180 \ days an(J. one"y~ar for U.S. and foieign_s,electors, respectively), and " c) tag, track'-. amfide.p.tify time-restrictecfseledq_rs. . If the conclititn1s in a;--b,Jmd c cannot be verified.;imm~~:liately develop and . implement interill'.l:plans'to-~~dress these weaknesses untill lean be . d .. . "· dif mo ie . ········... ... . SID Action Element: Chief,S2I4 with SV42 and T1222 \ \ 201~):(0/)FOU~)~IDcqncm·s SID RESPONSE (August with this recommendation. On 25 June 2010 the new selector martagement syste1:n;I I was activated and \ all deficiencies noted in the OIG report have been addressed. The OIG has been '\ provided real time updates associated with this release and has interacted with S2I4's I lliaison in order to perform their own: review of the application. Additionally, the Office of the Director of Compliance (ODoC), Office of General Counsel (OGC), SID Oversight and Compliance (SV), Office of the Inspector General (OIG) and Department of Justice representatives have all hadl functionalities demonstrated to them and expressed their approval (see additional information in Explanatory Remarks section) I POC: 1 ....._ _ _ _.....!Chief, S2I4, CT Homeland Se:~~!YHJ.\r.:t~llysis,j.__ __.1969-0224 Derived From: NSA/CSSM 1-52 Dated: 20070108 Declassify On: 20320108 FOP SECRE'fYlCOl,,JIJV'fYfNOFOltlf (b)(3)-P.L. 86-36 REF ID:A4197247 DOCID: 4230249 TOP SECR:Ef'fl'CO!dll,TT!'J'f•lOPO"JJ:lil (TS#S;wNii) Quick-Reaction Draft Audit Report of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records - Control Weaknesses (ST-10-0004C) (TS//SI//HF) March 2010 Non-Compliance Incident- Additional Information (TS//SI//HP) SID Oversight and Compliance/PISA Authorities (SV4) emphasizes that all of the items listed in recommendation 1 are procedures and features of th,....e_ ___, ...... , !program that h~~~ been in ~l. a.ce sine~ ~une -~-~'...~9.~9.: .NSA.Wayl ..... . hmtial operati:ng. capab1hty·was concluded by T12 personnel on June 22, 201 Q: .Ihis.acceptance·snouid serve as the testing verification for the ..requiremerifSset out in recommendation 1 of the subject report. I · _,..:: ... :· ('iJj(:J")~P .L. 86-36 (TS//SI//fqf) Operational testing and evaluation is on-going under real-world use while the developers and technical oversight personnel are monitoring "bug reports" and user feedback with a keen eye toward compliance issues. In addition, an . _Emergency Change process is established with a cross-organization technical and 6v~rsight team in place to resolve any compliance findings or to determine adjustments to the program should changes in the legal environment occur. •, (U) SV42 p~oposal related to Recommendation 2. (TS//SI//NF) ~elow are the DIA roles and specific functions as defined in the Data Integrity Analyst-I !standard Operating Procedures (SOP), dated September 28, 2009, while the DIA's were assigned to the SIGINT Directorate. ! _____.Ito-···· ··(b}(3)-P.L. 86-36 (TS//SI//P .L. 86 _36 t ~ ~T p~~ uct line. (b)(3)~o <\ use 3024(i) ··········... ············ ············ . B. The second availabletol~.__,,__ _ _ _ _ _ _ _ _ _ _.....IWithin this second role was a list of typical support: 1. Reviewing telephone identifiers prior to and or after the issuance of a serialized report or a Request for Information (RFI) in order to verify the accuracy of thq ldata; "(b)(3)-P.L. 2. Helpirig analysts interpret and understand the results of their queries. 3. Confir:rrtl ] m ,.m ...... .... m,um..... m ,.mmu"U \. . ....-, I - - - - - - i - - - - - - - - - - - - - - - "I Derived From: NSA/CSSM 1-52 Dated: 20070 I 08 Declassify On: 20320108 TOP SECREIWC01l!BVTA'NOPOR:A' 86-36 . ~ .. R.EF ID:A4197247 DOCID: 4230249 TlhQ SECREnSWiW1JV1/sCJVOFORJV C. The thirc:l.role hich provides the DIA by-pass capability. This third tool was described for use in technical and data integrity purposes only and the ~~~g~-P.L. 86 _36 ~y-pas.s capability was specifically called out not to be used to support functions (b)(3)-50 use 3024(i}n sections A. or B above. ..................... ·····:::::; ... ."''''''''' (TS//SI//tqf) SV4 recommends that those offices that have taken on the functions, previously or currently known as the Data Integrity Analysts, establish a policy that clearly defines and prohibits the use of RAS by-pass modes while working on data for or assisting other analysts for intelligence analysis purposes. (T5//5I//~JP) The policy should state that the use of any RAS by-pass functions should be limited to processing and data formatting purposes to ensure that the metadata is accurate and usable by analysts and to ensure compliance with the FISA Court Orders. (TS//SI//NF) The policy should allow that technical support personnel or DNR Subject Matter Experts working with BR FISA metadata should be able to continue to provide technical support to intelligence analysts for the purposes of assistance with accuracy and technical interpretation of the metadata with or without any RAS by-pass function enabled. (TS//SI//NF) However, the policy should strictly prohibit the use of a RAS by-pass function by technical support personnel or DNR Subject Matter Experts as described above to assist with or provide any analytic interpretation of results of queries against the BR FISA database that would supply any information of intelligence value. roe:I ...lsv42, 969-0024 Appro'-v-ed-by-:..-l- - - - -...,..~...,.J... 1ief SID Oversight and Compliance, 2 August 2010 (b)(3)-P.L. 86-36 DOCID: 4230249 REF ID: A41:9724 7 FOP SECRET/lC:OlrfTN1YJ'NOFORJl{ RECOMMENDATION 2: (T8#8J#NF) Clearly define and separate the duties of Data Integrity Analysts and Foreign intelligence Analysts. Specifically, implement controls to prevent an individual from querying BR metadata for data integrity and foreign intelligence purposes and issue formal guidance to differentiate such queries. (U) (ACTION: Chief, S2I4 with SV42 and T1222) SID RESPONSE (August 2010): (TS/;'SI//HF) SID does not concur that this is an action for Chief, Homeland Security Analysis (S2I4) as stated in the rec01mnendation. Counterterrorism (CT) Production Center (S2I) does not intend to retain individuals in a 'data integrity analyst' (DIA) capacity and is working to transition those functions to where they fit better within SID. The DlA function is one of the legacy consh·ucts tracing back to a former NSA compartmented program. The DIA' s role was not clearly distinct from target analysts. S2I4 determined during the end-to-end reviews that data integrity analyst functions should be moved out of the production organization and aligned with other corporate elements within SID' s SIG DEV Strategy and Governance (SSG) and Deputy Directorate for Data Acquisition (S3), who perform similar functions related to data integrity and fidelity at the point of ingest. Transition of DIA functions, not DIA positions, is ongoing with Cryptanalysis and Exploitation Services (CES) (531)/Exploitation Solutions Office (ESO)( 5313) and SSG. S2I has been working with Chief, Protocol Exploitations (S31323) on this h·ansition of functions. S2I4 leadership has asked TD to relocate the single remaining DIA (a TD resource) to T spaces. The analyst who performed the March 2010 query recently took a new job in SSG. Poe: I roe..· I lcruef,$4.J..4! . S::.!..~°.ll!eland Secur.ity Analy~ ..• 969-0224 1-....ehlef.>?t..~,I . g~~~~~~::~~~~9~-~~~'._L_jiq1o3- ~01 (U) Background and Context: ............ · . .... "==""'"'""····..(b)(a)-P.L. _ 86 36 EC//REL TO USA, FVEY) Where 5214 diverges from this report as written is in the description of the query performed in March 2010 as an 'Incident of Non-Compliance'. The report fails to provide adequate background context. (Te//81//NJ!i') The following was provided to OGC and DOJ for review as an explanation of the chain of events in the course of DOJ filing an initial 10c: (b)(1J•······················ (b)(3)-P.L. 86~36 (b)(3)-50 use 3024(i ror· SECRE1W€0iWLV1WNOFORl'l DOCID: 4230249 REF ID:A4197247 TOP 8ECREJWC01~Ili/JWNOFOR::lV (b)(1-) ...... . (b)(3)-P.L~·86•36 .... : (b)(3)-50 USC 3024() (TS//61/ fNF) S2I4 has no contention that the query performed I land noted in an OIG audit highlighted specific deficiencies in the legacy applications/used to manage RAS approved selectors. These same findings were noted during the/End-toEnd reviews of both the Business Records and Pen Register Trap & Trace FISA / programs. S2I4 leadershi~ strongl.y agreed with the recommendation to delay the release of thel applicati.Qr.:t~J::l~il such time as: 1) the End-to-End revi¢w findings were complete and had been fully.discussed withPQJ..(;llld 2) those fin;dings could be incorporated intol ho·address·compliance·vuhler:abilities"Hll"i ......... ·:::;:;;;>"·' (b)(3)-P.L. 86-36 ········ .. ·;:.··· : (S//NF) A new revalidation process was.~.stablisheffand;;ieni~h~~d in the fall of ;:9, albeit a ,::.:r cor:i~~!~;:n;~;:~::!sep;;~~::~!f re-engineet. Prior underpinned by its own appl.~s;ation~leaving NSA with a purely manual process during . this transition. S2I4 a.,1J.d-TD.counterparts validated all previous 'customer requirements' forl lancfworked through the 'NSA Way' process to completion. SV and OGC are also 'customers' of this application and along with ODoC, had visibility into the entire revamping process. This engagement continues to address any issues noted afterl frelease~····················m··m··· ..................................................................................... . .................. o:i(b)(3)-P.L. 86-36 (TS//SI//NF) Nonetheless, the legacy system's defi._c_ie_n__ c.__al_lo_w_,e9··~/DIA to·q:uery on a selector that should have no Ion er been retained in as)RAS approveµ. it should be noteq however, the DIA could still have queried on that selector as part of their 'data integrity' duties --- within the bounds._o_,...,t.....------.--.....-----' RAS approval. (U / /FOUO) Explanatory Remarks related to Recommendation 1: a) ES//NF) Any selector being reviewed for RAS that is a US identifier or is believed to be in use by a US person cannot be RAS approved without an OGC First Amendment review. As the nomination is entered into la field to note whether the selector is foreign or domestic must be ('6fr3)~;.:~. .:~~~~~ populated for the nomination to be processed. When the domestic field is p:opulatedJ . . lsends the nomination to OGC for review and no I b) ~~:~~~1/c?~~)c~~ ~e~!::~::t~~t;;~;!"J?t~~~:eeted. belector management system, a revalidation date is set tied to the date of approval TOP 8ECRETilCOMLVL$'/o{Of¥>R:1V REF ID:A4197247 DOCID: 4230249 TOP SECR£1WCOlJB"•l'f'J7'NOJi'ORJ•l and whether it is US or foreign. HSAC [Homeland Security Advisory Council] internal management guidelines are that all US selectors will be revalidated every 90 days and foreign selectors at 180 days. This protocol should preclude any instance of exceeding FISC mandated timeframes. A !will automatically move these selectors into a pending status 15 ....... days from the projected 'expiration'. If any selector in this status has not been ... revalidated by the cut-qff.dateJ lmoves the selector into an expired ..... state. The.select6r·!S··~~ longer noted as 'RAS approved' in the system ..·····:::..:,,.,,,,,,,,,,,,:,;:=::=:::'::::''::::::::.::::::! ............................................................................................ ·...................... j·andl hs informed of this ·c~){:3):P:i .. ~~-36 action in order to ensure this selector can no longer be queried in the '· · ··. ·······I IBRF or PR/TT repositories. c) (C//RI!:L TO DaA, FVEY) 'Time Bounded Query' restrictions have their own .. lcon.vv.:~ch :rompts an ~nalyst to check a sel~ctor's re~or.d withi1: the I ··. . J;ystem. This record notates the time restr1ct1on and mforms · :· . analysts of the··sp~cific timeframe they must focus on during the review of query results. Information outside of those boundaries must not be used in . , .., ._ the pursuit of their targets~ \._<::.:.·.·.·. .· :·:.:.· .· ·. ·.· . . · I ' . . .:·1,_- - - - - . - - - - - - - - ' - POC:I I Chief, S2I4, CT Homeland Security Analysis,I Approved by: DDAP,I l3 Au?. :.~............ ·• ... · · • (b)(3)-P .L. 86-36 TO:P 8ECRE1iVCOMill1i$'!\/rcJ-FORf{ 1969-0224 --- DOCID: 4230249 REF ID:A4197247 TCi3 SEC~E'f//COMlf7'NOFOKN TOP 8ECRE1;¥C02',RNT#P.lOFOR...\J REF I:D:A4197250 DOCID: 4230250 TOP SECRETh'COMINf'f1'lNOFOR:N OFFICE OF THE INSPECTOR GENERAL NATIONAL SECURITY AGENCY CENTRAL SECURI1Y SERVICE 30June 2010 IG-11174-10 TO: DISTRIBUTION SUBJECT: (TS//SI//ptF) Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records - May 2010 Test Results (ST-10-0004D) 1. (U) This report summarizes the results of our May 2010 testing using the continuous auditing methodology. 2. ('fS//Sf//MF) Audit Objectives We are conducting monthly testing of NSA. controls to comply with the Foreign Intelligence Surveillance Court (FISC) Business Records (BR) Order to determine whether these controls are operating as intended. 3. (U / /FOUO} Monthly Test Results and Objectives We found no instances of non-compliance for the month of May 2010. However, significant scope limitations remained in testing Office of General Counsel (OGC) reviews of selectors associated with U.S. persons because of control weaknesses reported in the Pilot Test Report issued on 12 May 2010 (IG11154-10). See page 4 for details of May 2010 monthly test results for the following objectives: • (TS//Sl//Jl'JF) Access: Were all queries to the BR metadata made by authorized individuals (e.g., intelligence analysts and data integrity analysts)? • (U / /FOUO' Reasonable Articulable Suspicion (RAS) Approval of Queried Selectors: Did all queries use RAS-approved seed selectors? • (U / f POUO) OGC Review of U.S. Person Selectors: Did OGC verify that RAS determinations of all queried seed selectors associated with U.S. persons had not been based solely on activities protected by the First Amendment to the Constitution? TOP SECRET//COfvHMfh't~Of?OIU~ I . DOCID: REF ID:A4197250 4230250 TOP ·snCRET/fOOMfNT//t~OFORN ST-10-0004D ' ' ~//REL TO U5A, FVEY) Chaining: Were'all qt,teri~S}i~~j.ned to • . ' ' . than three ' h ops. '? no more .· . . . ' · · · ·. . ,·,·, ·'<':' .. .. ' · · .;. •· {'/k, < :'.~r:.':}~\',(;,>;>:cL .. · ·... (U/ /FOUO~ Revalidation of Queried Selectors: w__·:~·.' , ,. .. ~ie~f foreign and U.S. person seed.selectors revalidat~d~i, .. \'..tP.e ·· Court's tirneframes-one year and.180 days, resped:iveiy':,...;and approved by an authorized Homeland Mission Coordinator (HMC)? . ·; I. . . : .'· . : . . . : . :·' • ~ I • (TS/ /Sl//~~F) We{!kly. Pisseniinatio,n Reports: l).id N.S~ . ., .: . accaj'.ately ~~ :coil:iv~etely report serialize4 qJssemiriatio,fl-· of BR · ·metadata, qj.itside NSA? .. · · · · · · ·· · • • '. : .. ; . • . • . • 1'1 (TS//SI//"NF) Dissemination of Serialized SIGINT Reports with BR Mefq.df1J~: Was ajl in,fqrmation ·~:t::;:;.~:::··t''::;;:::~I'.I'.:;}:,,;;;;;;;;;;;;;, ; '•' borl I"' '. · .· ';: · . .r., '"' .. ;.... ·;-; ~.-, . .,, ; • • '·· ; .~ :. 1 .,-;· .0~)(3)-P.~L .. ~~-~() ·. -.:1~~____.\r ·Assistant Inspector! General for Intellige~<;:e Oversight 'fOP 8BCM'fitc01vHM'fitt~OffOltM 2 · ·. · · REF ID:A4197250 DOCID: 4230250 TOP SECRETHCOI'vHMfh~~Of<'OitM ··... T1222 ··.... ... -. ··..- - ! ·.... poJ ~------------.....__ __. ···· ..:···. :.::·\., __ ····· (b)(6) ~~ (). DeLong) ~~tin >:: ;:.~~\~(b)(3)-P.L. 86-36 ./ GC P. Reyi:,9lds)::··::::, _,;: ::::::•.·:·/>::_:·:::::;..-/·/./ Sl ·...::> s / S2Ir..--_ _....__.:>.· . .-:> .· ~~;:-_-_-_-_-:_:"---'... : ..-· . / .. / i ________ .. T1222 . OGC I~G~P~O~C~.._-.....-;..., ..... SID IGPOC TDIGPOC ___.... Dl.._ D12 D13 D14 TOP SECRET//COMINT//tq-opoffiq3 ST-10-0004D DOCID: 4.230250 REF ID:A4197250 TOP ~F:CftE:"f77'COivffNfhHOf?ORH (U) This page left intentionally blank TOP SECRETHCOMINT//NOFOfil~ 4 ST-10-0004 D REF ID:A4197250 DOCID: 4230250 'fOF S'.ECIUS"f11COMIMI77'NOPORN ST-10-0004D (U) May 2010 Test Results ('f:'f>// 'SI//Mf!) Test results show that NSA complied with the requirements of the BR Order, with noted scope limitations, between 1-31 May 2010. The rating definitions are included on the last page of this report, Area Test Results Scope Limitations Rating 1. Access .9c9t;rl.~liC)nt;.···• ·. . w1tt1 st;()peo ;:. limitations • ';i· . · · •·.:. ·:z·-;··+··············· None 4\C.haining 5. Rev~lid.~tion of Querie'd · Selectors "· 6. Weekly Dissemination Reports 7. Dissemination of Serialized SIGINT Reports with BR Metadata Th~Oseeci'se\ectors queried for . foreign intelligenceo•.P.urposes were approVEi9 by authorizeq);J.M.Qs within the Court's t1meframes. TheL..J seed selectors th13t were not RAS ...a~oved were queried .f~. da integritYLJor "ident lookups"L,_J None V\(eekly Dissemination Reports . completely and accurately reported the D~erialized SIGINT reports issued. We limited testing to serialized SIGINT reports because briefings, litigation, and other types of dissemination were not easily testable using the continuous auditing methodology. The Chief or Acting Chief of Information Sharing Services (S12) approved thepserialized SIGINT reports issue . We limited testing to serialized SIGINT reports because briefings, litigation, and other types of dissemination were not easily testable using the continuous auditing methodology. Compliant (Taua111~1i=) 1(Chfl:E:t fe USA, FVEY) "!dent lookups" refers to querying a selecto~ l.,to determine the approval status of a selector. In such cases, the Emphatic Access Restriction controls prevent chaining of a selector that is not marked as approved for querying and return an error message to the ana,lyst. There is no violation of the Order, because the selector was not actually chained. · TOP SECRETHCOMI~ffH:NOFORN 5 (b)(3)-P.L. 86-36 REF ID:A4197250 DOCID: 4230250 'fOP SECRE'fh'CO"MH'ffh't~OFOfili ST-10-0004 D (U) Rating System Description A rating of green indicates that no instances of non-compliance with the BR Order were identified during testing. Any noted scope limitations were related to the application of the continuous auditing methodology, not known control weaknesses. A rating of yellow indicates that although no instances of non-compliance were identified, control weaknesses prevented us from testing the entire universe, as explained in the scope limitations. A rating of red indicates that one or more instances of non-compliance with the BR Order were identified during testing. (TSffSIHNF) TOP SECRE'fh'COMINT//NOFOl\Jq6 Rating REF ID:A4197432 DOCID: 4230251 TOP SECRET#CO!vBN'f}J'l\iTOFORN NATIONAL SECURITY AGENCY/CENTRAL SECURITY SERVICE Further dissemination of this report outside NSA Is PROHIBITED without the approval of the Inspector General. INSPECTOR GENERAL REPORT (TSHSIHNF) Audit of NSA Controls to Com ply with the Foreign Intelligence Surveillance Court Order Regarding Business Records June 2010 Test Results (ST-10-0004E) 20 July 2010 ~pproved for Release by NSA on 08-06-2015. FOIA Case #80120 (litigation) Derived From: NSA/CSS Classification Guide 2-48 Dated: 20090804 Declassifj> On: 20350709 DOCID: 4230251 REF ID:A4197432 TOP SECRE'fWCOMfNT!lNOFORN {U) OFFICE OF THE INSPECTOR GENERAL (U) Chartered by the Director, NSA/Chief, CSS, the Office of the Inspector General (OIG) conducts audits, investigations, and inspections. Its mission is to ensure the integrity, efficiency, and effectiveness of NSA/CSS operations, provide intelligence oversight, protect against fraud, waste, and mismanagement of resources, and ensure that NSA/CSS activities are conducted in compliance with the law, executive orders, and regulations. The OIG also serves as ombudsman, assisting NSA/CSS employees, civilian and military. {U) AUDITS (U) The audit function provides independent assessments of programs and organizations. Performance audits evaluate the effectiveness and efficiency of entities and programs and assess whether program objectives are being met and whether operations comply with law and regulations. Financial audits detennine the accuracy of an entity's financial statements. All audits are conducted in accordance with standards established by the Comptroller General of the United States. {U) INVESTIGATIONS AND SPECIAL INQUIRIES (U) The OIG administers a system for receiving and acting upon requests for assistance or complaints (including anonymous tips) about fraud, waste, and mismanagement. Iilvestigations and Special Inquiries may be undertaken as a result of such requests, complaints, at the request of management, as the result of irregularities that surface during inspections and audits, or at the initiative of the Inspector General. {U) Fl ELD INSPECTIONS (U) The inspection function consists of organizational and functional reviews undertaken as part of the OIG 's annual plan or by management request. Inspections yield accurate, up-to-date information on the effectiveness and efficiency of entities and programs, along with an assessment of compliance with law and regulations. The Office of Field Inspections also partners with Inspectors General of the Service Cryptologic Elements to conduct joint inspections of consolidated cryptologic facilities. TOP SECRETh'€0A>ffNTh'NOf'ORJ.•l DOCID: 4230251 REF ID:A4197432 TOP SECRE'IWCOAfEl'fI/$4'.rO:fOR...1¥ TOP SECRE'f'#COMlNT;J'tri or additi(>nal. _.. _; . .. ' - .. ~ .. ,.; . . .. : ;•. ~ •', . t' ... .. l . .l '· ......•: l •. . • ' ,. :: ',! ' • l '' ·.=. " '<).); -------··:.-:, .· ..:.......... · · .-::~::::::" ...:;;;;;;!/;;. fb;:(3)~P~L.86: ~6· · · · · ' ' ~- ,. ! .· •' . . : : • : . .' "' ' • _. .. . •. : . 6Ii963~Z988(s) ot~vfa ~jinfill iit ... ·n: 95z.;2171(s) 6r Vla'. 1~~~it' ' . infob:riation; ..t~ase contac 1 •• . ·' • ' . . . :_·;:" : :i .;,•·;·•.. ' , I''.~· '. • .'~· .'• : ~- .: . ~ . ' j ' .. · ;_;·i ' . ·": ',{ ·· ··: .. _·······!...._ _ _ _ _ _ _ _ ___.! Assistant Inspector <;;eneral for Intelligertce OV~rsight TOP 8ECRBT/,tCOMD+Tt&l0¥0v.;bT 2 ~ ~ J • .REF ID:A4197432 DOCID: 42302.51 TOP SECRETHCOtvfIN'fh~q=QFOitH ·.. ··....... ·· .. . ........... ·...... ··...... ··.... ·.. ··.. ·... ......... ·· .... ... ........ " \ .. ·. '• ·· ·· ... (b)(6) •, '··.'·· ··.:::.:-.. ·. __ .._______ .__ ...__ ........ "'--_: ..._____,.... ~~~.~ •, . . .. ~ ____ T1222 ·, ,' ' , OGCIGPOC -"""--....,;;. ..,.,___,: SIDIGPOC TDIGPOC,___ ___,...__ _~-~ Dl Dl..,___ _ ___. D13 D14 'fO~ SECR:ETHCOMINT//NOFO:fil~ 3 ST-10-0004E DOCID: 4230251 REF ID:A4197432 'fOP SECRE'f7'/COfvfflqTf71q0ffOftH This page intentionally left blank. TOP SECRE'f/ICOMIN''f/JMOPORN 4 ST-10-0004E REF ID:A4197432 DOCID: 4230251 'fO~ ~ECtuS'f//COivilNT/il'l;fOF'Om ST-10-0004E (U) June 2010 Test Results ('fS//SI/fWF) Test results show that NSA complied with 1) the querying requirements of the BR Order, with noted scope limitations, between 1-25 June 2010, and 2) the dissemination requirements of the BR Order for the entire month. The rating definitions are included on the last page of this report. Area Test Results Scope Limitations 1. Access Rating None · ... ·All [=:J RAS-approved U S person .. -:::::: ::B~sei~rd~ a control weakness reported in ..................... ...... . .. seed selectors queried inl I were th\3::f.>ilot Test Report, we limited testing to .)'("a"')· ;.,,~··•.L.. R~~iew .feV.ieW.~:iL!:?Y..,9.GG for First Amenclment .............:~~e!d selectors presumed to be associated (b &-n.iac. . ........... . ·. ·... ';t?.l\. p,,..on \ ·:::,:;; ::~~leqt?.rs " ":'\_:_::.:,·,.. :: ·....... ···.... ··.-: . ··.. I :~',':::~~~~~;',~;;;;; ;H~a e·ach was _// rev1ewe b y e forirst Amendment./ concerns as required. ·· · w1mtJs "'a"' ' ' "° ,.,;,. approva s o ore1gn selectors associated with U.S. persons, because the existing database does not track this information. None ... o no·more an ._ sel.\3C!or> . ·.. . ·-~~:eOseied ~electors queried for foreign of infel!ig!?nce purposes.yvere approved by 5 . Revalid.ation · authorized HMCs within the Court's Queried timefrai'he_s •. Thi3·0seed sele.ctors that Selectors were not RAS~roved were queiried for data integri!YL.J.~r "ident lookups" None 6. Weekly Dissemination Reports \J\leekly Dissemination Reports completely alid accurately reported the0serialized SIGU'-JT reports issued. We limited testing to serialized SIGINT reports because briefings, litigation, and other types of dissemination were not easily testable using the continuous auditing methodology. 7. Dissemination of Serialized SIGINT Reports with BR Metadata The Acting. Chief or Acting Deputy Chief of Information Sharing Services (S12) approved theperialized SIGINT reports issue . We limited testing to serialized SIGINT reports because briefings, litigation, and other types of dissemination were not easily testable using the continuous auditing methodology. D Compliant (TCHClmff) (TS//Sl//NF) "Iden! lookups" refers to querying a selecto~ ~()determine the approval status of a selector. In such cases, the Emphatic Access Restriction controls prevent.chaining of a selector that is not marked as approved for querying and return an error message to the analysf:-. ..There is no violation of the BR Order, because the selector was not actually chained. 1 'fO~ SECitlS'f//COMIU'f/fl~OFOfil~ 5 (b)(3)-P.L. 86-36 DOCID: :JREF ID: A.4197432 4230251 T'OP SECRE'F/i'COMINTi'i+mFO~J ST-10-0004E (U) Rating System Description Rating A rating of green indicates that no instances of non-compliance with the BR Order were identified during testing. Any noted scope limitations were related to the application of the continuous auditing methodology, not known control weaknesses. A rating of yellow indicates that although no instances of non-compliance were identified, control weaknesses prevented us from testing the entire universe, as explained in the scope limitations. A rating of red indicates that one or more instances of non-compliance with the BR Order were identified during testing. TOP SECRE'F/fCOMIMf/fMOrOIU~ 6 DOCID: 4230252 REF ID:A4197434 TOP SECRETn'€OM:Jl',l'fh'/1{0f'ORN NATIONAL SECURITY AGENCY/CENTRAL SECURITY SERVICE INSPECTOR GENERAL REPORT (TSNSIHNF) Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records July 2010 Test Results (ST-10-0004F) 18 August 2010 ~pproved for Release by NSA 011 08-06-2015. FOIA Case #80120 (litigation) Derived From: NSA/CSS Classification Guide 2-48 Dated: 20090804 Declassify On: 20350805 TOP 8ECRE1)$'€0AflNT#NOf'OAAr DOCID: 4230252 REF ID:A4197434 'f'OP 8ECttE'fJl'COlvflNf}7NOFOltN (U) OFFICE OF THE INSPECTOR GENERAL (U) Chartered by the Director, NSA/Chief, CSS, the Office of the Inspector General (OIG) conducts audits, investigations, and inspections. Its mission is to ensure the integrity, efficiency, and effectiveness ofNSA/CSS operations, provide intelligence oversight, protect against fraud, waste, and mismanagement of resources, and ensure that NSA/CSS activities are conducted in compliance with the law, executive orders, and regulations. The OIG also serves as ombudsman, assisting NSA/CSS employees, civilian and military. (U) AUDITS (U) The audit function provides independent assessments of programs and organizations. Performance audits evaluate the effectiveness and efficiency of entities and programs and assess whether program objectives are being met and whether operations comply with law and regulations. Financial audits detennine the accuracy of an entity's financial statements. All audits are conducted in accordance with standards established by the Comptroller General of the United States. (U) INVESTIGATIONS AND SPECIAL INQUIRIES (U) The OIG administers a system for receiving and acting upon requests for assistance or complaints (including anonymous tips) about fraud, waste, and mismanagement. Investigations and Special Inquiries may be undertaken as a result of such requests, complaints, at the request of management, as the result of irregularities that surface during inspections and audits, or at the initiative of the Inspector General. (U) FIELD INSPECTIONS (U) The inspection function consists of organizational and functional reviews undertaken as part of the OIG 's annual plan or by management request. Inspections yield accurate, up-to-date infonnation on the effectiveness and efficiency of entities and programs, along with an assessment of compliance with law and regulations. The Office of Field Inspections also partners with Inspectors General of the Service Cryptologic Elements to conduct joint inspections of consolidated cryptologic facilities. REF ID:A4197434 DOCID: 4230252 TOP SECRBT//COMR"~~solved a significant scope limitation in our testing of Office of General Counsel (OGC) reviews of selectors associated with U.S. persons, as all U.S. selecfors are now tracked. I I (U) See page 5 for details of July 2010 monthly test results for the following objectives: e ('f3//Si;'/HF) Access: Were all queries to the BR metadata made by authorized individuals (e.g., intelligence analysts and data integrity analysts)? e (U/ /FOUO) Reasonable Articulable Suspicion (RAS) Approval of Queried Selectors: Did all queries use RAS-approved seed selectors? Derived From: NSA/CSS Classification Guide 2-48 Dated: 20090804 Declassffj; On: 20350805 TO:P SECRETHCOMil~T/1~~0FORH 1 REF ID:A4197434 DOCID: 4230252 TOP SECRETNCOMil'ff//NOFORH e11 e Ill o 0 II ST-10-0004F (U/ /FOUO) OGC Review of U.S. Person Selectors: Did OGC verify that RAS determination.c; of all queried seed selectors associated with U.S. persons had not been based solely on activities protected by the First Amendment to the Constitution? (C//REL TO USA, FVfrY) Chaining: V\Tere all queries chained to no more than three hops? (U/ /FOUO) Revalidation of Queried Selectors: Were all queried foreign and U.S. person seed selectors revalidated within the Court's timeframes - one year and 180 days, respectively- and approved by an authorized Homeland Mission Coordinator (HMC)? (TS//SI//NP) Weekly Dissemination Reports: Did NSA accurately and completely report disseminations of BR metadata outside NSA? (TS//SI//HF) Dissemination of Serialized SIGINT Reports with BR Mef:adata: Was all information disseminated through serialized SIGINT reports approved by the Chief of Information Sharing Services (512) or one of the five other authorized individuals? 3. (U/ /FOUO) We appreciate the courtesy and cooperation extended to the auditors throughout the review. If you need clarification or additional information, please cor-n_ta_c""'~-------_.loJ::t_963-2988(s) or via e-mail at lon?5?::171(S)or~i:e-::il at terJ ·_. .· · ... .. ' .... ····:;.,: L---__,,.---~-~I Assistant Inspector General for Intelligence Oversight 'f Ofl 8E'.Cft~"f'//COMIM'f7'1HOFOIUof 3 86-36 DOCID: 4230252 REF ID:A4197434 TOP SECRETNCO'fvffiff//f cfOFO'R}if (U) This page intentionally left blank. TOP SECRETHCOMIHTHHOfO:lxH 4 ST-10-0004F REF JED:A4197434 DOCID : 4230252 TOP SECRETl/C01vill¢TNN OFOR1¢ ST-10-0004F (U) July 2010 Test Results ('fS//81//NP) Test results show that NSA complied with 1) the querying requirements of the BR Order betvveen 26 June- 31July2010, and 2) the dissemination requirements· of the BR Order for the entire month. The rating definitions are included on the last page of this report. Test Results Area Scope Limitations Rating .....----........---..........---jl--~~~~~~~~~~~~~~b-~- r-~~~~~--;~----~~~~ 1. Access None "•, Of theD~G~ries performed in . \ ., Ithe seed selecto~:?..ofO. . \ ·\ 2 ... ·RAS Apprq.11:<3l :::: "Were validated"as.. appr.oved: The ,.of Queried:::.. ·· remaining qµeries·'were for data i~"... · Nb .. Selee:t6rs· purooses:· 'The seed s6.1.E:Jsiers.. of '·: . .., .ne .,,,,:::=.. ·· l~w.e~ .1i\fe~e ... . .···· . ... .approve~ ... JD~..r.eroa1n1ngCjq~en~~ were ·~:~.. . ., . . . ...... .;:· ....................... for·data·mtegnty purposes. · ...... ./ I I .1iiii~~~;~;;;;;;;;;~~;;;;;;;~::::: .:::::::·:·:... (6)'(1 r::::::::::::: ................... ·· . (b)f~l~P .L. 6~3Ef' . ".::.:::<::.:. . 3. OGG Review "...".::: ....of U.S. Person ".... "·-$e.lectors " "·."··....·... " ··... .....·. "·..... ........ . " In accordance with our test plan, we did not test July 201 O data for this objective. Because we have 4. Chai~lh~ "......... ..noted no exceptions for the last six months of testing, we will not test this objective again until October "·· 20).0 (using September data) and January 2011 (using December data). ·... T·h·anse.ed selectors queried for foreign . inteWgerice purposes were approved by 5. Revalidation of ·authorized HMC~hin the Court's Queried tirn€)frames: .Th~LJseed selectors that Selectors were.11ot RAS·cipproved were queried for data integrity purposes. None 6. Weekly Dissemination Reports WeelT ST-10-0004 F (U) Rating System Description Rating A rating of green indicates that no instances of non-compliance with the BR Order were identified during testing. Any noted scope limitations were related to the application of the continuous auditing methodology, not known control weaknesses. A rating of yellow indicates that although no instances of non-compliance were identified, control wealrnesses prevented us from testing the entire universe, as explained in the scope limitations. Compliant, with scope limitations A rating of red indicates that one or more instances of non-compliance with the BR Order were identified during testing. TOP SECRET//GOMIHTNHOFORH 6 DOCID: 4230254 REF JJ:D:A4197437 i·;:: :::::··· ··. .'---..-.-··-·________. (bf(3}'P;t,86-36 .... • _ .•............... --IL..,___________..... . Assistant Inspector General for Intelligence Oversight TOP SECI.tB'fWCOMHff/fNOFORN 2 DOCID: 4230254 REF ID:A4197437 TOP SECRETh'COMINTh'MOf?OftM ST- I 0-0004G DISTRIBUTION: D21 SV42~---.....-~~-~ 512 S2I4 --------------- ... Tl_..J..----------1.....;;ii;;.-----. ······ :::::: ....···· (b)(6) cc: Director SIGINT Director D4 (J. DeLong) GC M. Olsen 86-36 sv "'---.....&......, SV4 Sl....-----.. ___ ______ ___ s2...,____-.1·· ......_ S2I ,.....,.... T12 ...._ __._...., T122 _._ T1222 ......_. OGC IGPOC SID IGPOCr'-1----..:..a...------ TDIGPOCI"------------~ Dl D12_ _ _ __. D13 D14 TOP SECRETHCOMIMfm~Of?O'l.tNT 3 DOCID: 4230254 REF ID:A4197437 TOI> SECI'tB1'i/COMfN''f/J1qQ'fOftM (U) This page intentionally left blank. TOP SECRET//COfvfIH'f/J!:>"" . •:;tnt DOCID: 4230254 REF ID:A4197437 'fOP SECitElY/COMIH'f//HOFOR:H ST- I 0-0004G (U) Rating System Description Rating A rating of green indicates that no instances of non-compliance with the BR Order were identified during testing. Any noted scope limitations were related to the application of the continuous auditing methodology, not known control weaknesses. A rating of yellow indicates that although no instances of non-compliance were identified, control weaknesses prevented us from testing the entire universe, as explained in the scope limitations. A rating of red indicates that one or more instances of non-compliance with the BR Order were identified during testing. 'fOP SECRE'fHCOMINTHNOFORN 6 $123025? REF DOCID: 4230254 REF ID:A4197437 TOP 3ECRl!:'t)YCOMfNT#P;,TOFOR..·"f).r TO-P 8ECRE1WCO-JllN'B~W-OFORi"'I REF ID:A4197439 DOCID: 4230255 TOP 8ECRETnl£01HH{f';]':NOFORN NATIONAL SECURITY AGENCY/CENTRAL SECURITY SERVICE Further dissemination of this report outside NSA Is PROHIBITED without the approval of the Inspector General. INSPECTOR GENERAL REPORT (TSh'Sl/iNF) Audit of NSA Controls to Com ply with the Foreign Intelligence Surveillance Court Order Regarding Business Records September 2010 Test Results (ST-10-0004H) 28 October 2010 ~pproved for Release by NSA on 08-06-2015. FOIA Case #80120 (litigation) Derived From: NSAICSS Classification Guide 2-48 Dated: 20090804 Declass(f'y On: 20351020 I 01' 5F:CttET!l'COMfNf#.NDFORJ..T DOCID: 42302.55 REF :ID:A4197439 TOP SECRETJ$<>>?!\ .. SV4 ... ,.......... . . ,_...__ _ ___._, ........ ...... . ,,:"'···:>·. . _>:.:.. Sl _ _ _..,..... .-:::· .. · ........· ... S2,,..._ .. ·· ' S2I \ 86-36 : .· .· .- . .- . ' / ---- . · -' .. T12 . T122 ... · T1222 OGC I._G_P_O_C_....___...,._..., SIDIGPOC TDIGPOC~---------~ D Dl.,,_2_ _ ___, D13 D14 TOP SECRET//COMINTh~(OFOIU( 3 ST- I 0-0004H REF ID:A4197439 DOCID: 4230255 TOP SECRET/i'CO:MHff//t40FOR~~ (U) This page intentionally left blank. TOP SECRET//COMHffHNOFORJ~ 4 ST-10-0004 H REF ID:A4197439 DOCID: 4230255 TOI' :SECfUff//COMIMfmq-oFORN ST-10-0004H (U) September 2010 Test Results (TS//SI//'P'W) Test results show that NSA complied with the requirements of the Foreign Intelligence Surveillance Court (FISC) Order regarding Business Records (BR) between 1 and 30 September 2010. The ratings are defined on the last page of this report. Scope Limitations Test Results Area Rating Aut~orjzed jndjyjdua!s rnad~):illll 1. Access !q1,1enes 'fi'om-' None Cham_ I .· >\ I'.. None .. ···· ...··· 3·;::0fflce of ..G. .. -:::: > eneral Counsel...... ..;.. · {OGC...).. revieVi(of U ·· · ~~~cf;(;on .,,,:;:::::::::::::: (b)f1l::"· (b)(3)::p·:Ls . -36 "<:.... 4·:.. Ch!=lining ···················- ... ~ "· "· " ' ..The RAS-approved t · d · I11;~ ~ersar".· · see se ~c ors ciuerie I D : \ \ None were reviewed by OGC for 1rs "~ Amendment concerns, as required. ../· b)(3 )-P. L. 86-36 \ · .... "At1C::Jchainl I' queries were ctiamed to no more than three hops from the seed selector. None lne-Qeed selectors queried for foreign mtelligence purposes were approved by authorized Homeland Mission Coordinators within the Court's timeframes. None ·· .... ··... ·. ··. ·. "· .... 5. R:evalid.ation of queried .~e1ei:;tors "··· .... 6. Weekly dissemination reports (WDRs) 7. Dissemination of serialized SIGINT reports with BR metadata _ "w~g~· c'omP.~e re ort~d the 1nfelligence and accurately serialized Signals NT) reports issued. The Chief of Information Shao· Services (S12 a proved the serialized s1dNf reports issue . We limited testing to serialized SIGINT reports because briefings, litigation, and other types of dissemination were not easily testable using the continuous auditing methodology. We limited testing to serialized SIGINT reports because briefings, litigation, and other types of dissemination were not easily testable using the continuous auditing methodology. Compliant .I TOI' :Sti:Citnil7COMIN I//NOFORN 5 .·· .... ·· .····· ········ ...... ······· (b)(1) (b)(3)-P.L. 86-36 (b)(3)-50 USC 3024(i) DOCID: 4230255 REF :ID:A4197439 TOP i;eCRBT/11COMHffHHOFORH ST- I 0-0004 H (U) Rating System Description A rating of green indicates that no instances of non-compliance with the BR Order were identified during testing. Any noted scope limitations were related to the application of the continuous auditing methodology, not known control weaknesses. A rating of yellow indicates that although no instances of non-compliance were identified, control weaknesses prevented us from testing the entire universe, as explained in the scope limitations. A rating of red indicates that one or more instances of non-compliance with the BR Order were identified during testing. TOP SECRETHCOl'vHNTNHOFO:RH 6 Rating DOCID: 4230257 REF ID: A41:97 511 TOP 8ECRET1$'COlrfiiVTt'/NOFORN NATIONAL SECURITY AGENCY/CENTRAL SECURITY SERVICE Further dissemination of this report outside NSA Is PROHIBITED without the approval of the Inspector General. (TSHSIHNF) Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records October 2010 Test Results ST-10-00041 01 December 2010 DERIVED FROM: NSA/CSS Manual 1-52 pproved for Release by NSA on 8-06-2015. FOIA Case #80120 liti ation DATED: 08 January 2007 DECLASSIFY ON: 20320108 DOCID: 4230257 REF ID:A4197511 TOP SfJCRE'f;5'COfvflNf;}NOPORl'l (U) NSA OFFICE OF THE INSPECTOR GENERAL (U) The NSA Office of the Inspector General (OIG) conducts audits, investigations, inspections, and special studies. Its mission is to ensure the integrity, efficiency, and effectiveness ofNSA operations, provide intelligence oversight, protect against fraud, waste, and mismanagement of resources, and ensure that NSA activities are conducted in compliance with the law. The OIG also serves as an ombudsman, assisting Agency employees, civilian and military, with complaints and questions. (U) Intelligence Oversight (U) The OIG Office oflntelligence Oversight reviews NSA's most sensitive and high-risk programs for compliance with the law. (U) Audits (U) The OIG Office of Audits within the OIG provides independent assessments of programs and organizations. Performance audits evaluate the effectiveness and efficiency of entities and programs and assess whether NSA operations comply with federal policies. Information Technology audits determine whether IT solutions meet customer requirements, while conforming to information assurance standards. All audits are conducted in accordance with standards established by the Comptroller General of the United States. (U) Investigations and Special Inquiries (U) The OIG Office ofinvestigations administers a system for receiving and acting on requests for assistance and complaints about fraud, waste, and mismanagement. Investigations and special inquiries may be undertaken as a result of such requests and complaints (including anonymous tips), at the request of management, as the result of questions that surface during inspections and audits, or at the initiative of the Inspector General. (U) Field Inspections (U) The Office of Field Inspections conducts site reviews as part of the OIG's annual plan or by management request. Inspections yield accurate, up-to-date information on the effectiveness and efficiency of field operations and support programs, along with an assessment of compliance with federal policy. The Office partners with Inspectors General of Service Cryptologic Components and other Intelligence Community Agencies to conduct joint inspections of consolidated c1yptologic facilities. TOP 8ECRE1WCOiWlNTAW.OFO/l:N DQCID: 4230257 REF DOCID: REF ID': A4197511 4230257 TOP SECRETh'COMlNT/1~mFORn OFFICE OF THE INSPECTOR GENERAL NATIONAL SECURITY AGENCY CENTRAL SECURITY SERVICE 1 December 2010 IG-11229-10 TO: DISTRIBUTION SUBJECT: (TS//SI//MF) Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records - October 2010 Test Results (ST-10-0004I) 1. f:PS//SI//NF} Background This report summarizes the results of our October 2010 testing, using the continuous auditing methodology, of NSA's compliance with the Foreign Intelligence Surveillance Court (FISC) Order regarding Business Records (BR). The Office of the Inspector General (OIG) is using this methodology to fulfill the oversight responsibilities assigned to it in the FISC BR Order. Specifically, from January to December 2010, we are conducting monthly tests of NSA's compliance with certain requirements of the FISC BR Order. Once monthly testing is complete, the OIG will make an overall assessment of whether the controls that are in place to ensure such compliance are operating as intended. 2. ('fS//81//NFj Monthly Test Results and Objectives For the month of October 2010, we found that one weekly dissemination report mistakenly listed as a serialized dissemination that was not derived from BR metadata. Although the error did not violate the BR Order, it underscores a weakness in the largely manual process to track and report BR disseminations that we noted in our Advisory Report on the Audit of NSA Controls to Comply with the FISC Order Regarding Business Records, dated 12 May 2010. The error is currently being corrected. (U) See pages 5 and 6 for details of October 2010 monthly test results for the following objectives: · ['FS//SI/OTF) Access: Were all queries to the BR metadata made by authorized individuals (i.e., intelligence analysts and data integrity analysts)? (U / /FOUO) Reasonable Articulable Suspicion (RAS) Approval of Queried Selectors: Did all queries use RAS-approved seed selectors? Derived From: NSAICSS Classification Guide 2-48 Dated: 20090804 Declassifj1 On: 20351115 'f'Ofl Sf:Crte:'f//COMIMi17MOflORN REF ID:A4197511 DOCID: 4230257 TOP ~ECReTJ/GOMftffi/tqOi 'OiUrJ ST-10-00041 • (U / /FOUO) OGC Review of U.S. Person Selectors: Did OGC verify that RAS determinations of all queried seed selectors associated with U.S. persons had not been based solely on activities protected by the First Amendment to the Constitution? • (C//RSLTO USA, FVEY) Chaining: Were all queries chained to no more than three hops? • (U/ fFOUO) Revalidation of Queried Selectors: Were ·an queried foreign and U.S. person seed selectors revalidated within the Court's timeframes-one year and 180 days, respectively-and approved by an authorized Homeland Mission Coordinator? • (TS//Sf//NF) Weekly Dissemination Reports: Did NSA accurately and completely report disseminations of BR metadata outside NSA? • fFS// Si// NF) Dissemination of Serialized Signals Intelligence (SIGINT) Reports with BR Metadata: Was all information disseminated through serialized SIGINT reports approved by the Chief of Information Sharing Services (S12) or one of the five other authorized individuals? 3. (U / /FOUO) We appreciate the courtesy and cooperation extended to the auditors throughout the review. If ou need clarification or additional information, please contac · on 963-2988(s) or via e-mail at :.\:~ on 952-217l(s) or via e-mail at I . .:;;;;::::::>;/ Assistant Inspector Gener for Intelligence Oversight TOP 8ECRET//CO:MfNT/flqQf0Rtq 2 REF ID:A4197511 DOCID: 4230257 'f0l3 S'e:CitETh'COMIMfNHOFORN ..·· ....... (b)(6) ·::::::<\· . . ·\ cc: Director SIG INT Director D4 (J. DeLong) GC M. Olsen > ,;,.~l,~~b )(3)-P ,L, 86-36 sv sv4'-----.... Sl s21r------...............· .· S2I ,•' / / ,.: Tl / T122,___ _ _.... ,. . ' '' ' \ , T1222 OGCIGa..-P-0-C...-__,__ _,._~ SID IG POC ___________, TDIGPOC,.__ Dl D 1,,,___ _ ___, D13 D14 TOP 8£CRETh'COPv·mffNHOFORNf 3 ST-10-00041 DOCID: 4230257 REF ID:A4197511 TOP SECR:E'f77'C01vHJl"OfORJil" 6 ST-10-00041 Rating REF ID:A4197511 DOCID: 4230257 TO'P ~f:CRB'fh'COMINTm~oFORH ST-I 0-00041 (U) Rating System Description Rating A rating of green indicates that no instances of non-compliance with the BR Order were identified during testing. Any noted scope limitations were related to the application of the continuous auditing methodology, not known control weaknesses. A rating of yellow indicates that although no instances of non-compliance were identified, control weaknesses prevented us from testing the entire universe, as explained in the scope limitations. ;,:·¢.<>mil1ia6t~;with·•······ · ':s~.C>l?e'li'1'!it~~i<>l1~,· A rating of red indicates that one or more instances of non-compliance with the BR Order were identified during testing. TOP SECRET//COMI~ff//JH0¥017.dN 7 REF ID:A4197511 DOCID: 4230257 'fOP SECRE'f//COM"IH'f/J'T:q=OFORlq= OFFICE OF THE INSPECTOR GENERAL NATIONAL SECURITY AGENCY .CENTRAL SECURITY SERVICE 1 December 2010 IG-11229-10 TO: DISTRIBUTION SUBJECT: (TS//SI//NF) Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records - October 2010 Test Results (ST-10-0004I) 1. (TS// SI/ OTF) Background This report summarizes the results of our October 2010 testing, using the continuous auditing methodology, of NSA's compliance with the Foreign Intelligence Surveillance Court (FISC) Order regarding Business Records (BR). The Office of the Inspector General (OIG) is using this methodology to fulfill the oversight responsibilities assigned to it in the FISC BR Order. Specifically, from January to December 2010, we are conducting monthly tests of NSA's compliance with certain requirements of the FISC BR Order. Once monthly testing is complete, the OIG will make an overall assessment of whether the controls that are in place to ensure such compliance are operating as intended. 2. (TS// SI//Mfi') Monthly Test Results and Objectives For the month of October 2010, we found that one weekly dissemination report mistakenly listed as a serialized dissemination that was not derived from BR metadata. Although the error did not violate the BR Order, it underscores a weakness in the largely manual process to track and report BR disseminations that we noted in our Advisory Report on the Audit of NSA Controls to Comply with the FISC Order Regarding Business Records, dated 12 May 2010. The error is currently being corrected. (U) See pages 5 and 6 for details of October 2010 monthly test results for the following objectives: !TS//SI//NF) Access: Were all queries to the BR metadata made by authorized individuals (i.e., intelligence analysts and data integrity analysts)? (U / /FOUO) Reasonable Articulable Suspicion {RAS) Approval of Queried Selectors: Did all queries use RAS-approved seed selectors? Derived From: NSA/CSS Classification Guide 2-48 Dated: 20090804 Declassifj; On: 20351115 TOP 8ECRET//COMIN"'f/J'T:q=Ofi'OttNf 1 REF ID: A41:97511 DOCID: 4230257 TOP SECRET//COf\iHN1'//P' TOP S!JI!:CRJITh'COMHJTh~~OFORN 6 ST-10-00041 Rating DOC ID: 4.23025 7 REF ID:A4197511 TOP SECRETHCOMH'fFN?ifOFORN ST-10-00041 (U) Rating System Description A rating of green indicates that no instances of non-compliance with the BR Order were identified during testing. Any noted scope limitations were related to the application of the continuous auditing methodology, not known control weaknesses. A rating of yellow indicates that although no instances of non-compliance were identified, control weaknesses prevented us from testing the entire universe, as explained in the scope limitations. A rating of red indicates that one or more instances of non-compliance with the BR Order were identified during testing. TOP SECRETHCOMHffh'l'ifOFORlif 7 Rating DOCID: 4230258 REF ID:A4197513 TO-.P SECREF15'Cf>lr1INF;5'NOFOR1V NATIONAL SECURITY AGENCY/CENTRAL SECURITY SERVICE Further dissemination of this report outside NSA Is PROHIBITEP without the approval of the Inspector General. (TSHSlh'NF) Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records November 2010 Test Results (ST-10-0004J) 20 December 2010 pproved for Release by NSA on 08-06-2015. OIA Case #80120 liti ation DERIVED FROM: NSA/CSS Manual 1-52 DATED: 08 January 2007 DECLASSIFY ON: 20320108 Tf>:P 8ECREF15'COM1IPOR:N DOCID: 4230258 REF ID:A4197513 TfJ:P S£CREf'/JCOMiiVT1l!v'()Ji'ftRJv 1OP SE CR1!1/lCtJM1lff!IJ.YO'P8RN REF ID:A4197513 DOCID: 4230258 I OP SECR'.Eil7'COMII SECiti!"Pf7'COivHH"Pmr;rofi'OftM ST-10-0004J (U) November 2010 Test Results (TS// BI// MF) Test results show that NSA complied with the requirements of the Foreign Intelligence Surveillance Court (FISC) Order regarding Business Records (BR) between 1 and 30 November 2010. The ratings are defined on the last page of this report. Test Results Area 1. Access 6. weekly dissemination reports (Wl;)Rs) Scope Limitations Authorized ;·ndividuals made all Chainl : I rom I ·.... ... . ·.. Rating None WDRs are no longer a requirement of the BR Order; therefore, this is no longer a testable objective. 7. Dissemination ·1 of serialized SIGINT reports _ with BR metadata 1- - - - - - - - - - - - We limited testing to serialized SIGINT reports because briefings, litigation, and other types of aissemination were not easily testable using the continuous auditing methodology. .• c.· ...· o.··.·.m.p'.lia. . nt.·.; ff'D//Ol//HF) {b)(1J ············ ············ , ~~~~~~~=~L~:~~~~4J,j~~(rtis!t:t//~sit:tJ;q4>~1F:tJ)IL_____________________ I J...,l (TS//SIN4>tf) "ldent lookup" refers to querying a selector using I. ho determine the approval status. In such cases, the Emphatic Access Restriction controls prevent chah1ing ofa selector that is not marked as approved for querying and return an error message lo the analyst. There is no violation of the BR Order because the selector was not actually chained. 2 TOP SECRETHCOMHH//~mForu;r 5 (b)(3)-P.L. 86-36 REF ID:A4197513 DOCID: 4230258 'f'Ofl Sl':CU'fh'COMIH'fh'MOF'OttM ST-10-00041 (U) Rating System Description Rating A rating of green indicates that no instances of non-compliance with the BR Order were identified during testing. Any noted scope limitations were related to the application of the continuous auditing methodology, not known control weaknesses. A rating of yellow indicates that although no instances of non-compliance were identified, control weaknesses prevented us from testing the entire universe, as explained in the scope limitations. '\'.·. ?<\~~·\:.}'·-.; .. ,."'',,' :,·. ;·:·-'.:-;·' ,·:-..:.:< _., . "'..''-~.·:-:- -<;.·:};: ;~~;i~~ii~it~tJW~:> A rating of red indicates that one or more instances of non-compliance with the BR Order were identified during testing. TOP SBCRBTkC~HNT//~mPORN 6 REF ID:A4197515 DOCID: 4230259 NATIONAL SECURITY AGENCY/CENTRAL SECURITY SERVICE Further dissemination of this report outside NSA Is PBOWB/TED without the approval of the Inspector General. INSPECTOR GENERAL REPORT (TSHSIHNF) Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records December 2010 Test Results (ST -10-0004K) 12 January 2011 pproved for Release by NSA on 08-06-2015. OIA Case #80120 liti ation Derived From: NSAICSS Class((ication Manual 1-52 Dated: 20090804 Declassify On: 20360/06 'fOP S'ECRE 'f7'7'COMINf17'NOFO-Rl1l DOCID: 4230259 JREF ID:A4197515 'fOP SECR£1i5'€0lvf.FNT;]'P•IOFO&V (U) NSA OFFICE OF THE INSPECTOR GENERAL (U) The NSA Office of the Inspector General (OIG) conducts audits, investigations, inspections, and special studies. Its mission is to ensure the integrity, efficiency, and effectiveness ofNSA operations, provide intelligence oversight, protect against fraud, waste, and mismanagement of resources, and ensure that NSA activities are conducted in'compliance with the law. The OIG also serves as an ombudsman, assisting Agency employees, civilian and military, with complaints and questions. (U) Intelligence Oversight (U) The OIG Office oflntelligence Oversight reviews NSA's most sensitive and high-risk programs for compliance with the law. (U) Audits (U) The OIG Office of Audits within the OIG provides independent assessments of programs and organizations. Performance audits evaluate the effectiveness and efficiency of entities and programs and assess whether NSA operations comply with federal policies. Information Technology audits determine whether IT solutions meet customer requirements, while conforming to information assurance standards. All audits are conducted in accordance with standards established by the Comptroller General of the United States. (U) Investigations and Special Inquiries (U) The OIG Office oflnvestigations administers a system for receiving and acting on requests for assistance and complaints about fraud, waste, and mismanagement. Investigations and special inquiries may be undertaken as a result of such requests and complaints (including anonymous tips), at the request of management, as the result of questions that surface during inspections and audits, or at the initiative of the Inspector General. (U) Field Inspections (U) The Office of Field Inspections conducts site reviews as part of the OIG's annual plan or by management request. Inspections yield accurate, up-to-date information on the effectiveness and efficiency of field operations and support programs, along with an assessment of compliance with federal policy. The Office partners with Inspectors General of Service Cryptologic Components and other Intelligence Community Agencies to conduct joint inspections of consolidated cryptologic facilities. T(}P 811CR:Efl7'CftM1i"ff'1tlNtJ1i'ttlJ.1 rf REF J:D:A4197515 DOCID: 4230259 TOP SECRE'i'?YCOMIN'f//MO'.POitl"I OFFICE OF THE INSPECTOR GENERAL NATIONAL SECURTIY AGENCY CENTRAL SECURITY SERVICE 12 January 2011 IG-11243-11 TO: DISTRIBUTION SUBJECT: (TS//81//MF) Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records - December 2010 Test Results (ST-10-0004K) 1. (T~// ~I//MF) Background This report summarizes the results of our December 2010 testing, using the continuous auditing methodology, of NSA's compliance with the Foreign Intelligence Surveillance Court (FISC) Order regarding Business Records (BR). The Office of the Inspector General (OIG) is using this methodology to fulfill the oversight responsibilities assigned to it in the FISC BR Order. Specifically, from January to December 2010, we are conducting monthly tests of NSA's compliance with certain requirements of the FISC BR Order. Once monthly testing is complete, the OIG will make an overall assessment of whether the controls that are in place to ensure such compliance are operating as intended. 2. ('fS/ f GI/ /Pl"F) Monthly Test Results and Objectives We found no instances of non-compliance with the BR Order for six of the seven objectives tested in December 2010. We did not test the seventh objective-compliance with weekly dissemination reporting requirements-because the new BR Order [BR 10-70, signed 29 October 2010] changed the weekly dissemination reporting requirement to a monthly reporting requirement. Because the report covers the period 20 November to 17 December 2010, not the entire month of testing, we can only conclude that from 1 to 17 December 2010, NSA was in full compliance with the BR Order regarding the accurate and complete reporting of serialized dissemination of BR FISA metadata. (U) See page 5 for details of December 2010 monthly test results for the following objectives: ('f~// ~I// Mff) Access: Were all queries to the BR metadata made by authorized individuals (i.e., intelligence analysts and data integrity analysts)? (U / /FOUO) Reasonable Articulable Suspicion (RAS) Approval of Queried Selectors: Did all queries use RAS-approved seed selectors? Derived From: NSA/CSS Classification Guide 2-48 Dated: 20090804 Declassify On: 20360106 TOP SECRETHCHMINT//NUFORN 1 REF ID:A4197515 DOCID: 4230259 iOl' 5t:i;CIU!i//COMmi/J'.MOFOitM ST-10-0004K • · (U / /F'OUO) OGC Review of U.S. Person Selectors: Did OGC verify that RAS determinations of all queried seed selectors associated with U.S. persons had not been based solely on activities proteeted by the First Amendment to the Constitution? an queries chained to no • (O//REL 'PO USA, F'VEY) Chaining: Were more than three hops? • (U / /F'OUO) Revalidation of Queried Selectors: Were all queried foreign and U.S. person seed selectors revalidated within the Court's timeframes-one year and 180 days, respectively-and approved by an authorized Homeland Mission Coordinator? • (TS//SI/fNF) Weekly Dissemination Reports: Did NSA accurately and completely report disseminations of BR metadata outside NSA? • . (TS//OI//NF) Dissemination of Serialized Signals Intelligence (SIGINT) Reports with BR Metadata: Was all information disseminated through serialized SIGINT reports approved by the Chief of Information Sharing Services (S12) or one of the five other authorized individuals? 3. · (U / /FOUO) We appreciate the courtesy and cooperation extended to the auditors throughout the review. If you .need clarification or additional information, please contact! Ion 9()3-2988(s) or via e-mail at .I orJ Ion 952-217l(s) or via e-mail at " ' .1 .;;:;;;.>"' ,,~3j:P:i:; 86'36 ··· I As....s'""'i_s.,..tan_t,........ In_s_p_e_c..,.t-or-...G,...e_n_e_r_aI_... for Intelligence Oversight TOP SBCRET/ICOMilff//NOPORH 2 DOCID: 4230259 REF ID:A4197515 TOP SECRE'FNCO:tvHMfh':MOFOR:H DISTRIBUTION: D21 SV42---~---~-~--' S12 S2I4r..--~------------. T 12 2"""T"""e'"""ch:-n-:i'"""ca"""'l"""'D"""'i,.....re-c-to'"""r......_ __,....._____,·.... I _J?g.J.f ····· (b)(6) ..... ·.. cc: Director SIGINT Director D4 (J. DeLong) GC M. Olsen sv SV4_ _ _ _..__,_ ·····.. :"-.. ·...... . ·." ·. Sl S2,....__ _ _ _..,,._ ·.·.·.·.·... ..... ·· .. ··· / ! S2I ..... ..·" ..· / T 12·1..-----''-i ...... ... ·· Tl 2""'2....,__ __....., .... T122.,._2r-----.... // OGC IG Poe_ _ _ _........,··_ .. - - - - - SID IG POC TD IG POC,...__ __..,.._ _ __,_...._..... I : Dl D 1,.,___ _ ____, D13 D14 'f'Ofl SECR:E'f//COMINT//NOFORJ~ 3 ST-10-0004K REF ID:A4197.515 DOCID: 4230259 TOP 5E:Cturt'i/COivHM'fh'N'OFOitl~ (U) This page intentionally left blank. TOP 8ECRET//COMHff//NUFOitl~ 4 ST-10-0004K REF ID:A4197515 DOCID: 4230259 'fOP SECRE'f//COtvffiffNHOFORU ST-10-0004K (U) December 2010 Test Results t'fS//SI//NF) Test results show that NSA complied with the requirements of the Foreign Intelligence Surveillance Court (FISC) Order regarding Business Records (BR) between 1 and 31 December 2010. The ratings are defined on the last page of this report. Test Results Area Scope Limitations Rating 1. Access t 3 ... o.f.·:::< ..... ·· ··r·····h···e· ~~~~ seed selectors ........:·~~\\ .Cfoheral·:Cc>unse( ...... · .· '·."'-\' · (OGC).... · f ... quene m lwerehhOh ....... h N··::'l .····::::::: ·U··-..-::: :::setei:Ctors Amendment concerns, as required. ·:/ office. w Qu ... •· ..• -:· .. ···i-..···.....··;;.."··------1------------_,.;.~----------i.;;;.;~~~~ ~i::i!;,i (;:'.: > ~ Ch8i0i~!i $:E::~:!ore man ,~~:: None ~~n!·~~~'.:t::::ss~s·_t=-..,,,,:::-"""--==---+-h-op_s_f_ro_m_th_e_s_ee_d_s_e_1e_c_to_r._ _--1r-----------1b7~~~+1 .. · ·rrre·Qeed selectors queried for ..foreign intelligence purposes were approved··bY.. authorized Homeland 5. Revalidation of Mission Coordinator§ w'l1t · the queried selectors Court's timeframes. The seed selectors that were not approved were queried for data integrity purposes. None 6. Weekly dissemination reports (VV_DRs) We limited testing to serialized SIGINT reports because briefings, litigation, and ofher types of dissemination were not easily testable using the continuous auditing methodology. 7. Dis semi nation · of serialized SIGINT reports with BR metadata I (TS//Slh"l'W) c. o. mpliant .I I ./ :::.:··1 TOP SECRET//COMINT//flfOFORllf 5 .... ·:.-··· .....· (b)(1) (b)(3)-P.L. 86-36 (b)(3)-50 USC 3024(i) REF ID:A4197515 DOCID: 4230259 TOP SECRET//COMUH//~~OFORH ST- I 0-0004 K (U) Rating System Description Rating A rating of green indicates that no instances of non-compliance with the BR Order were identified during testing. Any noted scope limitations were related to the application of the continuous auditing methodology, not known control weaknesses. A rating of yellow indicates that although no instances of non-compliance were identified, control weaknesses prevented us from testing the entire universe, as explained in the scope limitations. i--------------------__..,.''';.:_, . ·-,·i:'°:'.'::.<,,;.;"':)'··.-·.. _·.. A rating of red indicates that one or more instances of non-compliance with the BR Order were identified during testing. "f'Ofl SECU'f77'COMIMf//tq0f OR:H 6 REF ID:A4197515 DOCID: 4230259 TO·P 8ECRETA 1CO·lrfl1VT1~'!1ilOFORltat{';! the reas.on fortbe d('!lay and provide a.revised target completion date. Status :reports should be senttol !Assistant Inspector General for Follow-up, at OPS 2B, Suite 6.24 7, withip. 15 caJ.endar days after each target completion dat~. ' f .(b)(3)-P.L. 86-36 3. (U / /FOUO) We appreciate theH.<::.ourtesyancf~~~peration extended to the .auditors throughout ther.:~view;··F6radditional information, please dmtact I lon963-0922(s) or via e-mail atl /(~~ George Ellard Inspector General TOP SECREn~tCOMIHTWHOPDRi"r.' I DOCID: REF ID:A4177257 42302·60 1VP SEcRBl 1iCOivfiih 1;H&l ORfl ST-I 0-0004L (U) DISTRIBUTION: DIRNSA DOC (J. DeLong) SID (T. Shea) TD (L. Anderson) OGC (M. Olsen) cc: OGC STr--.1.~~~---~~~~~~~~---1 sv SV4w-~~~-------~~---. SV4,._._~-~-'-'----------S 12 __ S2.--~------~~-- ........, S2I ...._ S2I4 .S309a...-----S3209 S332.-i----...-.,,,......a· . ~)(3)-P.L. 86-36 Tl Tl......__ _ _ _..... Tl2 Tl22 D4IG~P,..,.,...,.,,.r-~"'""7""----/"--f---i--t--~; D4 OG.~C~I-G~P~O~C.,..,...----, '' ' SIDIG IGPOC POC,..,___ _ _ _...,..___...,._____. TD DL SIDIGLI DL TD_REGISTRY DOJ NSDIL-_ _ _ _ _ _ __.!··/··/·. · · · · · · IG D /,...IG_ _ ___, Dll Dll I D12 Dl3 Dl4 ror $f!:Ct<:ET'.· COi'vHNT ·: '1¥0{i0Pu.¥ 6 (b)( ) REF ID:A4177257 DOCID: 42302'60 fOf $1!:CRE foCOlviiivT/;Jv'QFORN ST-I 0-0004L (U) TABLE OF CONTENTS (U) EXECUTIVE SUMMARY ......................................................................................... iii I. {U) BACKGROUND ................................................................................................... 1 II. (U) FINDINGS AND RECOMMENDATIONS ............................................................. 3 (U) FINDING ONE: Querying Controls ...................................................................... 3 (U) FINDING TWO: Dissemination Controls .... ... ...... ............... ... ...... ...... ... ... ... ...... ... 9 Ill. (U) ABBREVIATIONS AND ORGANIZATIONS ...................................................... 15 APPENDIX APPENDIX APPENDIX APPENDIX AP PEN DIX A: B: C: D: E: (U) (U) (U) (U) (U) About the Audit Summary of Recommendations Monthly Test Results of Querying Controls Monthly Test Results of Dissemination Controls Full Text of Management Responses TOP 8ECRET:''COMINT:'.}{()FOR1'l DOCID: 42302 60 1 REF ID:A4177257 TOP SECRm 1>COivIJJv1 /1JVUr UH.JV ST- I 0-0004L (U) This page intentionally left blank. TOf /5ECl?.ET·: 'C(JivII}lh ~ '}f0F:;OIX1'l ii DOCID: 4.2302'60 REF ID:A4177257 TO? :Sf!:CRE11{-:0JviIN11/NOr ORN ST-I 0-0004L (U) EXECUTIVE SUMMARY (U) OVERVIEW (TS//81//NF) This report summarizes the results of our audit of National Security Agency (NSA) controls to comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records (BR). From January through December 2010, we conducted monthly tests of NSA compliai1ce with seven provisions of the BR Order to determine whether controls were in place and operating as intended. Five of the provisions were related to querying ai1d two to dissemination. (b)(3)-P.L. 86-36 (U) HIGHLIGHTS . . (TS//SI/OfF) Querying controls were adequate to provide reasonable '\ assurance of compliance with the five provisions of the Order that we tt~.sted. NSA's June 2010 rele_c,t.~e.od I a new selector-tracking applicati~n, corrected co11:trolweaknesses that we identified in our Pilot Test Report (iG1.J.1545:.:rOr Tests of controls resulted in a low error rate ofl J .········· ·· ..... bne error ?..~.<;1.l::r.:r..~.d.heforel !release; none occurred after. NSA ........................ ma:11ageiiient must remain diligent in monitoring these controls ai1d ... ,,:::::::·::·:"·:·· ensuring that they remain effective. (b)(3)-P.L. 86-36 ('t>RfF:::::::·:::::. (b)(3)-P.L. 8()::36·· ·· .... ('PS//81//NF) Manual controls over the dissemination of serialized Signals Intelligence (SIGINT) reports and the compilation of the Weekly Dissemination Report were inherently risky. However, risks of noncompliance with the two provisions of the Order that we tested were .....ma:nageable···given·-the . amount·oHnformation. disseminated·I I during 2010). Tests of controls revealed no instances of non-compliai1ce. All ···. Oserialized SIG INT reports derived from BR metadata had been approved by an authorized official and included ·in Weekly Dissemination Reports. (TS//81//NF) The manual dissemination controls will be increasingly difficult to mai1age if the amount of information disseminated outside NSA increases. A recent chai1ge to the BR Order that removes the limit on the number of analysts authorized to access BR metadata will likely increase BR-related dissemination if im lemented. As art of a two- hase 1011 to ..,:::.:.::::.:··:::.:::::: ........ ··············· (b)(3)-P.L. 86-36 query BR metadata, the Counterterrorism Production Center (S21) began training ai1alysts inl I Recognizing the ai1alytic limitations, NSA plai1s to seek relief on foreign dissemination tracking requirements through a motion to amend, which in turn will lessen the compliai1ce burden ai1d risk in this area. (U) Management action (U / /~) The SIG INT Director concurred with the Office of the Inspector General recommendations. In addition, the Office of the Director of Compliance, Technology Directorate, and Office of General Counsel concurred with the recommendations assigned to them as the secondai·y action officers. The plaimed actions meet the intent of the recommendations. TOP SECRET'.('QU,WT 1'N.(),F()ll) T ;;; 1 DOCID: 42302·60 REF ID:A4177257 TOP SEC'lcET;. COiviiiv·r;iffJ-FOttiv· ST-! 0-0004L (U) This page intentionally left blank. Tt9f ,'5ECRET· :, 'C01vfJl{Tr :. ?iOFOR1'{ iv DOCID: 4230260 REF ID:A4177257 TO? 8f!:CREFCO:ivfJNJ;;HOfiDfb.Y ST-I 0-0004L I. (U) BACKGROUND (TSHSIHNF) The Business Records (BR) Order (T8//81//~LF) Pursuant to a series of Orders issued by the Foreign Intelligence Surveillance Court (FISC) beginning in May 2006, the National Security Agency {6)(1}:::::::: ........... ~has been analyzing certain call detail records or telephony metadata from (b)(°3)~P.L~·gs;36::.:::::... ··L__Jtelecommunications providers. NSA refers to the Orders collectively as the (b)(3)-SO)JSC 3024(if·· ·······"BR.Order" or "BR FISA." ffS/J'~~·;;;~;l The.BROrd(':r provides NSA access to bull( call detail records that primarily include records of telephol1.~ calls between the United States and abroad or wholl within the United States··· This collection of ....... information is not wholly available to NSA through its other foreign intelligence information collection. It is valuable to NSA analysts tasked with identifying potential threats to the U.S. homeland and interests abroad because it enhances analysts' ability to identify, prioritize, and track terrorist operatives and their support networks in the United States and abroad, primarily using call chaining analysis techniques. ...,_.,,...~...,.,.~...-~~...,,... ~--,.,.....,,.-.-~....,,.,,,..,...---~--..-...,.... (TSHSINNF) Provisions of the BR Order (TS// SI// MF) The Order defines a series of requirements that NSA must follow to protect the privacy rights of U.S. persons (USPs). To access this data, a designated approval authority must conclude that, because of factual and practical considerations of everyday life on which reasonable and prudent persons act, facts give rise to a reasonable articulable suspicion (RAS) that an ..............................identifier-I I =~~:~--~~~~~~~~~~~~~~~~~~~~~~~~~~- (TS//SI//NF) The provisions of the Orders issued for the first 10 months of our review remained constant. On 29 October 2010, substantial changes were made to the BR Order to be consistent with the terms of the new Pen Register Trap and Trace Order issued in July 2010. The most significant changes related to our review were the elimination of restrictions on the number of analysts allowed to access the BR metadata and a requirement for monthly rather than weekly reports of BR-related dissemination. We adjusted our test procedures for November and December 2010 data to accommodate the changes that affected our monthly control tests. (U) Tests of Controls Using Continuous Auditing (TS//81/0IF) To assess the effectiveness of NSA controls for complying with the BR Order, the Office of the Inspector General used the continuous auditing methodology, performing monthly tests of NSA's compliance with select requirements for 12 months. Continuous auditing is one of many tools used 1 DOCID: 4230260 REF ID:A4177257 ST-I 0-0004L within the audit profession to provide reasonable assurance that the control structure surrounding fill operational environment is suitably designed, established, and operating as intended. Details on the scope and methodology we used to test controls are in Appendix A. TOP 8ECRET~'COMN.~F:~::. ~:;;~~::: . ::::::.::.:(ii/;s1//M.Ff.coiitro1S. were·1n-place··inl....- - -....l to ensure that only cleared "·· . .,.,.. ersoi'ii'l:el·w:ere able to access BR meta.data. These controls compared . ,,..,_ ·..... lo -i~~5···v:iith groups and prevented access to those not (U/tP:OU8' Weaknesses in querying controls have been corrected (TS/ /SI/ /NF) During pilot testing, we identified weaknesses that precluded the use of automated controls to ensure compliance with two of the five requirements that we tested: OGC review of USP selectors and revalidation of selectors. Specifically, the BR FISA Database that NSA used to track the approval status and justifications of BR selectors was not designed to tag and track selectors associated with USPs and selector revalidation dates as separate and distinct fields. i As a result, NSA increased its risk of non-compliance with these requirements, and the scope of testing on selectors associated with USPs was limited. (TS//81/0TF) On 25 June 2010, NSA corrected these weaknesses by replacing .................:the . BR..FISA.Database withl Ia new selector-tracking application. {6)'('3Y,:P:i:;::3&~:3& ····I I created the required data fields, allowing NSA to implement automated controls that prevent analysts from querying for foreign intelligence purposes 1) USP selectors without an OGC review and 2) expired selectors without revalidation. ('ff5//f5I//MF) We found no wealmesses in automated controls over the remaining three of the five querying requirements that we tested: access to BR meta.data, RAS approvals, and chaining. (U/f.FOUeJ Querying controls were adequate to provide reasonable assurance of compliance with the Order (TS/f SI//nF) Querying controls were adequate to provide reasonable assurance of compliance with the Order for the five re uirements that we tested. Monthly tests of 2010 ueries of BR metada ta in •:!" ..(b )(3)-P .L. 86-36 Ohe error occmred ._b_e_f_o-re-.--------r-e-le-·a.-·s-·e""';·""'rui""···_d_1_1... on=-e-o_c_c_u_r-re_c_l_af_t_e_r_._W ___e_f_o""'u~d one error in the . ····················· (b)(1) (b)(3)-P.L. 86-36 .. queries made in 2010 for one of the fiye HRf~quirements tested. We ha a significant scope limitation in~~sting comp~i~1ce. with OGC reviews of USP selectors before L lretease and no lnn1tat10ns after. (U) RAS approvals (TS//SI/OIF) We found no eU"ors in our tests of coi1trols over RAS approvals. All selectors queried were documented as RAS-approved at the time of the query 1 (TS.'.'St::NF) During pilot testing, we identified a third control weakness not directly related to our testing: the BR FISA Database was not designed to track time-restricted selectors so that analysts could be made aware of the time for which the RAS determination applied as mandated by the Order. I lalsoresolvect·tllis·cOilfrol"" ..... Th)(3)-P.L. 86-36 weakness. We did not include time-restricted selectors in our monthly testing because the provisions of the Order allow for the application of analytic judgment to queries on time-restricted selectors, which is subjective and makes objective assessment of compliance difficult. TOP 8ECRET•-COlvHNT',?v'-OFVR:N 5 DOCID: REF ID:A4177257 42302'60 TOP 8ECRELt~OMINLX01'"0RN ST-I 0-0004L in the BR FISA D_§:t.abase-orl We therefore judged these controls ad.equatettf provide reasonable assurance of compliance with the Order. ........................ fB){s:)~e~k:.::86.~.~~-. (U) OGG reviews of USP selectors ···.::::::::::::::::::::::::::.]TS// 811./NI) We found no errors in our tests of controls over OGC reviews of ····lfSP::~ele.G~ors·. · After release, all selectors queried that had been docuiii"eiit~(fJlt ·····... as being associated with USPs were reviewed by OGC, as document~cfih ··· ... · . However, before June 2010, there was a significant scope limitation in.ouf"testi,pg of OGC approvals. Because the BR FISA Database did not identify selector~ ..assoGiat~d with USPs, we did not know whether our tests included all U.S. selectors. Because corrected this control wealn1ess and we found no instances of non-compliance after its release, we determined these controls adequate to provide reasonable assurance of compliru1ce with the Order. I I (U) Revalidation of selectors .. . . ·(~)(~)-P.L. 86-36 (T~/)SI//MFJ We .~?..~p..dno·etiors in our tests o~·c;mitr~~s to revali~ate sele~to~s after_ lvtas released. All selectors queried had h~en revalidated w1thm the prescribed period, as documented inl 1 Before! Irelease, we found one error that was a brealcdow n in controls, which the Department of Justice (DoJ) National Security Division later reported as a com Hance incident pursuru1t to Rule lO(c) of the F..I.§.G.. ..Rnle.s....of.Procedures;·····2. (U) Access (T:i;//:i;l/j~~) We found no errors in our tests of controls over access to BR metadata. Only authorized personnel, as documented ml lru,:i.d the Special FISA Division's (SV42) list of authorized accesses, queried the EfR·· metadata for foreign intelligence or technical analysis (e.g., data integrity) (b)(3)-P.L. 86-36 purposes. We therefore judged these controls adequate to provide reasonable assurance of compliru1ce with the Order. (U) Chaining (TS//SI/OfF) We found no errors in our tests of controls over chaining. Accorcling..fothel audit logs, no selectors had been chained to more thru1 three hops, except for selectors queried for data integrity purposes as permitted by the Order. In following our test procedures, we did not test these controls from July through November 2010 because we found no errors within the first six months of testing. We therefore judged these controls adequate to provide reasonable assurru1ce of compliru1ce with the Order. I (b)(3)-P.L. 86-36 (U//FOUG7 Periodic monitoring of querying controls is needed (U / /FOUO) Although our evaluation ru1d tests of controls determined that controls were adequate to provide reasonable assurance of compliru1ce with the Order, mru1agement must continue to monitor the effectiveness of these controls in a manner commensurate with risk and value added. 2 (U/~) Because of amendment to the FISC rules, lO(c) incidents are now referred to as 13(b) incidents. 6 DOCID: 4230260 REF ID:A4177257 TOP SECHET"'C'OirrfLVT'HOFORN ST-I 0-0004L (U / /ii'OUSj Monitoring is the final standard for internal control in the federal gove1nment. Agency internal control monitoring assesses the quality of performance over time by putting in place procedures to monitor internal control as part of the process of carrying out regular activities. Monitoring includes ensuring that managers know their responsibilities for internal control and control monitoring. In addition, separate evaluations of internal control should be performed periodically and the deficiencies investigated. The scope and frequency of separate evaluations should depend primarily on the assessment of risks and the effectiveness of ongoing monitoring procedures. Separate evaluations may tal<:e the form of self-assessments as well as reviews of control design and direct testing of internal control. (TS// SI// NF) At the time of our review, SV42 was monitoring the effectiveness of controls by conducting weekly manual reviews of audit logs to ensure compliance with three of the provisions of the Order that we tested. Given the strength of the automated controls sin.eel l-release;mru.1agemerif.. ·(bj{3)-P.L. 86-36 should reassess the timing and extent of these reviews. In particular, we recommend that management base the type, duration, and frequency of monitoring on risk and value added. For example, in lieu of weekly and 100 percent reviews of audit logs, management should consider such options as periodic testing, sampling, event-driven reviews, or automated exception reporting. (U//F'OUO~ Develop a comprehensive plan to provide long-term monitoring of the effectiveness of querying controls. The plan should be commensurate with risk and value added and include the means to manage changes in factors such as personnel, Information Technology systems, software applications, and legal authorities. (ACTION: SIGINT Director with TD and ODOC) (U) Management Response //(b)(3)-P.L. 86-36 CONCUR (TS//SI//tff) Currently, Oversight and Compliance (SV) manually monitors querying compliance with a weekly 100 percent audit of all queries. SV has found no errors in querying since the Emphatic Access Restriction (EAR) was implemented I I which technologically limits the selectors used in a query. SV is in the process of developing a long-term querying compliance strategy aligning the weekly 100 percent auditing with SID's Auditing Modernization Strategy. SV will work with D4 to develop a "Sampling Rigor" to identify a sampling standard by the end of August 2011. In addition, SV will continue to work with.I I (6){3)-P.L. 86-36 developers to fully automate auditin rocedures such as a web-based interface to perform t}J& . audits • ................... ,... ,... ,. ,,,,.,,:;::::··:::::::::: ..::::::::::::··::::.:::::::::: (b)(1) (b)(3)-P.L. 86-36 '--~~~~~~~~~~~~~~~~~~~~~~~~~~~~~---' (U) OIG Comment (U / /FOUO) The planned actions meet the intent of the recommendation. 7 DOCID: 4230260 REF ID:A4177257 TOP 8ECR:t:t::COMiltf'i;j·v·o1' OR.N ST-I 0-0004L (U) This page intentionally left blank. 8 DOCID: REF ID:A4177257 42302 60 1 TOP SECRELCOMINT1;xoFOR:N ST-I 0-0004L (Jj)(1J. .... (b)(3)-P:t;··86::.~6 · \ (U) Fl NDI NG TWO: Dissemination Controls (T8h'-SlHA'FJ .. Manual controls over the dissemination of serialized SIG/NT reports and the compllatiQ_n of the Weekly Dissemination Report (WDR) were inherently risky. However, risks··of..non-compliance with the two provisions of the Order that . we tested were manageab/e·I Although we found no instances of non-compliance, monthly testing identified one error in which a WDR incorrectly included a report that was not derived from BR metadata. This error was not a violation of the Order but underscores that the largely manual process for tracking serialized SIG/NT dissemination and compiling WDRs is inherently risky and would requ.....i.._re_ __ management attention should the amount of dissemination increase. I I I I .. / / / / (U) Criteria Used to Assess Dissemination Controls ... (b)(1) (b)(3)-P.L. 86-36 (TS//SI//Mfi') To protect the privacy rights of USPs, the BR Order includes a series of requirements to track and control information shared outside NSA. With the exception of exculpatory material used in litigation, the BR Order requires th.at all disseminations of USP information derived from BR metadata made outside NSA, whether in formal reporting or in response to requests for information or other forms of communication, be approved by one of five NSA officials. One of the five officials must determine th.at the information identifying the USP is in fact related to Counterterrorism (CT) information and th.at it is necessary to understand the CT information or assess its importance. The BR Order does not state that the authority may be delegated. Until BR 10-70 was issued on 29 October 2010, the Order also required th.at NSA provide the FISC a weekly report of all dissemination. BR 10-70 changed the weekly reporting requirement to every 30 days. (TS//SI//NE) We evaluated the adequacy of controls to ensure compliance with two BR requirements against Standards oflntemal Control in the Federal Govemment and tested the effectiveness of these controls monthly from January th.rough December 2010. 1. !TS/fSI//Nfi') Weekly Dissemination Reports: NSA must accurately and completely report disseminations of BR metadata outside NSA. 2. (TS//gl//~fpt Dissemination of Serialized SIGINT Reports with BR Metadata: All information disseminated th.rough serialized SIGINT reports must be approved by the Chief of Information Sharing Services (S 12) or one of the four other authorized individuals. (TSHSL'i'P4F) Process to Track and Disseminate BR Serialized SIGINT Reports (Tg//SI//WF) The process to track serialized SIGINT dissemination is largely manual and maintained outside the infrastructure used to handle normal TOP S'ECRF:T'.'COMIHT'.'}l&FORtvr 9 DOCID: 4230260 REF ID:A4177257 TOF 8ECRET'i CO:ivfIHTidv·o,-1Diciv· I ST-I 0-0004L reRorts involving the release of U.S. identities. Homelru.1.d Security Analysis Center (S2I4) policy was that serialized reporting ru.1.d S12-approved responses to Requests for Information were the only acceptable forms of dissemination outside NSA. Therefore, e-mails and other informal types of dissemination were not permitted, ru.1.d any information derived from BR metadata required S12 {b){'.1'.):·············"''''''''"'''"''''"'''''''''''''" (b)(3)-P~L;·s6-36.... (b)(3)-50 use 3024(il :::::::::::::::::!:~~~::~~~~~;:.tu:ar. s1%;~:~c~~:~l;e~~!~e~0 ~:~;:~~:e d;~I~; from BR information that res onds to.The··s·····e·crarte··uirements·ofthe- .... ('FS//Sl//NF) The Chief ru.1.d Senior Editor of S2I4 track BR-related reports and submit requests to disseminate information outside NSA to the Chief of Sl2, via e-mail. S12 documents approvals to disseminate through one-time dissemination authorization memos. The Chief of Sl2 signs the memos ru.1.d retains a copy for the record. ('b)(:$.)•P.,L. 86-36 ... ······ ······.... ········· ........ ··.... ·· .... ff8f/8i//PfF) S2I4 manually tracks BR-related dissemination for inclusion in WDRs to the FISC, and SV42 maintains a spreadsheet to track report ····· ........ qissemination authorizations. ....... . . . ,................................................ :::::·······. ..fFS//SI//NF)I I a management information system for SIG INT production ~~~~!·t~pJ::;··s 6 _36 _ ··········c-o?-t~~s··statis~cal--in!o~ation..and.. G.µ::;t.<>..l'.1:1..~!....~~-:,~.?..~~~- ~~-~1:1t _s~:1alized re orts. (b)(3)-50 use 3024(i)········ ... Within dissemmation based on BR anal sis can be 1dentif1ed (Ul/FOUO) The Manual Process to Disseminate and Track Serialized SIGINT Reports Was Inherently Risky but Manageable Given the Amount of Information Disseminated f6ff1) · (C//RBL 1:6 USA, FVB¥) During pilot testing, we noted that the process to obtain ru.1.d document dissemination authorizations for serialized SIGINT reports signed by the Chief of S12 ru.1.d the process to compile WDRs were largely manual ru.1.d, therefore, dependent on the diligence of the staff and the strength of standard operating procedures. During monthly testing, we found one error that underscored this weakness but found no instru.1.ces of non-compliance with the Order. The largely mru.rnal process to disseminate and track serialized SIGINT ......... !.~P.?rt~-~~~ ~~~·efo~e in~eren~y risky but manageable given the relatively small amount or mrorm:a:tio11 .. d1ssemmatedt I (b)(3)-P.L. 86-36 (T8,~'81N~JI=) Weekly Dissemination Reports (TS//SI/OIF) In our tests of controls over the accuracy of the 44 WDRs covering 2010 dissemination, we found that a WDR mistal(enly listed a se1ialized dissemination that was not derived from BR metadata. Although the error did not violate the BR Order, it underscores a weakness in the largely mru.1.ual process to track and report BR disseminations that we noted in our Pilot Test Report (IG-111545-10). ('FS//SI//NF) We did not test NSA compliance with weekly dissemination re ortin re uirements in November ru.1.d December 2010 because the BR Order ......... ................... . ........ ······ changed the reporting requirement from weekly to (b)(3)-P.L. 86-36 every 30 ays. Because the 30-day reports did not correspond with our IO DOCID: 4230260 REF ID:A4177257 TOP SECRELCOivfJNJ;;xorof&of ST-I 0-000"1L monthly testing, we were unable to draw a conclusion about whether NSA was in full compliance with the BR Order regarding the accurate and complete reporting of serialized dissemination of BR FISA metadata for the period from 1 November 2010 to 17 December 2010. (U//FOUO) We judged the manual process to compile WDRs to be inherently risky but manageable given the small amount of information disseminated. (U//FOUO) Dissemination of Serialized SIGINT Reports ('fS//Gi//WF) We found no errors in our tests of controls over approvals of ..................................................... ······serialized··-SIGINT-reports;···AHQreports issued in 2010 had been approved by 3 35 35 (b)( )-P.L. the Chief of Sl2 or one of the four other authorized individuals. We judged the manual process to track serialized reports as inherently risky but manageable given the small amount of information disseminated. (U//FOUO) Manual Processes Will Not Be Manageable if the Amount of Dissemination Increases (T~//£I/0lF) We noted in our Pilot Test Report that management should consider automating dissemination approvals and tracking if BR-related dissemination increases. A change to the provisions of the Order signed on 29 ·..fj Octob_er 2010 (BR 10-70) might significantly increase the amount of information ('")('1:).,,,,,,,,,...... disseminated. Specifically, BR 10-70 removes the limit of 125 analysts (b)( 3)-P~L::~~~~6... . (b)(3)-50 usc···SO?~{i). . ...... _authorized to qu~ry B~ m~tadata ?ut maintains re~uirements for NSA to report .............. · "··· . .to··th~. FISC all dissemmation outside NSA. Before issuance of BR 10-70, only .......... ·E:]pers:C:i:~s-··ha.4._query access to the metadata, well below the 125 limit. The Chiefof.. S'.214·eS._timated. that if NSA implements this change, S214's query access might expancl. !?. tlte[""]p~rson..P.:~1.. already authorized to ~i~~. query results and to the more thaitc::=ranalysts wrthl lcT responsibihties. Mai1ual processes would not be maiiageable if the amount of dissemination increases with the number of analysts authorized to query. ~:~{!~~P;L.:·a6~3Ef ......... (b)(3)~50_U.SC--~Q24(i) ··.. _(T~//£I/0fF) S2l issued a Staff Processing Form (SPF)I I··· . . (6)(3)-P.L. 86-36 annou11cing_plai1s to ex ai1d BR and Pen Re ister ai1d Tra and Trace PR TT) access to uer ·-resU:lts The Chief of S214 stated that the expansion relies on training and controls, such as the EAR ai1dl Ito ensure . co:i:µpliai1ce. The SPF states that training I I ....... . (b)(3)-P.L. 86-36 fFS//Slj/~!F) NSA OGC recognized that increasing the number of ai1alysts authorized to query BR metadata and expai1ding access to BR query results might increase the risk that informal disseminations would not be documented ai1d therefore, would be untracked and ultimately out of compliance with the Court Order. Specifically, the BR Order requires formal, documentable tracking of foreign-target BR disseminations, a practice th.at runs counter to traditional NSA ai1alytic process and hence requires additional, non-standard training to accomplish. This practice also constrains the full ai1alysis of bulk metadata. NSA OGC is therefore drafting a motion to amend the BR (and PR/TT) Order. A draft of the motion, I lst~tes that NSA seeks relief from the TOP SECREP'.'COMEVT:'.'NOF01rNll (b)(3)-P.L. 86-36 1, DOCID: 42302'60 REF ID:A4177257 TOP SECR:t 1,; COiviiN1, 1NOr OR1v ST- I 0-0004L requirement to include in a 30-day report "a statement of the number of instances since the preceding report in which NSA has shared, in any form, information obtained or derived from the metadata with anyone outside NSA," only to the extent that the dissemination applies to non-USP information obtained or derived from metadata. NSA OGC expects to file the motion soon, but there is no definite period, and it is uncertain whether the FISC will grant the motion. Hi-tm~+Nt-.. Develop a plan to mitigate the risk of non-compliance with the Order in disseminating information derived from BR metadata when expanding access to BR querying and results. (ACTION: SIGINT Director with ODOC and OGC) (b)UJ (b)(3)~P.l-. 86-36 (b)(3)~5o. (U) Management Response US.G 3024(i) · .. CONCUR (TS//SI//NF) SID acknowledges that BR Order compliance, in .. " . ·. ........ ··.. "·... terms of tracking and controlling the dissemination of BR-derived . .._information, is indeed essential as the Signals Intelligence Directorate expqnds BR and Pen Re ister Tra and Trace PR TT access to uer . resuitsw1 e mcremen y execu e 111 a me o 1ca manner to mitigate t e ns of non-compliance. Expansion of access and use of the BR and PR/TT authorities will be conducted apace of the requisite compliance and oversight infrastructure to minimize the risk of incidents and violations . .Tra.i.P.-.i.1."I:~ serves as one of the key elements for success and much progress has beeii . iii.ade·;··1 I SID expects the launch of the National Cryptologic School course OVSC 1205. This course incorporates required Office of the General Counsel (OGC) indoctrinations for both BR and PR/TT with analyst focused material. Com letion of OVSC 1205 will ·r.edentials and constitutes be the basis for granting the a prerequisite for the granting of the upon nomination by a production element manager ····.f justification. (b)(1) (b)(3)-P.L. 86-36 (TS//SI//WF) In addition, SID fully understands the Office of the Inspector General's concerns regarding tracking disseminations of BR-derived information and continues to evaluate approval and dissemination processes to optimize current procedure, where possible. When planning to disseminate information derived from BR FISA, USP information must be identified and its dissemination (with a report) must be reviewed and approved by the Chief or DI Chief S 12 (or one of the other offi.cials as noted in the BR Orckr.) .. j I ···:..... ······················ .......... ,,.:::::::::::::::::::::: ....... (b)(1) (b)(3)-P.L. 86-36 (b)(3)-50 USC 3024(i) Although non-USP disseminations do not require prior approval from S 12, known disseminations shall continue to be documented as they occur and 12 DOCID: 4230260 REF ID:A4177257 TOi" 8ECRELCOivfliv7·1;tfOFOtW ST-10-0004L .... -· ....... -· (b)(1) (b)(3)-P.L. 86-36 (b)(3)-50 USC 3024(i) reported, along with disseminations of USP info, every 30 days to the FISC. As BR is expa11.d.ed·I !process shall be expande.d.. to·ensure the same degree of controls and oversight. Expansion . . :wm··be. .gradual to ensure requisite training is conducted and control and compliance processes are in place, and SID shall consider further controls/process enhancements (such as further automation) as the situation warrants. S2I shall ensure expansion remains aligned with current scalable processes that have resulted in substantive compliance with the Court's order to date. * (U) Refer to Appendix E, Pages 3 through 5 for full text of SID 's management action plan for Recommendation 2. (U) OIG Comment (U / /FOUO) The planned actions meet the intent of the recommendation. TOP 8ECR£T'.C01vHNT''NOFOm.1.~.. i:tf!iculable suspicion approvals documented in the Foreign .,,,,,,,. ....Tiifellige11ce ·slli;v.eiifaiic:e.::A~(BR'd·atabase·orl I and Office of General ·::,. Counsel OGC) reviews docuniente·a·mthel lor ···::>: .. We also counted the number of hops chained for each selector in tiie audit logs. We researched anomalies to make a final determination of compliance. (U / / PeUe-)" For tests related to dissemination, we verified that serialized SIG INT reports issued in 2010 were supported by dissemination authorizations. We also reviewed Weeldy Dissemination Reports and supporting documentation. (U / / filOUe-)" We met with individuals from OGC, the Office of the Director of Compliance (ODOC), the Signals Intelligence Directorate (SID), and the Technology Directorate, including the SID Office of Oversight and Complia11ce, Information Sharing Services, Homeland Security Analysis Center, SID Issues Support Staff, Analytic Capabilities,! landl I Operations. , . ; / ": (b)(3)-P .L. 86-36 A-1 DOCID: 42302·60 REF ID:A4177257 TOP 8£CRET:;COivfINT:;HQFORN ST-10-0004L (U) We conducted this performance audit in accordance with generally accepted gove1nment auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions according to our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions according to our audit objectives. (U/,'FOUO) Reports Issued {'f8//8I//NF} Advisory Report on the Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records (ST-100004), 12 May 2010 (TS//~I//~lFt Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records -January to March 2010 Test Results (ST-10-0004A}, 1 June 2010 (TS//Sl//NF) Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records -April 2010 Test Results (ST-10-0004B), 10 June 2010 (T:?3//SI//Nii't Audit Report of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records - Control Wealcnesses {ST-10-0004C), 29 September 2010 (TS// SI// NF) Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records - May 201 0 Test Results (ST-10-0004D), 30 June 2010 (TS//81/0fF) Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records - June 2010 Test Results (ST-10-0004E), 20 July 2010 !T:?3//Sl//NF} Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records - July 2010 Test Results (ST-10-0004F), 18 August 2010 (TS//Sif/NF} Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records August 201 0 Test Results (ST-10-00040), 28 September 2010 ('FS//SI//NF) Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records - September 2010 Test Results (ST-10-0004H), 28 October 2010 (TS//81/0IF) Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records - October 201 0 Test Results (ST-10-0004I}, 1 December 2010 (Tg//81//NF) Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records - November 2010 Test Results (ST-10-0004J), 20 December 2010 TO? /.SECRET·:. (?Olvfl}ifT, :, 'J{QlYQR}{ A-2 DOCID: 4230260 REF l:D:A4177257 TOP SECRELCOMIJ'{f,;f{QTOt?:J"NOtOR:N ST-I 0-000'-f.L (U) This page intentionally left blank. TOP 8F:CRF:T':'COA1L\T'.'NOFORN DOCID: 42302'60 REF ID:A4177257 TOP SECRET;CCJMHiT;ffOT'OitH ST-10-0004L (U) MONTHLY TEST RESULTS OF DISSEMINATION CONTROLS 1 2. Dissemination of serialized 0 SIGINT Total No. of WDRs Total No. of Serialized SIGINT Dissemination (b) ( 1) (b) (3)-P.L. 86-36 *~:VONGIN~ll=l Not tested because the reporting requirement changed from weekly to every 30 days. {Tf/;'81//IWr (b)(3)-P.L. 86-36 D-1 DOCID: 42302 60 1 REF ID:A4177257 TOP /5£C1?.ET.·:,·cOi'viINT ·;·woFOl?:}f ST-10-0004L (U) This page intentionally left blank. TOP 8ECRET'.1COMINT'.'NOFfJRN D-2 DOCID: 4230260 REF ID:A4177257 T01° SECmj11;coJvfiN11!NOrOR.N ST-I 0-0004L (U) APPENDIX E (U) Full Text of Management Responses TOP SECRET'.'COML\rT''N014{)R·N DOCID: 4230260 REF ID:A4177257 TOf ,~E:CRm 1T~OivfiN11dv'OFORN ST-I 0-0004L (U) This page intentionally left blank. TOP SECRET'.-'COM:INT'.'HO-F-O·R:N DOCID: REF ID:A4177257 4.2302 60 1 ror S!!:CR.Bl ,~CO.lviINl 1;N01 OR:H 1 ST- I 0-0004 L l Ul' i!ieUH::Llh'UUMm l/J'fJUI Ul<:f4 SECURITY CLASSIFICATION NSA STAFF PROCESSING FORM TO OIG THRU I I KCC CONTROL NUMBER E!Xft.eO CONlROl..\\iUMBSR 2011-3073 ACTIOM ~s~u~"'"'""'EC~T~~~~~~~~~~~~~~~~~~~~~-i (U) SID'S Management Response to the DR.D..FT IG Report for ST-10-0004L. 0 EXREG SUSPENSE APPROVAL 15 Apr 11 KCC SUSPENSE D D SIGNATURE ELEMENT SUSPENSE'. INFORMATION DISTRJeUTIOH S, S02. 8023, Sl, S2, 83, ST, SV, D2, D4 SUMMARY PURPOSE. (FS//Si/;'nF) To provide the Signals Intelligence Directorate's (SID's) response to the DRAFT Inspector General Audit Report on NSA Controls to Comply witl1 tlie Foreign Intelligence Surveiliance Court (FJSC) Order Regarding Business Reoords (BR) (ST-10-0004L). BACKGR.OUND: ff5//Sf//}~f} SlD received the DRAFT audit report which summarized the results of the lnspector General's year-long review of NSA Controls to Comply with the FISC Order Regarding BR Although the querying and dissemination controls tested were adequate to provide reasonable assurance of compliance with the terms of the order, two recommendations with SID Lead were documented in the report. DISCUSSION: (U/ /'flOUO' SID reviewed the document for.factual accuracy and concurs with the document and recommendations as written. SID Oversight and Compliance (SV) will lead the effort for Recommendation 1, with SID'~ . ~pearheading the effort for Recommendation 2. The Office of General Counsel agreed to be a secondary'action officer to assist SID in its development of a management action plan for Recommendation 2/The SlGINT Directorate's coordinated response is attached. · End: a/s jt.q)(3)-P.L. 86-36 ,l "!' COORDINATIQ~IMPROVAL 50232 lih.:_,)j I Sl :iMayU D2 82 3Mayl1 D4 ORG. I Ism IG Liaison FORMA6796 REV NOV9S l2B~rn· 1 SV S0232 Derived From: NSA/CSSM 1-52 Dated: 20070108 OeclasslfJI On: 203(ti10~ 12Mi'ly11 f29Aprll PHONE (Socur11) 966-5590 DATE PREPARED .. I 04 May 2011. SECURITY CLASSIFICATION fOP E-1 SEC:R:~'fWCOMIH'fti~~OfORJvork __, de\~elopers to folly automate auditing procedures such as a web-based interface to perform the audits. U1is longterm plan will take into consideration the expansion of BR and PRTil....... · · · J· I (3)-P .L. 86-36 DOCID: REF ID:A4177257 42302,60 TOP SECRET.; 'COlvflNT;;Horol&f ST-10-0004L TOP SEERET//COMINT//NOFORr~ ~~iljjjjj:~~tt~~ ~--::=/ ,'::,','-::~-::;Develop a plan to mitigate the risk of non-compliance with the Order u1. d.issenilli.ating i:nforni.ation derived. from BR metad.ata when expanding access to BR quet.ying and results. ,.. .. ".(b)(3)-P.L. 86-36 (ACTION: SIGINT Director with ODoC am:,l.cfGC) SID Leadl ,,.... .. Concm"fNon-conctu-: CONCUR as written /(~)(1) / (b)(3)-P .L. 86-36 (b)'{~)-50 USC 3024(i) / - . SID Response (April 2011): (TS//Gi//?.W) SID ackno·wle.,dges that ~R Order compliance, :in. terms of tracking and contrnlling the disq.~~i.ination of)3R-derived :information, is indeed essential as the Signals Intellige~{~e Directorate ~,Jv,ue of progra.il.1 backgronnd a.s '\·'\'ell as cti:.iTent enviro1unent the FISC Jroduction ele111ent inana em credentialedJ based on re"c01nrnendations .._.,.....,~....,----------------------------0 f the £onner . . (TCi//•:,±//NF) SV oversight protocol for auditing, DO} spot checks, ,utd 30 d.:tv reports reva111ped to include unique ch,,.llenges th.at cotild arise bclsed lln._I_ ___. access a.nd use \ ! . ·· ·· (TS//SI//Nt') ODoC in prngress reviei;v of each increme11.taJ stt;:>.p t~ ensure co1npli,u1ce re1nain:. apace of irnple1nentation . .· (b)(1) (b)(3)-P.L. 86-36 ,..,~i th . . .<:: (b)(3)-P. () 0 (b)(3)-P.L. 86-36 (b)(3)-50 use 3024(i I TOP SEERET//COMINT//NOFORN T01° /?19CRET. ·ctJlvlf}{T,.'~}{Q1VQl?:lel E-4 DOCID: 42302·60 REF I:D:A4177257 TO? 8ECRETi:COiviJNf;:NQfiDJ("H ST-10-0004L TOP §ECRET//COMINT//NOFORN (U / /Ji'OUO) SID Points 6£ Contact: (UI /'.f!CUO) sm..-L-..ea....d.._~....__ _ _ _ _ _,_,,_,...,....,,,..,,..,.,,___ _ _-.,.::1..I (UI /FOUO) 51 ~----------.... 1·966~3.9.J?............ . (U/ f FOUO) 52 1··969-0224.....................:::::::;m.,,,.(U / /E*OUO) 53 l969~069~f:::::·::::::::::::::::>··· (b)( 3)-P.L. 86 -36 (U//FOUO) SV - l969:53i3°i3° ........ ... ·· (U / /FOUO} Coordinated with: ODOC -f.____....l:A~sistant Dit:ector fo1· Special Compliance Activities, D4, 963-1705 (U//FOUO) OGC concurs with Recomm.endation 2 and '\·dll work with both SID and ODOC to prepare a managem.ent action plan. TOP SECRET//COMINT//NOFORN TOP 8ECRET'.f'OM1Nn;NOFO!i.N E-5 DOCID: REF ID:A4177257 4.230260 TOP 8£CRET;'COMH'ff,·;·NOFOF6.Y ST- I 0-0004L b.s~-011 USA CIV From: Sent: To: Cc: Subject: Attachments: Follow Up Flag: Flag Status: Classification: TOF' Sl:!C~E'ftteciMrr41 ffl40P'ORN .-------.1/ / . . -· ODOC (D4) concurs with Recommendations 1 and 2 (see attached form). NNNNNNNN~NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN Rceommcndation 1 : Action: Concur/Non-concur: Mgmt response: ,___ _ _las.p4 POC Concur - with ciaNfication that SID has the lead The D4 POC will work ~itffSID.(who has the lead for the action) in the development of an Action Plan .......... . Completion date: For SID (as action lead) to define ........:::· "·-·:::::::;!ii•·(b)(a)-P.L. Rceommcndatfon 2: Action: Concur /Non-concur: Mgmt response: Completion date: 1 ... ···· -............ ........... 1 .... ·· . ......·.:: ·.·."' "--c-on-c-ur____.with clarifrcati~1_1..thafs~.D ha~.tl1~ lead The D4 POC wfll llJ.Prkwith SJIY(whc;dfos the lead for the action) in the developme.ntof·a·~ Actio.n·Pl~n ,... .... For .§J:D(as action le.ad)"to def(ne; Thankr~:·····//.//···· ... ······· (U//FOUO) I I / Office of the. Director of Cornplia11ce; D4 OPS 2B, Rtn. 2B8054, Suite. 6242 963-2199 ~ 86 36 I fOf-1 3f:CRt£T: COlvflNT'.?JOF::OR-H E-6 DOCID: 4230260 REF ID:A4177257 ~~ECREl f0l1 ,/COivfiH11/HOr ORJV ST-10-0004 L ~.$A-D11 USA CIV From: Sent: To: Cc: Subject: Follow up Flagged Follow Up Flag: Flag Status: ·· .. ·.. \ .. Classification: COPdFIDEfffli!llL ii .·· ..... ·· ·:>\,{~;(.3)-P.L. 86-36 .... ···· (U//~I l····ap;·l·ogize for the clelay, I was out yesterday. Tl, TD DoC and TE6 have reviewed the report and all concur and do not recommend any changes. Specifically to REC 1, they all concur as well. REC No - REC 1 Action SIGINT DIR with TD and ODOC Concur with the recommendation Mgmt Response - TD has no comments to present on the audit. Completion Date - To be determined by all parties as the actions within REC 1 begin to take place. POC - -I -I -I Tl TD DoC TE6 Thank you, I 1. ,....... ......· I.. ····:::: .... ·· ;. (b)(3)-P.L. 86-36 :::::::'ii!!i""b)('>) PL 86 36 .IN·sA"o1··1 .. usA··c1v·······..·······.......................... · .................. ··············::::;;;:::::·· \ . . ... ~ ~.- From: Sent: To: Cc: Subject: Follow Up Flag: Flag Status: Follow·G~ __ flagged Classification:TOP SEGRET/fGOMIPHh'NOFORN Hic:J / Please use this email as confirmation that OGC wants to be added to Recommendation 2. OGC concurs with Rer.0111111Pnd;itinn 7 ;rnd will work with both SI[) ;ind Of)Of to prPpnrP ;i m;inagPmPnt ;ictinn pliln. Pleas!' ll't me know if you nPP.d ;inything furthPr. Thanks. E-7 DOCID: 4230260 REF ID:A4177257 !Of! S!!:CREl 1d-:OJvfIN1 .1~NOr ORN ST-10-0004L (U) This page intentionally left blank. TOF 8F:Cfi:F:f ;colvIIlvT; NOFOft:N E-8 D-OCID: 4230260 I REF DOCID: 42302 62 1 REF ID:A4177267 TOP SECRE'f77'8Ii7'NOFORN NATIONAL SECURITY AGENCY/CENTRAL SECURITY SERVICE INSPECTOR GENERAL REPORT (TS/ISIUNF) Report on the Audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records Retention ST-11-0011 20 October 2011 @\pp roved for Release by NSA 011 08-06-2015. FOIA Case #80120 {litigation J DOCID: 423026.2 REF ID:A4177267 TOP SEC1tE'f'!l'8I!lNOf101tN (U) OFFICE OF THE INSPECTOR GENERAL (U) Chaitered by the NSA Director and by statute, the Office of the Inspector General conducts audits, investigations, inspections, and special studies. Its mission is to ensure the integrity, efficiency, and effective1iess ofNSA operations, provide intelligence oversight, protect against fraud, waste, and mismanagement of resources by the Agency and its affiliates , and ensure that NSA activities compl y with the law. The OIG also serves as an ombudsman, assisting NSA/CSS employees, civilian and military. (U) AUDITS (U) The audit function provide s independent assessment s of programs and organizations. Pe1formance audits evaluate the effectiveness and efficiency of entities and programs and their internal controls. Financial audits detennine the accuracy of the Agency's financial statements. All audits are conducted in accordance with standards established by the Comptroller General of the United States. (U) INVESTIGATIONS (U) The OIG administers a system for receiving complaints (including anonymous tips) about fraud, waste, and mismanagement. Investigations may be undeitaken in response to those complaints, at the request of management, as the result of irregularities that surface during inspections and audits, or at the initiative of the Inspector General. (U) INTELLIGENCE OVERSIGHT (U) Intelligence oversight is designed to insure that Agency intelligence functions comply with federal law, executive orders, and DoD and NSA policies. The IO mission is grounded in Executive Order 12333, which establishes broad principles under which IC compon ents must accomplish their missions. (U) FIELD INSPECTIONS (U) Inspections are organizational reviews that assess the effectiveness and efficiency of Agency components. The Field Inspections Division also pa1tners with Inspectors General of the Service Cryptologic Elements and other IC entities to jointly inspect consolidated c1yptologic facilities. DOCID: 4230262 OFFICE OF T}JE INSPECTOR GENERAL NATIONAL SECURITYAGENCY QE;N1'RAL SECURITY SERVIC.E 20 October 2011 IG-11345-12 TO: DISTRIBUTION SUBJECT:. (TS// Si//NFJ Report on the Audit of NSA Controls t() Comply with the Foreign Intelligenc.e Suryeilhl!lce Court Order Regl:lidingB11siness Records Retention (ST-11-0011) -.. ACTION MEMORANDUM 1. (U) This report summarizes the results of our audit of NSA Controls to Comply with the Foreign Intelligence Surveillance Court Order Regarding Business ·Records Retention and incorporates management's rnsporise .1:o. tJ:ie draft report. (U//FOUO) A.s .requireci by NSA/CSS Policy 1,.60, NSA/GSS Office of the Inspector General, actions on OIG audit recommendations are subject to monitoring and follow,..up until completion. Therefqre; we asik that you prqvkle a, written statµ§ report conc;erning each planned t:orrective actibn categ():rized as "OPEN." If you propose that a recommendation be considered closed, please provide sufficient information to show that actions have been takeQ to correct the deficiency . .If a planned action will not be corr:tp~eted by the original target completion date, please state the reason for the delay and provide a revised target completion date. Status reports should be sent Joi...._____, I lt\ssistantlnspector General for Follow-up, at 0)?~_.2B·~ Suite 6247, within 15 .cal~ndar dGl.ys after each target completion cJ,ate·. 2. ·· .. .... ······· 3. (U //FQUO) We appreciate the coyrte~y and cooperation extended to the auditors throughout the review. For.additional information, please con tac~ lo.~ 963-092~(s) or via ~~~c=.i.i.l.atf....._ _ _ _ _ ___. ·::,,,'"· ...... ~,:::::::: :::.::: .......................... ············ (b)(3)-P.L. 86-36 ArCEleuiJ George Ellard Inspector General 1'0P SECltETt/Slf/N·O-FOR:N REF ID:A4177267 DOCID: 4230262 'fO:P SECRE fJ7'8117'N'OFORbl ST-11-0011 (U) DISTRIBUTION: DIRNSA SID (T. Shea) TD (L. Anderson) cc: EXDIR (F. Fleisch) COS (D. Bonanni) DOC J. DeLong) D4 ______.._..__ _ _ _ _ _ _ ___, OG~C ST svi------__. ___ , SV4 .....,... SV42 83132.....3~----...;;;....-_..;;;._.;;.;._......;o.....,;;_.__.. S353 _ _ ___,, TE,........ TE6 Tl,.....__ _ _ (~)(3)-P.L. ~ Tl2 T121o.-r------__., Tl222 T131.....___ _ _ _ _ __,,........_.___, Tl31 L-___r--...._------~':"'-------~ D4 IG SID IG POC TD IG POC ...----------";__i DL SIDIGLIAI DL TD _REGISTRY DOJ NSD _,_ _ _,,,___ _ _, . (b)(6) IG D/I_G_ ___ Dlj Dll D12 D13 D14 I 86-36 R.EF ID:A4177267 DOCID: 4230262 TOP 8ECR£1WSlt7'NOFttRN ST-11-0011 (U) TABLE OF CONTENTS (U) EXECUTIVE SUMMARY .....................................................................................................iii I. (U) BACKGROUND ............................................................................................................. 1 II. (U) FINDING AND RECOMMENDATIONS .......................................................................... 9 (T8//81f/NF) FINDING: BR Retention Practices Must Be Documented ................................ 9 IV. (U) SUMMARY OF RECOMMENDATIONS ........................................................................ 17 V. (U) ACRONYMS AND ORGANIZATIONS .......................................................................... 19 (U) APPENDIX A: About the Audit (U) APPENDIX 8: Testing Methodologies and Results • • (U) Selector Pair Testing (U) Metrics Analysis (U) APPENDIX C: Full Text of Management Response DOCID: 4230262 REF ID:A4177267 TOP SECR£'f17'Sfl/NOT'OltN' ST-11-0011 (U) This page intentionally left blank. FOP 8ECR£'f1S'8f;S'PfOFOR:1V ii REF ID:A4177267 DOCID: 4230262 FOP SfJCR£T1S'8hS'N{)Ff»l:l'l ST-11-0011 (U) EXECUTIVE SUMMARY (U) Overview ('fS//SI//NF) This report summarizes the results of our audit of National Security Agency (NSA) controls to comply with the Foreign Intelligence Surveillance Court Order Regarding Business Records (BR) Retention. From April through June 2011, we performed testing and procedural reviews to assess the Agency's compliance. On the basis of the information reviewed, we found no instances of non-compliance with the terms of the Order for BR retention for calendar year (CY) 2011. However, we noted three areas for future improvement: (1) develop a plan and written procedures to document the Agency's BR retention process, (2) develop a process to research quarantined records, and (3) accurately document parser configurations. (U) Highlights (TS// SI// NF) The Agency should document the key initiatives and procedures that will be used to comply with the Order for BR retention in the future. (TS//81//~dF) • No formal BR retention plan or procedures (TS// SI// '[}IF) The Agency does not have a coordinated plan that documents the major initiatives for the BR retention process. Furthermore, the organizations responsible for maintaining BR systems, databases, and backups do not have written procedures to document their processes. • (U//FOUO) No process to research quarantined records (TS// SI// Pff) The Agency does not have a formal process to research quarantined records for reasonableness before their introduction into the Agency's BR repositories. • (U/iLPOUet Inaccurate parser documentation ('FS//SI//Pff) Documentation was not accurately maintained for the current parser configurations used to filter BR metadata. (U) Management Action (U //FOUO) Technology Directorate and Signals Intelligence Directorate personnel agreed with the Inspector General recommendations . The planned actions meet the intent of the recommendations. FOl.... 8fJCRET1S'Sf15'NOFORH iii DOCID: 423026.2 REF ID: A41772·67 TOfJ !fECttETt!Sl/JJVOFtHtN ST-11-0011 (U) This page intentionally left blank. T01" SECRETtS'SESW-0-...n.ORN iv DOCID: REF ID:A4177267 42302·62 TOP :S'ECttET17!'51t/NOFOltN ST-11-0011 l.(U)BACKGROUND '(b)(1} (b}(J)•P.. L. 86-36 (b}(3}-50"U~C 3024(i} ····..... (TSNSIJ.!N~)·Th~ Business Records (BR) Order (TS/fSI//Pf.F)""·Pµrsuant to a series of Orders issued by the Foreign Intelligence Surv·eiijance Court (FISC) beginning in May 2006, the National Security Agency (NSA))J.as been receiving certain call detail records or telephony metadata froinl !telecommunication providers. NSA refers to the Business Records (BR) Orders collectively as the -BR Orderll or -BR FISA.11 ............. . ::::::::::::::::::::::........:................ : (b}(1} (b)(3}-P.L. 86-36 (b}(3}-50 use 3024(i} (TS//81//Pff) The BR Order provides NSA access to bulk call detail records that include records of telephone calls between the United States and abroad or wholl within the Uni.t~.d ...States·; This collection of information is not wholly available to NSA through its other foreign intelligence information collection. It is valuable to NSA analysts tasked with identifying potential threats to the U.S. homeland and interests abroad because it enhances analysts' ability to identify, prioritize, and track terrorist operatives and their support networks in the United States and abroad, using call chaining analysis techniques. (TSh'Sh'"fMf) Provisions of the BR Order for Retention (T£//£I/OIF) The Order requires that BR metadata be destroyed no later than five years (60 months) after its initial collection.I The Office of General Counsel (OGC) reviewed the Order and concluded that BR retention compliance should be determined using the date when records are received from providers, and not the call communication date. 1 • (TS/fSl//PIF) Record receipt date is the actual date when telecommunication carriers electronically provide BR metadata to NSA. Although record receipt dates have no analytical value, the Agency uses this information to determine BR retention compliance. Record receipt dates are separate and distinct from call communication dates. • (T8//81//PIF) Call communication date is the date when a telephone call is made from one person (Selector A) to another person (TS.'/SI.'/HF) BR Order 11-57. dated 13 April 2011. defines telephone metadata as comprehensive communications routing information (e.g., originating and tenninating telephone number), trunk identifier, telephone calling card numbers, and the time and duration of calls. Telephony meta data does not include the substantive content of a communication or the name, address, or financial information of a subscriber or customer. 'f'019 8ECRE TM~J.i~'!'.T.fJFQ./J.A( 1 DOCID: REF ID: A41772 67 42302 62 1 1 'fOP SECR£f1$'81WNOPOR:N ST-11-0011 (Selector B).2 This date is important for intelligence analysis because it establishes a call association for a particular time between two selectors. (U) Timing differences with call communication dates and record receipt dates .............;.;..~/. ::.....,..~;.~~··I ('6)(:1f : :......... . I (b )( 3)~P~L.~ .86~36"· (b)(3)-18 USC.f'.98 (b)(3)-50 USC 3024(iP-------------.,..._IB-ec_a_u_s_e-of""'th_e_s_e_d..,..,i..,..ff-e-re_n_c_e_s_,-th-e_A_g_e_n_c_y___. '-----..,,--___,._,.....,,..-_,,......,.""=-' must track receipt dates for BR metadata to document compliance with the Order. ,_~I \ .L /nT I /1\TTC\ I '-"'I / ...... . ............. ························ ('6fr1):::. " ....... . (b)(3)-P.L. 86~36· ··· ....... (b)(3)-so use 3024(i) (TSHSWNF) NSA Repositories that Store BR Metadata .. . .. .. ..fT§/./~iH//M¥). BR metadatal I ... .. ..... ........ I (b}(3)~P.~·~ ..SS~as· .. . . ··.. . ... ~~ I USC _ _ _ _ _. . . . . ._ .. - . . . , , , . , . . - - - - - - - - - - - - - - - - - - - ' (b)(3)~50 3024(i~ • (T5l//~I//Jl,Fyj h·~·t:he"Ag«::ncy.'s. .c0..r.pp.rate contact chainini?: databast-i"'that accepts metadata from multiple ·s-c;·urces·I " · ....··········· ....... ···· .. ··· , ... ····· ---:· ...............--·· \ .... --···· ('b):(;;.P..k'. 86-36 ·····........ . '"' · · · · · · · · ~· · .fff!t:ti1!b:ttlfil____Jl§sJth £e. . Jc~ourJ2.QoIJra~t~e. . JCWla jt~aJ;ib~a.§.seur~eQ .os§jiJ2toQ.Jr[Y_j;lth1Ja~t;_ _, 2 (S.','SL'REL TO USA. FVEY) A selector is an identifier used in dialed number recognition (e.g., tele hone ····ri1iiribeiToYlirt:ligital network inteHigenee In this (b)(3)~P.L. 86'~36report the terms Selector A and Selector (b)(1f: :: rc'RE~ ;~~£,:\~ ¥ViY~----------------------------. --1 I 3 T01., SECREf1$'8b$'NOfi'O-R:JiF 2 REF ID:A4177267 DOCID: 4230262 TOP SECRET/·/SE~'NOF&Rl·l ST-11-0011 .....-········ ...·· ······· ..········· • ff.§././.SI/./N-F)1 lis the contingency database for (b){jj::::::::::::::::·::::::................ (b )(a)::P.. L"ss. 3s. .... (b )( 3)~sQ.: us.<:; 30 24{i).......... ······..... "·.. . "· . ..... '----..,,,..---------------------------' (T'S ;~·I"~"'i'¥i'F)·I Iis the system backup that stores an ·. ·l (( Tl • I I · .• ('fe5//SI//NF) Backup tapes are . . . . . . . . .. _ _ _ I _ _ m~~~~~i~~~d at,...I- - - - - - - - . . . , _ _ _ __ _ _ . _ (TS// SI// WF) Figure 1 illustrates the BR dataflows within the Agency and the various BR repositories. (TS//Sl//MF} Figure 1. Business Records Dataflow (b)(1) (b)(\3)· P.L. 86-36 (b)(1a) 50 USC 3024(i) ('b)t1').. .. . . ('fSi.St:tWJ! (b)(3)-P.L. 86~36 ..... , (b)(3)-50 USC 3024(i). TOP 8ECRETi~<\r/:;~\7V.()PQ.J>_}\T 3 DOCID: REF ID:A4177267 42302 62 1 rep 8£CR£1W8J;sqv-oFORl'tl ST-11-0011 (Ctt~l!!t TO USA, fi'V'l!'t') How information is stored (TS//81//WF) I I .. ..... ·····················fbff3)-P.L. 86-36 lare···th;;;···~-~~-databases used to store BR metadata for intelligence analy§is..... 1 I ..-·::.:.. ··::::::::::::::: .... (ilfr1):::«::::: ······················· (b)(3):PJ.,~ 86-36 (b)(3)-50 USC..3024(i) ·· .. ·· ... ···................ . ··· ..... {Ch'RE1:1' Figure 2. ._I_ _ _ _ _ _ _ __.IArchite·etu-re··5......... (6}"(3)-P.L. 86-36 I v11....,111 (b)(1 (b)(3 -P.L. 86-36 (b)(~ -50 USC 3024(i) ' ' i ' \ (TB//GIHNF) (T~/./.~l/../UF) · ('6fr1) : :. .···· (b)(3)-P.L. 86:36·· (b)(3)-50 USC 3024(i) TOP SECR:F:1?~\\'J,~qv.oY0£7!1f 4 DOCID: REF ID:A4177267 42302,62 TOP 8ECR£1W8ft7'1¥0FfHtN ST-11-0011 ······ ······· ········ {bl(1)················ (b)(3)-P.L. 86:.:36·· · ... (b)(3)::SQ_USC 3024(i)···· ·.. ,_:··;-;;;~·1·r"'r.' ·I \.I. V/ I VJ./ I J. I .... '· "···... (TSHSIHNF) NSA's BR Age-Off Process ('il)l3')~P:c·s 0 :.:3s······ ('FS//SI//NF) To remain compliant with the terms of the Order (which states .............thatBR.m.e.tl'!9:gt.9.:...T.g.1:J.St be destroyed no later than five ears after its initial Ma 2011. collection , the A enc ....c.oiii. "leted"···1g·fifst""BR"·a e~off {b){1J"•················· ············ (b)(3)-P .L. 86-36 {il)(.1Y:'"'.'.'" ' ""'"'""'' ':··:::········· (b)(3)~P,L 86-36 (b)(3)-50USC 3024(i) ··., ~ (U//trOUe+-A relational database stores data in tables using a standardized data format. This allows similar information to be organized and queried on the basis of specific data fields. 5 f'.f8/,'Sl/.'l'ff.j . ./1 . . . . · ...,:.·· ::::.. -····· (b)(1) (b)(3)-P.L. 86-36 'f0"/1 StsCltET17'8I15'!tlOFORfil 5 DOCID: 4230262 REF ID:A4177267 ST-11-0011 ('t)Jf1)'·. ······· ·········· (b)(3)-P .L. 86-3() .. ······· · .. (b)(3)-50 USC 3024(i) (T£f/8I//NF) Table 1 shows the procedural timeline used to complete the CY2011 BR age-off effort. TOP S£CRET1SShVi'\rOFOR:N 6 REF ID:A41772'67 DOCID: 4230262 TOP S1!CltET17~W/J'l'VTOF&RN ST-11-0011 (TG//811/MF) Table 1. Procedures Used to Complete the BR Age-0~ : :::'::: ::::;;:::::;;:::::::.i ;7,b)(3)-P. L. 86-36 1 ~~=,.;:, lhard drivesjhafp~~viously secure destruction. ..........··· ...... · ....... · ......... :::: ::fo:iil ·········· •11 tlj: ::::'.~ '. '. '.'.'.'.'.'.'.'.'.:::::::: ;;:;: :::::: ;:'.:::::::' ••', •• (o){.1}"·!..:· : . ... . . . (b)(3):.pi;86~36 ":" •.......:::............·.·__ .. :::: sfg~~fBR transactions/,ere subn{itted fdr ~~~~~~~~~a!,:~:;.~~~~~~:~~r~~~i~~~metadata._I__,.,___ _ _--;..___...,..\! r· -. . . . . . . . . "lfif~~I-" . Iwere deleted. ... ....._ _ 1 _,l~~s taken The liryl.-..-.---; remaining fitesl /.-" out of service during the r~b.uild. s'ib·aq_alysts were redirected froml.___ __.ltothel fbackup database I I ,___ _ _ __, {b)(3')-P..L.... ~6-36 ··············· .. ···· .·" .. ·· ........................ f6)'('1) (b){3)-P.L. 86-36 (b)(3)~5Q USC 3024 i) SID an~IY.~~~ 'l_l'E:lr~ rnolr.ected .. baek tcl..____ __. ... ,,,,,,:(............. ... ..... ... . (b) 3);f>;L· 86-36 The continuous flow of BR metadatal ................ -·· ..... ___ lwas restored so that it again ~ielated the AJ~;bc~;~ ~::do~~t1~!~~da~~~:t~t~c~~~n~~s ~~tad~~~~~~~~i~ed during "- - - . . ________ "-,1.._______ _ _ _ _ _ _ _ _ _ _ ____..... BR transactions manually sa~~dto._I_ __,l..._.....----....------.;.;;·-&.,·I Iwill be deleted and reloj~ed with the post-rebuil9 transaction.!$._!--...------'I v•::: TOP SEC.7?.:ETP8Ji~q'l{)FORN 7 ••...-··· (b)(1) (b)(3)-P .L. 86-36 (T~//Blh'Mpt DOCID: 4230262 REF JCD:A41772'67 TO{' SECRET!HY!JS'NOPO-RN ST-11-0011 ( U) Organization fProje·ct·rearrc········· ··· (TSMREL TO US/\, FVP()I 7 ······ :::::·: (b)(3)-P.L. 86-36 .........:~__:============--...:._~~~~~__,...;;;.;:;:__, ........{TS//.SI// nF) ·I mi:r~:t~~~~~4;,; .... 1...._________________. . -_. . . _·_· · ·______ ·· .... (DUFOJ;J.O~ Structured Repositories ···.... ·.. l I/ . . (T§//~If/i>rwH.--------====~-----------. ·. ·.. ····1.____ _ _ _ _ _ ____. (U) Audit Universe (U) General control environment assessment ...··· _ .... -~::::: ........ .. 11!ig(~~:::::::::·:.:.:.......... (b)(1) .-··· -······ . .· (TS//Sl/fNF) The audit scope focused on the manual and automated controls used to maintain compliance with the terms of the Order for BR retention for the Agency's BR repositories, system backups, and backup ta es. The BR re ositories reviewed included the o erational com onent of ... ·· 1-----------.,..------------~-".""'-.'~---~- we also observed the process for deleting BR files and the physical storage and destruction procedures - - - - - - - - - - - - - - - - - . W e excluded from review BR ......-·········,__...,__ __,.._ _ _ _ _ _....-_ _ _ _ _ _.,.. m ormatlon 1ssemma e m 1gn s ntelligence reports. 1----------,.---! (b)(3)-P.L. 86-36 (b)(3)-50 use 3024(i) (U) Reviews to assess Agency compliance with the terms of the Order (TS//SI//PfF) We performed five levels ofreview and observation to determine the Agency's compliance with the terms of the Order for BR retention. We also determined whether the Agency has a plan and organizational procedures to document the systems, resources, and organizations involved with the BR retention efforts. • • • • • (U) (U) (U) (U) (U) Review Review Review Review Review 1: 2: 3: 4: 5: Tape, disk, and system backup data destruction Quarantined records System parser controls Selector pair testing Metrics analysis (U / /fi'OUO) A summary of the audit test results for the five levels are in Table 2 in the Finding and Recommendations Section. A detailed summary of the audit methodology and results for the selector pair testing and metrics analysis are in Appendix B. TO{' 8ECRETiS'!:ll>S'NOF0Ri"I 8 DOCID: REF ID:A4177267 4.230262 TOP SECRETt't'S}17'NOFORl•l ST-11-00li II. (U) FINDING AND RECOMMENDATIONS (TSHSIHNF) BR Retention Practices Must Be Documented (TSh'SlffHF) We found no instances of non-compliance with the terms of the Order for BR retention for CY2011. However, the Agency does not have a formal plan or written organizational procedures to document the systems, resources, and practices used to maintain compliance with the Order. Furthermore, jwas not documentation of t~-j accurately maintaifle . As a result, the Agency has an mcreased nsk of non· compliancejnthe future . ... ····· (UJ.. Cfiteria Used to Assess the Agency's Compliance with the Order ...·· (T~//~I//J)IF) ··" .... · (b)(1) (b)(3)-P.L. 86-36 (b)(3)-50 USC 3024(i) The BR Order requires that BR metadata be destroyed no later than five years after its initial collection. To maintain compliance, TD and SID decided to delete annually from A enc databases and ta e storage BR metadata whose record receipt date is ·Fot···· ·· ······(b)(1) CY201 l, the Agency deleted BR metadata received (b)(3)-P.L. 86-36 ('f8//8I//MF) We performed five levels of review to assess the adequacy of controls and to determine whether the Agency was in compliance with \the terms of the BR Order. In addition,, we determined whether the Agency\ has a formal plan and organizational procedures to document the key proce~ses and the roles and responsibilities for the 01ganjzatjon~ involved in t!t.~. . I.?,J~ rebuild. A majority of the audit focused 01~ j.anct·the.fap:e;:;~ti_lf'. ~b)(3)-P.L. 86-36 system backups because they stored BR metadata from the.dat¢>ofili.e \ original Order (May 2006). ..:::",. ··· .- .· syst~mba~~~;iJ~~;uct,fon:practices ~or 1. (T8//8I//WF) Tape, disk, and BR metadata received befO(E}.I evaluated the secure \ stora e and destruction ··rocedures·for the backup tapes, used to store BR \ "'"m-e~t~a""'c-a~~?-...-.... ,,..e-.-s-0-0.....-s-e-rv-e"""'"""'J"'l'-e-p_r_o_c_e-ss__,..o-r-T'eleting the Qsfstem backup.Jiles and reviewed for compliance the data stored 111 thel lcOOP backup system. l:We I "1"'1. 2• .... .:::::::::::::::::······ ·--I 1 L ....,1 rl~T ~·r r ·~ • LU , Quarant'me d I d s-·I r~.c..or ............................................... . (b)(1) (b)(3)-P.L. 86-36 (b)(3)-50 USC 3024(i) to develop a system quarantine process to document the records that TOl.0 S:ECR£1W8lt7'NOFORN 9 I REF ID:A4177267 DOCID: 4230262 T(}P 8ECRETiS'8};S'NOFORN ST-11-0011 .....1 ....... =================:l'""w=-e-r_e_v.,..ie_w_e_d.,,....,.th.--e_q_u_a_r_a.I_1'""ti.,...,n_e_p_r_o-ce_s_s...,...to_ _ _ ___. determine whether an adequate audit trail was maintained to document the Agency's compliance with the Order. ..:: ::: .. :1 .. ··· ::::..... ·· ..··::::::> ···· ·····=· (b)(1) (b)(3)-P.L. 86-36 (b)(3)-50 USC 3024(i) (TS//SI//H:P) System parser controls System parsers are used to filter BR metadata of unwanted data before records are saved to Agency databases. To determine whether the parsers were working as intended, we performed testing in a simulated environment to ~~~~;I that parsers Ill J~~~~i~~~~ee~e~:~o~~~P~~~;~!ealr~i4::i~7~,~~~(3)-P.L 86-36 information before processing records. We also review~d·wlieTh(f~ documentation of the parser configurations used to filter· B"R : ~ metadata was accurately maintained. · ...- · / ........ 3.~~i::!!.1;:ti.j!!1...:S~e~l~ec~t~o~r~a~ff~t~e~st~Jn~..g~·[··_··~~~~~~~~~..L.-B.J ! c~ dates befor..~ ..... System testing.\:tfas performed to d(i'term~\ne ........ ·· \11.hethet·fhese records were correctly.1frocessed with one of/the ii ........... ··········following outcomes after the BR ;ebuild was complete: · ii ... ··· Deleted;...-P~~-;~rmed ...··::::::::.... ········ {b).(1J.. . (U / /-F!OUE7) for records that ha'.d call / communication __ c;la:tes and receipt dates before (b )( 3)~P~L··s6~.~6 1 (b)(3)-5CfU$C 3·024(il.. ·· ... • (C//REL TQ. tfSA, FVB5l) Modified: Performed f...0-1/.,..ir-e-co_r_d_s_that I ·· .. · ... ·······.... ·... -. ... h_ad s~-~pes.sive call dates that occurred on or a~t'erl 1 1 • ··... . i''· / 4. (Cf/REL 'PO USA, :PVEY) obtained I · Metrics analysis Summary metrics were / (but before the to verify that .~~~=~-::::-::=~=~=-=-::~-=~=%"1:"':=-:'~~'-~ck (b)(1) (b)(3)-P .L. 86-36 (U) Audit Summary Results (b)(3)-50 use 3024(i) (U / /FOUO) Although we found no instances of non-compliance with the terms of the Order, we noted three areas for future improvement: 1. (U) Develop a plan and written procedures to document the rebuild. 2. (U) Establish a process to research quarantined records. 3. (U) Accurately document parser configurations . (U / /FOUO) These deficiencies are discussed in detail in the next section. Table 2 shows the summary results for the five areas reviewed . TOP SECR:ETiS1!AW:Po{0FOR.1¥ JO DOCID: REF ID:A4177267 4230262 fftP 8ECRf T1S'Sf15'NOFfHtN ST-11-0011 (U) Table 2. Audit Summary Results ff8//:51i~MF) Compliant: Tape, disk, and system backup destruction practices We found no instances of non-compliance with the Order for BR retention. • .·" • T~eObackup tapes were submitted for secure destruction . The backup ...ta es were secure! stored in a locked cabinet inside a restricted access room -----....-.!-~I.es· ······:: We"f\fdeletea ; ...: ::::::: ::::::::::::::::::=:=::'"" "'"''"'"'"""'""""(b )(3)-P .L. • _Th 1====1.;.fi;.:;a;.,;rd;;.,···.::dr~lve·s·we:r.~·:~·r.~~~-9.. cin.ct submittediof""sec"i.fre destruction. ;; Th backU was deleted of all BR metadata and ··repopulated ··········· ..._ __, _____________ .... ····················· .... ................ : Compliant (with an exception noted): We found no instances of non-compliance with the Order for BR retention. However, T1222's process for reviewing and researching quarantined records must be documented . ·~·····"Ad"equ.ate··aoci:i"mei"iitatron·was"fi'iaintaifiei"a forth·e··a ···········-re~ords ·····... ··· .. \"··.::. ··::.. ·<:·.. • ·.·..... \ .. ·····. S~sfe~.::p~rser.. Compliant (with an exception noted): ···con.!rols·· .. ··· ·········....\Ne.foy_nd no instances of non-compliance with the Order for BR retention. ·..... ·.8'bwey_e('the..f.\gency must ensure that configuration documentation accurately ··....... rerl'ects)he.c.urreI1t-0perating parser configurations in use. " ·.·. .. "<::::,-... · .... ·.. ·... ·... ··.... -~ We peffor~:~d0t~~fs·ina ..l>.imulated environment that confirmed arsers ····correctly proc·essed.tr-·r13........s~a:-ct..,io..,n... ··s...__ _ _ _ _ _ __,,_.,...,.-,......,__. quarantined records and deleted suspected .predit card information::frorn... _records. ···... Se;ledt~r.. pair testiti g. :_-·...... . CoJ11 pliant: ·····... ········...... · \JV~ ·found no instances of ~on"Qompliance with th~ brd~r foi'"BR..r,etention. • ·sa~·~le. testing verified that integrity of the data (r~ceiv.ed b~fore-.tbe · .. " :"-... rebllild be an could be relied on t"o·conduct our selector testln . In total· ·. · "......... records .. ..,,,.....,..,,__..,.._ __,,;.,-,.....,,..._,,__.,,.__.,.,..,....,......,..,.iwer.e;_ randomly selected to verify ~hq,t it met our criteri~ fo.r testing. (Note: We could n~t ferform a statistical ··· ~·· . . Verifo;:ation because of.the size of the sample universe_ . ·•. Th~I I records were found to be correctly dele.. te-d-or_m_o_di-fie-d-(w-i-th.... .latter thel lrebuildwas com 151ete: the· 1 I Metrics analysis . Compliant: ·.. ·. m u ...... /····· · .. ·We found no instances of non-compliance With the OrdE!rforBR ~etention. ~\ The metrjcs analysis verified that no recc.rgs / ··· Jafter th ei:=:=:::Eebuild....w-a_s_c_o_m_p"'"le""'t-e-.""T""'hi_s_v-a""lid"'"a""ti-on"" was performed before thec==Jquarantined files were re-introduced. I (U) See Appendix B for a detailed explanation of the testing methodology and results for the selector pair test and metrics analysis. Only the areas with exceptions noted are presented in this section. TOP SECRfT1$'Sf1VNOFOitf!l 11 u DOCID: REF I:D: A41772·67 42302'62 TO{' SECR£f1S'SbS'!iOPORH ST-11-0011 (TS/i'SIHNF) BR Retention Practices Must Be Better Documented (U) The Agency does not have a process to research quarantined records (U //FOUO) We found no instances of non-compliance associated with the newly developed quarantine process. However, the Agency must establish a formal process to research quarantined records before their introduction into Agency repositories. ·--I, __ I /°!\.TT""\\ \ ~ ~I I - -1 I • ' I I /."I i..... /·································· ...·... .. ... ... ..·· t~n1r-- . . . . . . . . .. (b)(3J~P.L. 86-36 (b)(3)i1:~. use 798 (b)(~)-5(flJ.~C 3024(i) ·.. ·.... . ······················ "-' -~·· . . ·:-r:--~··I I "· .. "·. ·. .. •, 1· ... ·. ·-- .... I (T5 TT I '5I II II I>TJ1)1 I TOP SECH:Ef;S'8ESWO-YORH 12 DOCID: REF ID: A41.772'67 42302 62 1 TffP S£CltE'tWSlt7'!1lOFO-RN ST-11-0011 {TSh'SINNF) Establish a process to research quarantined records before they are removed from quarantine and included in the Agency's BR repositories. (ACTION: I ........ 1 ~··· (U) Management Respon..§i;t................... · ············ ·········· (b)(3)-P.L. 86-36 ......... CONCUR (T5l//~H//NFtt lwill"write a process to research and analyze any quarantined records in order to determine if any records should be included in the Agency's BR repositories. (U) OIG Comment (U / /FOUO) The planned actions meet the intent of the recommendation. (U) Documentation did not accurately reflect parser configurations ('f'S//SI//NF) Although we found no instances of non-compliance for the system controls used to filter BR metadata, the Agency must ensure .that documentation accurately reflects the current parser configurations . ('f'S//SI//MF) The Agency receives raw BRm.e.tadafa... froml l""''". (b)( 3)-P.L. 86 -36 ......~.~.~~,S9:¥:l:Jf:),µ~~~§t.ti.Qn::::p:rovid:et~................... I Pa,i;s;fs' are used to .,..........................................................................filter the BR metadata of unwanted information before jti~.-saved to Agency (b)(1) databases. l·D'ata Integrity ~~~~!)):~~u:~~~ (i) Analysts (DjAs) s~ated that they have been actj.vely working to docum.ent the .... ·. parser configurations! I· However, the DIAs confirmed that the suooortinl! documentation has not been consistentlv uoclated to 24 f ··· ... (U) Action taken .........................................{U/fFOUO)l------------.IDIAs fin1shed their update of the (b)( 3)-P.L. 86 -36 parsers' configurations. This documentation was subsequently provided to the Office of the Inspector General. . . - ..·.-·;. llll'Ri;dOMIVl~NDAt.10N: 2 Y .·;. ~ •. -·~--- 1 ·•• ; -: ''.-·;. • , .. '.' 1... ·".:···:.·.• .... '··· -· -· (U//.pQUO) Update parser documentation to reflect accurate! y the current configurations in use. This documentation should be updated as new configuration changes are made to the parsers. (ACTION: l _land DIAs) (b)(3)-P.L. 86-36 TOP SECllf!:nS'SN'NOF'f111l'I' 13 REF ID:A4177267 DOCID: 4230262 TOP Sl!CltEf1)Sl17'NOf?OftN ST-11-0011 (U) Management Response CONCUR (U /tFOUO ll l·completed·its·upt:l;~t~::o:r:tne::parser::: . ,.,,,,,,,,,, .......f*)(3)-P.L. 86-36 configurations! l·anct·wm··continue to update the documentation as configuration changes are made to the parsers . will also review the documentation on a quarterly basis to verify it is up-todate. ._I_ __ _ . (U) OIG Comment (U / fFOUO) Status: CLOSED. The actions addressed the recommendation. ('f~//~l//l~F') The Agency does not have a formal plan and organizational procedures to document its BR retention practices {Tg/ / g1/ OTF) The Agency does not have a formal plan to document the BR retention requirements, the methodology used to maintain compliance for the enterprise, the systems and databases and other storage media that store BR data, the organizations responsible for carrying out the plan, and the milestones for completing future rebuilds. Furthermore, the organizations responsible for maintaining BR systems, databases, and b~_c,:J. simulated environment to verify th~~ P.arsers (1) quarantined records (b) ·< 3 > .::·i?·;·L> BQ~ 36 with call communication dates before land (2) deleted (b) (3.) -50 USC j()'24·(i) · · s:µ.spected credit card information before processing of records. We alsoteviewed whether documentation of the current parser configuratioiis was accurately maintained . I 4. ;::,, ::, , :·::'; Selector;~i;t~silngl.-----------------. TOP 8ECR£1W8fM.VO:PORN A-1 REF ID:A4177267 DOCID: 4230262 'fffP tiECREf17'S/J7'NOFOR1'l ST-11-0011 .......... ·········· ············ 5. (C//REL 'fO USA, FVEY) Metrics analysis Summary metrics were o~.!ajned·l !{but before the ·cfuarantined records were introduced) as another check to verify that no records had first call dates before I Im•• ----- ,m•• n m•'' ,,.,m''(6)(3)-P.L. 86-36 ········· ············· (b) ( 1) (b) (b) ( 3 )-P.L. 86 _ 36 ( 3 ) _ 50 use 3024 (TS//Si//NF) We also determined whether the Agency has a formal plan and (iprganizational procedures to document the key processes and the roles and responsibilities for the various organizations involved in the BR rebuild. (U / / FOUO) We corresponded with management and personnel from the Technology Directorate, the Signals Intelligence Directorate, the Office of the Director of Compliance, and the Office of General Counsel. (U //FOUO) We conducted this audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions according to our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions according to our audit objectives. (U) Use of Computer-Processed Data 6 36 3 (b) < >-P.L ...13 - . ... (Qf/REL TO USA. iVEY) To perform this audit, we used data that originated I. ""'"' ":::·· (b) ..(-1}..... (b) <3t--P.:;"r;·~·· '86 .... 3.f~. (b) (3) ._ from" the s stem. We used this information to ---- -- .............................. seleetor-p-air.. testing- erform our ------------------------------- -so""usc ··....3024'(1 ...... ·· ... (U) Prior Coverage ('fS//SI//N'fi') The NSA Office of the Inspector General (OIG) has not performed a previous audit or review of the Agency controls to assess compliance with the terms of the Order for BR retention. The NSA OIG completed on 21 December 2007 the Audit of Retention of Domestic Communications Collected Under the Foreign Intelligence Surveillance Act (ST-06-0007); however, that audit focused on inadvertent collection of Foreign Intelligence Surveillance Act domestic communications and whether the Agency was in compliance with the U.S. Signals Intelligence Directive (USSID) SPOO 18 special minimization procedures. In addition, the Audit on the Assessment of Management Controls for Implementing the FISC Order (ST-06-0018), completed on 5 September 2006, focused on the adequacy of management controls for processing and disseminating U.S. person information. Most recently, the BR Capstone Audit (ST-10-0004), 25 May 2011, evaluated the BR querying and dissemination controls. TOP 8ECR£1W8hS'NOPORN A-2 DOCID: 42302'62 REF ID: A41772·67 ST-11-0011 (U) Management Control Program (TS//SI//PfF) As part of the audit, we assessed the organization's control environment pertaining to the audit objective, as set forth in NSA/CSS Policy 7-3, Intemal Control Program, 14 April 2006. The 2010 Vulnerability and Process Assessment completed byl .... ·······I 1-~id not report concerns ap_p.lic.able. to BR retention. ..···· . · I " ········· (b) (3) -P. L. 'fOP ,'fECREf/,'8bV·JVOFffR:i'¥ A-3 86-36 DOCID: 4230262 :REF ID:A4177267 TOP 8ECR£nS'8l:A'-NOYOR1¥ ST-11-0011 (U) This page intentionally left blank. T01Q 8ECREnS'SbS'NOFORN A-4 DOCID: 42302 62 1 REF ID:A4177267 TOP 8ECR£T1$'Sh$'NOT0f01rl ST-11-0011 (U) APPENDIX 8 (U) Testing Methodologies and Results TOP SEClfF:f;S'SEV-NtJf"ORN DOCID: 4230262 REF ID:A4177267 'lQP ~'ECRETiS1!ilAWOFf>RH ST-11-0011 (U) This page intentionally left blank. DOCID: REF ID: A41772,67 42302'62 f'(JfJ .'5ECltET!7i'51t/!llOFORJ<::...1..-.. .-. . -.. -.-,-,-------------------' --~-------L . ~=~~J_/_FO~_-_O_)j~~~~~~~~~~~~~~~~'l fiJ)t3j;;p~L;··86-36 ......... .. ........... · I FVFt) _ ······· ... (C//REL 'PO criteria. USA, these above ,___ _ _ _ _ _ _ _ _ _ _ _ _ ___. (U) Step 2 - Verify that the data received in the Before and Span listings contained only selector pairs that met the criteria in Step 1. -~~~~~~~;~ (b)(3)-P .L. 86-36 __ .. __ :-~--~~--------------------------' ... .-;':.::::::::::::.:::::: ........................ . refi==~~i--===~-r-1-'/_R_S_L_T_O_U_S_A_,_F_V_&_l)-,~~~~~~~~~~~~~l TOP SECRET/,'f,Y1S'l'IOFfYR:N B-1 REF ID:A4177267 DOCID: 4230262 'ffYP SECftEfJ~S11~'NOFO-RN ST-11-0011 1..__ _ _____.I··· ··················································· /(1;>)(1) / /(~)(3)-P.L. 86-36 (U) Step 3 - Establish a test methodology and criteria for the 1~eleh6~. pair testing. . (U) Before listing: t'C'Lt'REL • ·1 'PO USA , FHE"ll V I I ·········································· t'il)H). ········ · (b)(3)-P.L. 86-36 (b)(3)-50 USC 3024(i) • (C//REL TO U£A, FVEYl! {p)(1) (U) Span listing: (b)(.?J-f>_.L. 86-36 ·. . · 1 \\I • (C//REL 'fO USA, FVEYJI "('b)t1)::·:::::················· ................... ,.~ ( b )( 3 )~p.[:;~"86-36 ..... (b)(3)-50 1Jsc. .~024(i) ·· ...... ·· .. • ,. ... .. ,. 'V ...... 'AJI .. .. · · - • v v ......... ..L ' - ' -· .L; .I .J.J.L I • (C //REL TO (bi{jj (b)(3)~P.L,~=· 86-36 • •· • • • • : u Q~'\, ...FVEY) I I I.- - . . . . . - - - - - - - - - - - - - - - - - - - - - - 'I ,___ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _...,.__ _______ (U//FSUG~ Step 4 - Verify that ..___..,_..,.........,....._ work as intended for the selector pair testing. • (U I /fOUO) I I 'ftJ"f' SECRET1$'8hS'NOFOR:I•l B-2 ___. would _.....L. 86-36 DOCID: REF ID:A4177267 42302·62 T01n SECR£T17'Sh7'NOFOfOV ST-11-0011 t~ij__ ... ·~·.·--1~·-·~~~~~~~~~~~~~~~~~~~~~~ (b)(3)-P.L. 86-36 (U) Selector pair test results (U / / FOU 0) We found no instances of non-compliance during the selector pair testing. Our testing found no error cases during the selector pair test. ,IT'\,., -- .,...,.T""\·\·I ... .....,, l.J.L .1. ....... I ·········· ······ ..(b)(1") :::·::·: .......... . (b)(3)•P.L. 86-36 (b)(3)-50 USC 3024(i) 0 Quarantined r~cor.~s added back (after rebuild) ....:::::::::::::::::::::::::::.::::::··. (6)(1)•••·········· . (b)(3)-P .L. 86~36 (+~M/SIN~JF) fff.F 8ECR£n7'SfJSC!VOFOAA~ B-3 REF ID:A4177267 DOCID: 4230262 fffP 8ECR£T!7Sf!J'fv'tJFOJVV ST-11-0011 (U) Metrics Analysis Methodology and Results (U) Metrics analysis (U / /FOUO) We found no instances of non-conipliance during our metric analysis. However, we were unable to perform both parts of the metrics analysis as originally planned in the Audit Guide. (TS// SI// ~ff) The intended purpose of the metrics analysis was twofold: (1) confirm that all records before I lwere·Te:tnoVecr'froiii . lhe ............... T6)(3)-P.L. 86-36 ........................ 1 I database, and (2) verify that the total number of records could be {b)fa:)-P:L. 86~36" ····aeeounted..forhefore·ai1cl·afterthel Irebuild was complete. . Although we were able to verify the first part, in that the selector Fair testing . confirmed tha.:!Jhel !records with first call dates before_ (bj(3)-P.L. 86-36 coulcL.be·atEounted for (i.e., deleted or modified, or excluded by parsers on .·· ...... tJiitbasis of current configurations), we were unable to verify the second part ....... --············ of the metrics analysis because the net number of records decreased by ..........approxima!~.1Y1 ~e Table Bl). However, l.pfficials (b)(1) that there would be some differences before the rebuild\~ffort antic..ipattfd (b)(3)-P.L. 86-36 ....began, and their explanations seem reasonable. "· I .... ----·· --···· ...:::::::::............ (b)(1) (b)(3)-P .L. 86-36 ............................ .....(b)(3)-P.L. 86-36 .....(.TS/../.£1/. /.HF..} ..Specifically.,l....................... lconGlud{id:::.fu~t:.fuedecrease ...-0fl I records happened because the rebuild had been performed using the most current parser configurations, which had been refined over time to capture more precisely records with a Counterterr orism value. As a result, the current parsers allowed fewer records in aggregate to be downloaded into I lthanJ:h.~ . total sum of the records previously allowed during the parsers' five-year sp'iiii': (b )(3)-P .L. 86-36 (U / /FOUO) Examples of modifications made to the parser configurations designed to prevent certain types of records from being processed include instances in which: - • • (TS/fSI//NF) were exciuded because they had little or no Counterterrorism value, (;~;/si//nP)..------------------was excluded, and fffP 8ECR£ T1~'Sf15'!1/0f?&RH B-4 DOCID: 42302 62 1 REF ID:A41772'67 TOP 8£CRETl-/Sb$'}il0T'OftN ST-11-0011 (U) APPENDIX C (U) Full Text of Management Response DOCID: 4230262 REF ID:A4177267 fOt' Sf:!CltEf17Sl!J'NOFOitN ST-11-0011 (U) This page intentionally left blank. REF ID:A4177267 DOCID: 4230262 Tfff /fECR£'BS,SSJFICATIO.N NSA STAJ•'F PR()Cf<:SSING l'ORM ·cc C0:\"1'1101, Nl.'~IBER EXIU:G CO:-\'i'ROLNl'~IUf,lt TO QIG 10690-11 ==,,,.----,----.......,.-------------,1-~'\l'l'ROVAL SUU.JECT (U//FOUO)SIGlNT Dirc.ctorate Response to the 010 Draft R~portoq che~Audit (fNSA Ci:mt.rol t(l C~mplvwit11 the FISC Qraer Regardmg }3usmess Recprcl Retent1on (ST• l l-0011) ·cc SllSPENSE SlGNA'lllllt X INFORMATION Dl.S'l'l{lllll'l.'10.N S1D,S02,S3 SU!\l~J.\ln' PURPOSE: (U/~ To pl'ovidethe SIGINT Directorate's (SID) response to the Office of the Inspector deneral (OIG) Draft Report on the Audit of NSA Control to Comply with .the F(weignfntellige11ce Surveillance Court (FISC) Order Regarding Business Record {BR) Retention (ST-I l~OOJ l ). · BACKQROUNQ: (U/ITEffi91 From April through June 201 I I.he NSAOIGperfonned testing and pro~edt1ral reviews to assess ~he Agency's gompliance with ]3It ret.enti()n. On the basis of th(! infcirmation revic;wed, the CHG found no instances ofnon~compliance for tl1r;:.CY20Jl terms. How~ver, threea.reas were noted t()rtl1t11rejmprove!lle11t: (I) develop a plan and written. proc:e!lures t() 4ocmme:nrQ1e Agency's BR retention pmcess~ (2}dev~lop a pr9cess to research quarantined records, and (3) acct1rately document parser configi1ratforis .. As aresulta sjngle i·e(:onimcndation(RccQmmendatio112) was assigned.to tl).el I Jand,.itslf.1~':'.s, "(U/lFOUOJ Updateparser docu111entation .t.o:fejlect qccural~~)' tlie cw~re;ii co1!figuriltioi1sin r~se. .Goi11gj(f1;iiafii; thfrdocum(!,1.1fdtfonslioi1ld be #pifaN!das new cotifJgµmf/On cbangi;:s are made to the parsers. " ...... ' · ·· ·· · I .,.....::::::::/('b)(3)-P.L. 86-36 CV/tfQPO) Recommendation 2 was subsequently tasked Lo ~JP:ihf\~ftten a?knowledgr:ment on the validit); · oflh~:i!:isµe an;[) DATF. lsm10 Liaison '''""11,··~~.:\i('S:1'M·1-s2 Dtri,•·d Dared: 201170108 l>trl11»if1 On: 203611607 S023 .. 966-2464 SECURITY CLASSIFICATION ...... ··· . ~·HONE (Sei·ure) r,\TF. PRUARED 21 September 2011 UNCLASS/FlfJDl!FtJ'.R tJFfi'fiAL USE BPl'L l' ::~·· (b)(3)-P .L. 86-36 f()P 8ECltEf1}S11}'falOf?OR:N C-1 DOCID: REF ID: A41772 67 4.2302 62 1 1 'fftP SECltE'fWSI!lfllOFO-R:J-)'(s) : D /,....IG _ _ ____, Dll Dll D12 D13 D14 I·. TOP SECRE1Y/81/JUOFCtB:iV REF ID:A4177277 DOCID: 42302'64 TOP .'SECltET!7~Y117'!c/OFORH ST-12-0003 {U) TABLE OF CONTENTS (U} EXECUTIVE SUMMARY ......................................................................................... iii I. (U} BACKGROUND ................................................................................................... 1 II. (U} FINDING AND RECOMMENDATIONS ............................................................... 5 (TSHSIH~ff) FINDING: BR Program Processes Must Be Improved ......................... 5 Ill. (U} OBSERVATIONS .............................................................................................. 11 IV. (U} SUMMARY OF RECOMMENDATIONS ............................................................ 13 V. (U} ABBREVIATIONS AND ORGANIZATIONS ...................................................... 15 APPENDIX A: (U} About the Special Study APPENDIX B: (TSHSIJ.'NF) Business Records Systems APPENDIX C: ('f'SHSIHNF) Agency Foreign Intelligence Surveillance Act Business Records Collection Stakeholders APPENDIX D: (U} Full Text of Management Response DOCID: 4230264 REF ID:A4177277 'fttP SECRET1S'8hS'f'{{):POR N ST-12-0003 (U) This page intentionally left blank. TOP 8EC"'fE1W8JiS'}ol{H!O:R:i¥ ii REF ID:A4177277 DOCID: 4230264 ST-12-0003 (U) EXECUTIVE SUMMARY (U) Overview ('PS// SI// MF) This report summarizes the results of our special study of National Security Agency (NSA) controls to comply with the Foreign Intelligence Surveillance Court Order regarding business records (BR) collection. From December 2011 through March 2012, we performed testing and procedural reviews to assess the Agency's compliance . Other than one incident NSA reported during our review, we found no other instance of non- compliance with the terms of the Order for BR collection during calendar year 2011. However, we noted areas for improvement. (U) Highlights (T8//81/0fF) The Agency should improve BR processes to strengthen controls and help reduce the risk of non-compliance. (TS//SI//Pl'F) Program material BR FISA program material is not centrally located or accessible to stal<:eholders . • (T8//81/0fF) Meeting notes Meeting notes .of mandatory quarterly meetings with Department of Justice National Security Division are not kept. {b}tfF::::::::::::::::::: . :..................... ····· . .fl'S/fSI/ INF) I ~~~~;~~:o\J~~;~~~ · · ·····!.______________ (Tel//§I//~Tlf) Management oversight Management does not have a process to review sampling . (U / /FOUO) Structure code test There is no process for periodic reviews of the structure code test used in sampling . (U) Management Action (U / /f?OUO) Signals Intelligence Directorate personnel agreed with the Inspector General recommendations. The planned actions meet the intent of the recommendations. iii DOCID: 4230264 REF 1? 1 ST -1 2?0003 (U) This page intentionally left blank. 11? REF DOCID: 4230264 J:D:~4177277 f&P SECRETtS'SLS'PlOirzOKl•l ST-12-0003 I. (U) BACKGROUND (TSHSIHNF) Business Records (BR) Order {QJ(1y........... ('TS// SI/ OW) Pursuant to a series of Orders issued by the Foreign Intelligence Surveillance Court (FISC) beginning in May 2006 to comply with (b)(3):::5o.µsc 3024(i) the··Fo.re.igp. Intelligence Surveillance Act of 1978 (FISA), the National . Security Ageiicy·(NSAJ has been receiving certain call detail records (CDRs) ·. . . or telephony metadata.fi:'orul !telecommunications providers. NSA refers · ·. to these BR Orders collectively as the "BR Order" or "BR FISA." (b)(;3')~P.L. ss:·35· ..... ·······... ·· ......·.·.· ·.·.· ·. .(Te// ~I// ~IE) The BR Order provides NSA access to records of telephone calls . J:>etween the United States and abroad or wholl within the United States> __ ............................ .........,.-_.,,.... This collection of information is not available to NSA through its other foreign intelligence information collection. It is valuable to NSA analysts tasked with identifying potential threats to the U.S. homeland and interests abroad because it enhances analysts' ability to identify, prioritize, and track terrorist operatives and their support networks in the United States and abroad, primarily using call chaining analysis. ... ~~~~~~~~~~~~~~~~~~~~~~~~~...a (TG//81//tff) Collection provisions of the BR Order (TS//SI//NF) The BR Order requires providers to provide daily an electronic copy of all records or telephony metadata. The Order defines telephony metadata as comprehensive communications routing information, including but not limited to session-identifying information (e.g., originating and terminating telephone number, International Mobile Subscriber Identity number, International Mobile Station Equipment Identity number, trunk identifier, telephone calling card numbers, and time and duration of call). Telephony metadata does not include the substantive content of communications or the names, address es, and financial information of customers. (TS//81//NF) Exhibit B For each renewal of requested authority, NSA must file with the FISC a report that describes, among other things, proposed significant changes to the way in which the CDRs are received from providers and significant changes to the controls NSA has in place to receive, store, process, and disseminate BR metadata. (TS//81//NF) Exhibit C At least once during the authorization period of an Order, NSA's Office of General Counsel (OGC), its Office of the Director of Compliance (ODOC), the Department of Justice's National Security Division (DoJ NSD), and other appropriate NSA representatives must meet to assess compliance with the FISC Orders. Traditionally, this meeting must include a review of a sample of records obtained to ensure that only approved metadata is being acquired. The results of this meeting must be submitted Ttf? 8ECJtET1S'8El'NOFOR1¥ I REF ID:A4177277 DOCID: 4230264 fOfl SECRE'fWSES'NfJFOR N ST-12-0003 to the FISC in writing as part of any application to renew or reinstate the authority requested. Exhibit C of the application summarizes the quarterly "meeting..;·! Ithe requirement of the Order to review a sample of records obtained changed to a review of NSA's monitoring and assessment to ensure that only approved metadata are being acquired. {6)fff ..... (b)(3)-P.L. 86-36 (TSHSJ,£/~Jq tl'T'l.r. .... .._, I BR collection process I /,.,TI /1\TP\.1 - - .. .... -::::::<· ....... t~_ji·~·1··············· (~).f~l~P;·L~. 86-36 (oJm:)~OJ:iSC.~024(i) \\ ......... ... ·..... ··· .. ·· ..... \\"<\::.> :::: ....".:·· .... ·... . ._ '. ·· .. ',', \ \. . . . . . . .>.... ....... ~ \ ". \ " :··.. "-..::··..... ". .. ... ', ·." ·." " _,__ ____...,......,..-_____________________________________________________ ·... ·... '(TS . II T'SI II 7i'NF)' .. \ \ " " \\ (TS TT I 'SI I '~!Frj Tl TOP lfECRET1S'Sf1S'NOFOR:N 2 ___, REF ID:A4177277 DOCID: 4230264 ST-12-0003 (6fr1) ........ (b )(3)-P .i.:~··s6·~36 ... (b)(3)~5Q USC 3024(i) ·._····..;;.······;;...·. . . . , . . . . - - - - - - - - - - - - - - - - - - - - - - - - - - - ' ffS//sr//UF) ·... (TSHSIHNF) Criteria to Assess Agency Compliance with the BR Order /(b)(1) /(b)(3)-P.L. 86-36 (TS//SlmJF) BR Order I f (TS//£1//WF) Minimization procedures in FISC BR Order require that NSA and DoJ NSD conduct oversight of NSA's activiti'-e-s.-T""'h""e Order states that at least once during the authorization period: ['fSh'Sf//NF] NSA's OGC, Office of Director of Compliance (ODOC), NSD/DoJ, and any other appropriate NSA representatives shall meet for the purpose of assessing compliance with the Court's Order. Included in this meeting will be a revie'v of a sample of the call detail records obtained to ensure that only approved metadata is being acquired. The results of this meeting shall be reduced to writing and submitted to the Court as part of any application to renew or reinstate the authority requested herein.,D hhe.r~_quirement of the Order to review a sample of records obtaine4< changed to a review ofNSA'.s... monitoring and assessment to ensure that only apprbved metadata is being acquired. For···ertch.re.gewal of requested authority, NSA must,AJ.le with the FISC a report that describes, among etlie'r-·tb.ings, a description of any prop\)sed significant changes in the \Vay in w-Iuch records 'voilla··be-re.ceived from the Ploviders and any significant changes to controls NSA has in place to receive,.stqre, p~:6cess, and disseminate the BR metadata. ·(b)(1) I (b)(3)-P.L. 86-36 (U) Internal controls (U) NSA/CSS Policy 7-3, Managers' Internal Control Program, 14 February 2012, implements the Government Accountability Office's Standards for Internal Control in the Federal Government, November 1999. Policy 7 -3 requires managers "to institute the needed controls." According to the policy, internal control is 1 I ! C/1ML 'fO USA. FVffY21 .. ···· ( 1 ('f3f.:3Ii.1<1F) On the basis of factual and practical considerations of evervdav life bn which reasonable and rudent persons act, there is a RAS that the selection term to be queried is ...··· ..__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ..,_,,:; (b)(1) ............ TOP 8ECR£vS'ShS'NOFOR::lV 3 ................. . (b)(3)-P.L. 86-36 (b)(3)-50 USC 3024(i) REF ID:A4177277 DOCID: 4230264 f(JP SEC'ltl!:Ttl'SI/1!VYJF"Olt !"1 ST-12-0003 [U] a system of guidance, instrnctions, policies, regulations, procedures, rules or other organizational instrnctions intended to determine the methods to be employed to carry out mission or operational actions or objectives, and ensure that programs achieve intended results. (TS//SI//PfF) Table 1 depicts NSA/CSS Policy 7-3 categories of internal controls, which we applied to the BR FISA Program to evaluate the risk of non-compliance . (U) Table 1: NSA/CSS Policy 7-3 Categories of Internal Controls (U) Documentation (U) Established written procedures that are complete, accurate, and available for examination. Consists of regulations, policies, procedures, and/or standard operating procedures. Record (U) A written description of what has happened. Structure (U) Key duties and responsibilities in authorizing, processing, recording, and reviewing official NSA/CSS transactions should be separated among individuals. Managers should exercise appropriate oversight to ensure that individuals do not exceed or abuse their assigned authorities. Authorization (U) Procedures are in place to prevent people from exceeding their authority or misusing government resources. Management (U) Consists of the assignment, review, and approval of work. This control requires that management provide guidance and training to reduce loss of resources and increase achievement of results. Security (U) Any method or device that can be used to restrict access to government resources. This control may utilize safes, vaults, locked rooms, locked desk drawers, computer log-on identification , and passwords. (U) TOP SEGlfEJ;,@'J.«NOFQRN 4 DOCID: 42302 64 1 REF ID:A4177277 TOP SECR£ T1?fYl!7'f'<1 l. OH(~)~P~·L •. 86-36 (b}(.3)~0 USC-3.Q24(i) (TSHSIHNF) Fl NDI NG: BR Program Processes Must Be Improved \,:··...... ·."tf.SffSHtMr)O~her than the compliance incident described, we found no other \..... inst~nce of no·n~·compliance with the Order for BR Retention during CY2011. \ . .....Howev.er, the Agency·myst centralize program documentation and make it . apcessib.{e to all stakehoiderS..~.. Notes of mandator uarterl meetin s with DoJ NSD ersonnel should be ke t:·· . Management reviews of sampling thaf personnel perform is required. The structure code used in sampling should be reviewed and updated periodica y. mprovmg processes wt s rengthen the controls already in place and help reduce risk of non-compliance. (U) Non-Compliance Incidents ················· .····· ···· ... · :::"'::·:'°(b).(1) . (bj(3)~P .L. 86-36 ....... ... ············· \ ···-.::·-... f'fS / /SI// PJ.f.) . .O.urifig·ot;:·; review, . N-SAfffed with the FI~C la. Notice of Complil:lJJ.ce··Tiicident in accordance :With _ _ _ _...._.,..,......._ Qf the FISC Rules of Procedure ex lainin that l·tec~~ds ·contained credit card·-..... information. Notification to the FISC was not provided upon recognition\. because it was OGC's understanding that this must be reported only wheh credit card information was viewable by analysts inl Subsequently, OGC learned that DoJ maintains a different view: (b){3)~P;:~:·:·86,,3.~. ........ Identification of credit card information, regardless of whether the credit ·······... ..... ·cafl:linformati.QP.:. was viewable b intelligence analysts, must be reported to ·the·C.o.µrt. Accord.fri···to analysts, credit card information never .~ntered ·. ···...... because the parser rules prevented the fields .................... . ....... · ··· · ··· ··· from being ingeste·d·; · is responsible for updating parser rules and (b)(1) performs daily and weekly sampling of records to identify non-compliant (b)(3)-P.L 86-36 data. l I I : .·· _,:·:-'····· (b)(1) (b)(3)-P.L. 86-36 (b)(3)-50 use 3024(i) TOP b'ECllETIS'!·if;S'NOf'O-JtN 5 REF I:D:A4177277 DOCID: 42302'64 TOP 8ECR£T1S~ }1$'hYJFOR N 1 ST-12-0003 (U) Program Documentation ('FS//Sl//NF) The BR FISA program has two SharePoint sites that contain historical information on the program. One site is maintained by the ........... ······I land contains e-mails from a former BR "('6)('3)~P~L;··ss::3s··'····FISA.Prn.g:r!'l:P.'.l.. M8.:~~l:l:.&.~.~· .... !.?~ second .site is sporadically maintained by ·... .. personnel from Oversight ana···com:phance···(SV);··I ··..... ·· ...... .. ·· ...... (TS//SI//PtF) The BR FISA Program Lead is concerned that program documentation is in different locations and is not updated on the Share Point site. He believes that all program material should be maintained centrally and be easily accessible. He stated that a recent response to a Congressional "Question for the Record" might not have been consistent with a previous response because the historical information was not included on the Share Point site for reference . (TS//£1//PtF) In addition to a weekly compliance meeting with NSA stakeholders , there is a Court-mandated quarterly compliance meeting of DoJ NSD, NSA OGC, NSA ODOC, and other NSA personnel. Representatives of the OIG attended a quarterly compliance meeting on 1 February 2012, and questions regarding previous quarterly meetings were raised. The OIG observed NSA participants recollecting decisions and discussions from previous meetings. NSA does not maintain notes of discussions and decisions made during these mandatory meetings . ..{TS/ /SI//Npt' Part of the reason that program documentation is not updated centrally and meeting notes are not maintained is that the BR FISA Program no longer has a Program Management Office (PMO). The Program Lead has no staff and relies on individuals from various orn:anizations;·I I (U / /FOUO) Effect Lack of complete and final historical documentation in a central location accessible to stakeholders could lead to misinformed decisions and reporting on the program. (TS/!Sl/l~dF) Consolidate and maintain all final BR FISA Program material in a central location accessible to NSA stakeholders. (ACTION: BR FISA Program Lead) TOP SECRET1$~'f1$'!IOFO-R:N 6 REF ID:A4177277 DOCID: 4230264 TOP 8tJCltE'f1Wih~'NOFOlfN ST-12-0003 (U} Management Response (U) AGREE All final program material will be maintained on the existing SV SharePoint site by 28 September 2012. (U} OIG Comment (U) Planned action meets the intent of the recommendation. Upon confirmation that all final program material is consolidated on the SV SharePoint site, the OIG will close this recommendation. Maintain meeting notes of quarterly compliance meetings with DoJ NSD. (ACTION: BR FISA Program Lead} (U} Management Response (U) AGREE Management will maintain quarterly compliance meeting documentation on the SV SharePoint site. (U} OIG Comment (U) Planned action meets the intent of the recommendation. Upon confirmation that management maintains quarterly compliance meeting notes on the SV SharePoint site, the OIG will close this recommendation. (U) Reconciling Data from Providers ... ····· ,,,_....... ..... ... I I .L .._,... ..._,I / 1\T '1.'1\ .i. I I ~ ... ....··· ('~)(1).. (b)(3)-PL 86"._~6 (b)(3)~so use 3b240l 'm~ \ .... ......, J ,·::; "-'.a. iT.:rT1'd--------------------------. I I - - TOP SECRETiS!f}ES'HOFORN 7 REF ID:A4177277 DOCID: 42302'64 f(}J> 8ECRET!751117v0Ftm IV ST-12-0003 I I J. ..:JI I .......... -····· ············· Recor.i~ue pEfrfocfr~ally ,___ _ _ _ _ _ _ _ _ _ _ ____. ItTSHSIJ/PdFl · INSA'receives. (ACTION: I (U) Management Response (U) AGREE For each provider, management will establish a reasonable periodicity for reconciliation that is technically feasible, yet meets OIG's recommendation. (U) OIG Comment (U) Planned action meets the intent of the recommendation. Upon confirmation that management has begun reconciling the number of BRs that providers send with the number NSA receives, the OIG will close this recommendation. (U) Management Reviews and Structure Code .ff.S.f/Sfj./PJF)Omanagement receives weekly BR FISA compliance reports ("b)'_(_;3):P~E:8'~~ 6 '. """'';;),p,f,\J?.i:W:eek.lY... ~t.::l,t:L.J:s reports and attends monthly project review meeyngj on ·········::~!~~~!~~:~£~~1.~ffI~~;~~~~l·:p·el;~;:~::i~i··~·J;t:~~1~~-~~~y0:=:;i~ng of records received from providers. ·······... (U / /FOUO) GAO's Standards for Intemal Control in the Federal Govemment include monitoring: Internal controls should generally be designed to ei1sure that ongoing monitoring occurs in the course of normal operations. This includes regular management and supervisory activities, comparisons, recori:ciliations, and other actions that personnel should take in performing their duties. (TS//SI//HFf0personnel run daily and weekly queries on CDRs to answer five questions, as part of the sampling process controls to verify NSA's compliance with the Order: \T TT'Sr, 'J\!Tj'l 1 . 1ms' I(J~-~-.__ ______________________________________ f()P SECR£f1~'S1W!fOF&RN 8 ..;;;;.._··~··· (b)(1) (b)(3)-P.L. 86-36 DOCID: REF ID:A4177277 42302·64 'fffP SECltEf17'S}";7'f'lOFORPl ST-12-0003 2. ffS//SI//NFj {b)(3)~PL. 86~36 ,........:==============::::::;-----------""'."."."'-!adhere to expectations? ...... · ...... ('PS// SI// M .F)-D-id-;:1======-..... . ,a~.~~:~:~:::t.o:::exp:ett~~t'i'~'~;~~!~-P .L. 86-36 3·;· ·(U//FOUO }I 4. 5. ('f3// 31// MF) Did I ladheie. to expectations? ('FS//SI//N.F) The res..1:1:1t.s.of0sampling are submitted to ODOC in weekly BR FISA COil:J,plianc«~···reports. ODOC compiles the information with other c.g.:inplia:rice··· reports and provides it to the Director of Compliance for review. DOC summarizes the weekly BR FISA compliance reports for DoJ NSD 's ........ ········· review before quarterly compliance review meetings. ('6)l3')'~P;'L!"86:':~?:.::::::::::::::::············ ...................... · ········ · :::···. (Sf'g/1~.Qfiffi'f::E:ar:JJ·qay;Duns manual scripts on all re~g.r.ds.for._I_ _ ___. that carry . cfilling can(#i'iiiihers·:···NSAi§.POt . g.u.thorize'dfo receive customer financial information· theiefOte ··:nchecks' ..fiies·e . feedsl Ifor credit card numlJ..er.sl I ··········a .... ··::::· ········· f6)Hf·::::::::::.:······· (b)l~)-P.L. .$&;;36...... (b)(l)·SO ... USC 3024(il ······ ........ '~~ .&....... I ·;·;:;·~·//T\1·1Nlr---------------------------.I -- ·. ·. ;f!W'/51//Nfl) Effect Management's review of the sampling of records that L.Jpersonnel perform will provide a la er of oversi ht and hel ensure com Hance with the.Order.· (U/lFOUO) Implement personnel perform. ~~anag~ment r~yiew process ~f. th~. .samplirig. thatO ···· ·· ·· · . . ........... (ACTION: J TOP 8ECR£Ti~ 18JM'lOFO/Ll\l 9 DOCID: 4230264 REF J:D:A4177277 TttP SEC1tE'f1nW1}'NOP&R N ST-12-0003 (U) Management Response (U) AGREE Management implemented a review process in June 2012 to review the sampling during the team's monthly project review meeting. (U) OIG Comment (U) Planned action meets the intent of the recommendation. This recommendation is closed. (UJ~FOUS) Implement a process to conduct periodic reviews of the structure code. I (ACTION:l...._ __,... (U) Management Response . ~~!~~~etoQ~·J.:f1"m~!~!!~';;;;;~n~~j;'~~:'.d!'~.~:.~~:.~. ~~~~/ (U) OIG Comment (U) Planned action meets the intent of the recommendation. This recommendation is closed. 'f'OP 8ECltELS'8bSCJYO:POfl::N JO (b)(3)-P.L. 86-36 DOCID: REF ID:A4177277 4230264 'f()P StsCR£T17'Sl17'NOF&R:Jil ST-12-0003 Ill. (U) OBSERVATIONS (U) Parameters for Defining Significant Changes (TS//SI//NF) Guidelines have not been established to define significant changes in the way that records are received or in the controls NSA has in place for BR metadata. (TS//SI//NF) For each renewal of authority, NSA must file with the FISC a report, Exhibit B, that describes significant changes proposed to the way records are received from providers and to the controls NSA has in place to receive, store, process, and disseminate records. (TS//SI//Nfi') The OIG asked stakeholders for the definition of "significant" or agreed -on guidelines for determinin!! sil!11ificance. but a common defhliti()P. ...<;.Q.nld...not . he . identified.-1 ('6)(1L::::::::::::.:::::::::·:::. :..... . (b)(3)-P.L··ss~.~s (b)(3)-50 USC 3024(i) .. ....___.,..,._.I OGC believes that items that could be considered significant would be discussed during the weekly BR FISA meetings. NSA OGC and DoJ NSD ultimately determine what is reported as significant. (TSl/Slh'PdF) Nonexistent guidelines for determining significant changes might cause inconsistent reporting. BR FISA program stakeholders should consider defining "significant. " (b)(3")~Pi. 86~36 (TS//SI//NF) NSA has been inconsistent in its Exhibit B reports to the FISC about whether there have been significant changes. The OIG reviewed all Exhibit B reports for CY2011 to identify the information included in these reports and determine whether significant changes to controls had been ...reported. OfthQExhibit B filings in CY2011, only one did not mention whether NSA is proposing significant changes to the way records are received from providers or significant changes to controls. t-+t-¥i'M~+l'd+-+lnconsistent reporting in Exhibit B might cause confusion about whether significant changes have occurred. Establishing consistent criteria in Exhibit B reporting should be considered. TttP St:CftE TJS'SLS'PlOFORI'! 11 DOCID: 4230264 REF ID:A4177277 TOP SECftET!7'Sf17'!'1'0FOR l,,_.// S./. / ... 3 86 36 (b)( )-P.L. - reviews of the structure code. / TOP 8£CRET1S'Sl.'WfilO"FOltlii' 13 DOCID: REF ID:A4177277 42302,64 f(jfl SEC'Jtf!:ftl:'5Jl/N(jF()R 1V ST-12-0003 (U) This page intentionally left blank. TOP SEC:ltEfJ~'Sff/l~rOFORN 14 REF ID:A4177277 DOCID: 4230264 TOP 8£CRE1W81WNOFORN ST-12-0003 V. (U) ABBREVIATIONS AND ORGANIZATIONS (U) (U) (U) (U) (U) (U) (U) (U) (U) (U) BR CDR DoJ FISA FISC FTP GAO NSD ODOC OGC (u)r---i L____J .... ··· ......... (U) PMO (U) RAS (U) SCIF ~g;EJ ... I Business Records Call Detail Record Department of Justice Foreign Intelligence Surveillance Act Foreign Intelligence Surveillance Court File Transfer Protocol Government Accountability Office National Security Division Office of the Director of Compliance Office of General Counsel I· · · · ············ Pri:fgram .. Ma.µ~gement Offke··. ···· ... Reasonable Artic.ula:ble··.Su~.picion ·· ···· ·· .... Sensitive Compartmentel InfOrrii.ation...F.a.GJlity······· ... r ~" ,~=~~";"";:,;,;;;;;"~"~"""'""· / ..-::· ... ::;; .... · -rehuild,.. analysfs were redirected...to . .the ../:·...·::;;;::::::::.:;:;;;;;;;;,:::'· ·········;···;··· ...,..·c············r·····"······· .................. -;.............................................. .___ _ _ _ ____, . ·<:;;;;:,,=;;>;'' '"' ...... ·~:::..J'f8J.::t....l3.Jf:I:f:~:~l:L....................... ......... ·lis the system backup that stores an exact ,..:;:i::::::;<;:; .. ·--::::;;::::::::::,::::::::::::::{urifotmatted) copy of the raw BR metadata received from telecommunications . ~i~i.~;:~. ;:o;~~,r~~Ft ~~~~~~'.r~{~·-~~;c;;'!!!':a~ -~S.)?.aved·! (TO//SI//t{:Fl e1ectronfoanrstored··ml...._______. ]to tape h'ackup,,!, ... is s stem that contain:·s softwar~,,;~t"~{i1i§"'otf"al....... servers. The system c'-u-rr-e=n'"':'t.,...ly""'"··"'"h-o·.,...ld.,...·s'"'"I_ _._____, 'ffJP /5t!CltEfll!~71ll>v'fJPOltN B-1 DOCID: 4230264 REF ID:A4177277 TO-P 8£CR£T17'8lt5'NOF&R l•l ST-12-0003 {U) This page intentionally left blank. T01" 8£CR£T15'8f15'-Ar0F()Jt!'I" B-2 t .· DOCID: 4230264 REF ID:A4177277 rep 8£CRlS1/~{S'JP-IVOFORlV ST-12-0003 (U) APPENDIX C (TSHSIHNF) Agency Foreign Intelligence Surveillance Act Business Records Collection Stakeholders 'fO? SECJtET17'SJ17UOFOR:N DOCID: 4230264 REF ID:A4177277 TOP SECftE'fJ'1f'Jf175'v'OFOR l'I' ST-12-0003 (U) This page intentionally left blank. TOP A'ECl?ETi~th'JASWJFORN REF ID:A4177277 DOCID: 4230264 TOP 8ECRETiS 18liVN.fJ.:.T:i'()RlV ST-12-0003 (TSNSIHNF) AGENCY FOREIGN INTELLIGENCE SURVEILLANCE ACT (FISA) BUSINESS RECORDS (BR) COLLECTION STAKEHOLDERS . 0 ::::"''(b)(1) (TS/f SI//NF) The following organizations are the primary stakeholgers.Jo:f ~R collection . ...... ·· · ('b)(;~)~g;:b~'86~36'' ··· .. • · .... · ··· :P;ogram I\ilaiiagement Divis~~n·: ··· manages the relationshi {b){1J:::::"'"""'"'"''"'"''· (b)(3)-P~L·86~~(f''" (b)(3)-50 USC 3024(i). .. :. / / witl;i. ..the i/l~~-P.L. 35 -35 \ .. .----'-........+-~~~-.......,.,--~~...,..,......;.,....1 prov1 er$ (TS// SI// pf'lt)I .. .. Iis responsible for NSA's r·ep.q.sitor of bulk metadata··ooll~.cted under the BR FISA metadata collection roces.s": centrall mana es tlie database mamtatn~· ..................... ...,_a_p_e..,....ackups("and ensures.that update !Pl Storage system are provide.4,.tol l • (TSJ/Si// ne.} I ........... ··· · ac mp an lfor 1 develops and implements ,,,stfucU:t,red,...database strateg~.~.s ..eotis1stent with ffie Agency's Enterprise .... · A.:r..chifectt1.r~ ..·I lma:fia~.~§. ...th4 Idatabase and provides continuous /;• ;• •:" :" ,:;!tX~tt?W:::~:::v:~:~~~:o:::::~~:~:~l:s~~~;;~n;:t · . .. ....:::::::::::::::::::::::::..:::::....:. ·····only····com·plia11t··d·ata··is·-i1:1:g·ested··i11to·· performs daily l th 11 d 'l d 'd 'f and weedy tests on e raw ca etai recor s to 1 enti y non-comp l'1ant d ata. Oalso completes the weekly BR FISA Compliance Report for the Office of the Director of Compliance. 86 36 ( )( 3)-P.L. .... .- . (TS// SI/ f NF) Office of the Director of Compliance (ODOC; D4), through its Monitoring Assessments and Special Compliance Activities divisions, is responsible for monitoring the Agency's activities for compliance with the terms of the Order. ODOC reviews the requirements of the Order and what NSA reports to the FISC, develops training, identifies end-to-end processes, ai1cl liaises with stakeholders. ODOC reviews the Weekly Compliance Report {6)(3)~PL86-36 ··········· prepareclbyCJ (TS// SI// NF) Office of General Counsel - Intelligence Law (D2 l) is responsible for completing authorization renewals, filing information with the FISC, interpreting the Order to the NSA workforce, and requesting modifications to the Order when necessary. f&P SECR£'f17~W17'/li'OFOttlvr C-1 DOCID: REF ID:A4177277 42302 64 1 T01n 8£CRETiS~WiSS¥0FOfl i'¥ ST-12-0003 (fS// SI// NF)' BR FISA Lead (ST) is fue Signals Intelligence Directorate representative for fue BR FISA program. The BR FISA Lead manages fue direction of fue program and ensures fuat fue program functions properly. The BR FISA Lead signs fue declaration fuat accompanies fue FISA BR application to fue FISC. _________ (8//SI//REL CfO USA, fi'VBYll ,..lis responsible .....,.. for the develo ment od I-software fuat runs on .a s stem fuat contain~ . servers·<' ..······· •, "· .• •'':::> .. .. " ··· ... ' ,' /,/ \/:.- (b)(3)-P.L. 86-36 'f&P 8ECR£f1S'8{1/'i.VO-FOR:N C-2 (b)(1) (b)(3)-P.L. 86-36 (b)(3)-50 use 3024(i) DOCID: 4230264 REF ID:A4177277 TOP ST:CttE'fWS:b'l/tlOFORlf ST-12-0003 (U) APPENDIX D (U) Full Text of Management Response TOL"1 Sti!CttE 177'Sill'N'()"JfrntJ \I DOCID: 4230264 REF ID:A4177277 TOP SECltE'f:WS1/J'NOFO-R N ST-12-0003 (U) This page intentionally left blank. TO:P 8ECRE'f1~'8IllNOFORPl REF ID:A4177277 DOCID: 4230264 f8PSECltE~'l'Sf»~OFOltl'V' ST-12-0003 SIGNALS INTELLIGENCE DIRECTORATE memorandum FROM:S02 16 July 2012 TO: Office of the Inspector Gemirnl (OlG) SUHJ: (1'5/;'Dl//Nf') Signals Intelligence Directornte (Sil)) Response to the OraftHeport on NSA Controls to Comply with lhe Foreign lntelligcmce Survt"illanc(! Court Orth~r Regarding Business Records Collection (Si'-12-0003) (U/~SID reviewed l:he subject draft reporl in its entirely and agrees with the recommendations. SfD'.s complete respon~, to include three requests for clos~tre, is attached. (U) Please cont;1~.~.__ _ _ _ __,ISID IG Liaison, on 963-2014s if you have any questions or concems. '(b)(3J•P.L.8.~:~~ . ·l. . ----------Deputy Chiefof Sti~ff for SIG!NT Policy and Corpora le lssuL'S End: a/5 CLASSIFIED BY: NSA/CSS!vl 1-52 LM'l'E D: 2UD701 OS DECl.ASSI FY ON: 20::171!717 'f'OI' TOP Sl:CR:ET/'Sli''~!OrOIH< .~t!CttEf1} S'f1S'NOFOMl 1 D-1 DOCID: REF ID:A4177277 42302 64 1 'fOP SECRET!'!'S.b?NOFOR l>l ST-12-0003 TO!' 91Xllf:'f/,'~f:i'NO!'OIH4 (1'9//Sl//NF) SID Response to the Drnft Report ()f1 NSA Controls to C~1111ply with the Foreign Intelligencc Surveilhmrn Court Ordt~r lJJ'? SID Comment; All final program mat~r:.ial wiU be maintained on the exi~ling SV SharePotnt site. While a good portion of the final prograh~ data exists on the site today, collecti011 nnd stomge of 011tsh1ndi11g dncumimts shotild be acrnmplishedl (V/ ;rouo) Poc:I J·"l~3-07lls .I R~alrii:h~ti'cfatfoHi ··. ·.. / (i$;5®11)'i'JfJ Mrrilltaill ofJiciiil mel'fi11g 11otes of q1111rterly c~·iiq1fj;lll~C w.~~ti11gs wit11 Do} NSD. (ACTION: BWffSA Prngm111 t1~ad) :.:.••.. b. )( )-P.l. _ 3 86 36 (U) Agree witl1 commt~nt .......... ········· ·· (l)) Estimnt11d Complct!n11 Datd ....::' ('f£//5E//W'7 SID Comment: SID re~1uesl:s that the Word ''<~f.fiti(. . meetings create suggested actions; however, these actions·:~fo not become o/Jicial until they •trVNOF&R::Ar ST-12-0003 .. ···:.,;::;;;;.'.}1(b)(3)-P.L. 86-36 .. -························ ...... -················ __ ... R.&oti:\in~riii~tioW4 .. · .... ···· . ...-::.-·· ..... / . . (ll/~ 1111vle111e1Jt 11 m1i.1y1g1mrl.'fifr~~;iew process of tl1e ~!1J11jill11,ftl11rtO)~rs9imcl pe1form. (ACTION: I I········· // ::....... / (U) Agree ·· · · (U) Es Lima led Completion Date: J\c uesti n ,. (U/ /~SID Comment: iiilpJertfonted a manage1mml: revi~wprocess to address this rec,immendation. lh<1nagementuow reviewOampling each month at the team's project review meeting. This procedure (s(>e chart below) is part ofa cw1tmlll~d collection of html.-formatled files that only team members have access to. SID resi1ectfurt}' requests Wat this il~!ln l>L~ considernd satisfied and closed. M(J11ihl!/ Tt1sks Task !Dh·cctions !Monthly project !i\i> . . . . (Ll/!~lniplcnwnt 11 prncess to crmd11ct periodic tt!l!Ji:ws ilftlu~ st1·11ct11re code. (ACTION: I I·· .......... .· (U) Agree ······· : ; ;.;.... (U) Estimated Completion Dale: I{eguesling doSU:1'e:· ·.:' (b){3)"P;L. 86-36. (U/ ~SID Commend .. · ····· lverifiei:fwith the Advanced lnteW. e!i.ce'ReseMcl\.ServkeB (ATRS) that the versions 9f..th~rsfa.ndards that.__,......... .-_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __. are valid and ri;ipmitrlli·e most recent versions . ./ ias implemented a process (see char! below) to review I lthe'siructure code to verify that software rcmaltl.S up-to-date. SID respectfully requests that this item be considered satisfied and ckisl~d. Revhm "il~o~r h:£c-li1 \'1 o;i,isr,;.li, .re.,c~e:rni: :lrt:vei l.r : si'fo"'i i"!s:"~Mt._1e--.do_c_u_11-1e-·n-t lf ,.....1s_t_c.,..,..1..,e'"'"o-\\....1'"'"11_0.... rt~~~~t~ 1~1~~ :~1~:~:~~~~ ::.:~~ :,;,t ,___ __,to ete.1tmi1e if ·which can de ect:-rl1e. resen.ce of credit card num?.~.~ ..!1t.clat:Lnlias a requin~ment to c 1a11gcs tm~ nui!d~!' In the Rcqt1est box pm th!! following 1cx1: (U//:-oc:::) Ple;ase lej: me know if the o o o f.o:.lowia . s1~anda;::cts tlave been 1..rnctatecl.: . :::'){b)(3)-P.L. ·: .. ··:..... ____________________ _..... 0 0 ,_ ..._ __. .··' .... ···· .. ··· ,.-·· o 0 () 0 0 In theJustification for Research Request,be:r(p~tthe follo,lii;1g text:.·_ .. ·· ·;':"'.1; ).17" Y.'r) 5022 SUbject: fF5H5t/fldF) Questions answered - Slgnals Intelligence Directorate (SID) Response to the D1aff: Report on N5A Con1rof.s to Comply with the Foreign Intelligence Su1velllance Court Order Regarding Business Records Colfectioo (ST-12-0003) - Recommendation 2 Date: F1iday, July 27, 2012 2:03:25 PM Classification: TOP SECRETHCOIVllNTh'MOFORt~ . . c::I (U / /FOUO) ~pologies for the delay in getting this to you - . ....········ fFS/:/~I/./MF)LJeviewed tl1e entire document and provides specific comments in response to the / ·· .......... ":: : : : : , , . .. . . . . · · · · · : : . ::::#~!ii. @\?£\..R!;'P?..1K.i.1h1g.Da.tn.frn111..P.i:o;:1.ide.rs.and.Recv1n111endtttio11··l;LJupporls tlle OIG's findi11gs in ( b )( 3)-P. L. 86-36 this section and provides tlle following technical feedback as considemtion for the action cited in Recommendation 2. Comments are specific to the i:ncliviclmu telecommunications providers. {'b)(1-) ....... (b)(3)-P .L. 86-36... (b)(3)-50 USC 30 '4(i) (1'9//61//?&) For each provider,Ozvill establish a reasonable periodicity for re<:onciliation Uiat is teclutlcally feasible yet meets OIG' s recomn:i:~dation. Additional teclutlcal discussions with tlle providers and developers of follow-on in-hous~·.µctivities >\':ill be required for full consiclerntion and implementation! .I (U//POUO) POC: ..... ~~~~~~~~~~--' (u/ /~._I_ _ _ ___,j .... 769-4058s / .............. ::./ (b)(3)-P.L. 86-36 SID IG LWSON OPS1 2N006 Suite 6245 dl sidigliaison 963-2014 iU/ /l"OtJO! T(H' S£CREnS'8bS'NOFOR:1¥ D-5 DOCID: 4230264 REF ID:A4177277 TOP 8ECRF:TiS'8hS~7V0/20R N ST-12-0003 (U) This page intentionally left blank. T01° SECRETiS1YhS'NOFOlllV D-6 DOCID: $230264 REF ID3A4177277 DOCID: #230264 REF