Writing XKS Fingerprints @Vember 2010 455"." TOP SECRET . Agenda Naming Fingerprints Simple Keywords Boolean Logic Variables Context?Sensitive TOP SECRETHCUMINT TOP SECRET a? MFingerprints 101 What?s in a name? The XKS Fingerprint naming convention can help organize fingerprints and make searching easier so its important to make sure you name your fingerprint inline with the existing convention TOP SECRETHCUMINT TOP SECRET . a" A i - Ida-1H?- I What 5 'n a name For example, fingerprint names look like this: archive rar archive pkzip archive pkzip Notice the directory-like structure so that all fin erprints are within the same ?folder? and al encr tion/ archive fingerprints are within the same Wolder? TOP SECRETHCUMINT TOP SECRET .. - . - WWhat?s in a name This allows for smarter searching because you could look for all fingerprints by searching for or search for all archive fingerprints by searching for and etc. TOP SECRETHCUMINT TOP SECRET a: . i?HWhat?s in a name When you want to submit a new fingerprint, look to see if it would fit into any existing fingerprint folders. Best way to do this is to use either the ?Field Builder? or ?Tree Field Builder? next to the AppID+Fingeprints field in the search forms [fulltext]: TOP SECRETHCUMINT TOP SECRET - - . . K: WFWhat?s in a name The field builders allow you to browse existing fingerprint directories to see if one already exists for your new fingerprint TOP SECRETHCUMINT Field Build-er - [+Fingerprin13] .- prliea'tituna - EFEJradring E?j _ I3Ell'u1 - advertisement - analytite ant-n'y'rnizer - antivirua - application applications :a an - leaded-Jere blag _betnet 1* it TOP SECRET directories Field Builder Field Builder e. In +Fin er rinte eanD [+Fingernrint5] ?u '3 . . 113;! ic?w iranf irielfed i1 Echetjedw; tep icfw rridfiranfirielfed EHEWP?UWm?l-?hman?hiEldE? tap ic?w red;i iran iriel fed i1 H?lerla 113p ic?w rridfiran irielfed i1 furl tap ic?w rridlfirenfirielfed i2 tap icfw rridfiranfirielfed i3 FField Builder 15.an (+Fingerprint5] hemetrhleckl mp TOP TOP SECRET .. What?s in a name If no existing directory makes sense for your fingerprint, you can always create a new one. TOP SECRETHCUMINT TOP SECRET Fingerprints 101: Getting Started The first step is to define the name of the fingerprint. 0 To do that, follow the syntax below: archive test_new?) 2 TOP SECRETHCUMINT TOP SECRET Fingerprints 101: Getting Started Note that fingerprint names can not have spaces or any other punctuation other than which denote directories and which can be used in the place of spaces to make fingerprint names easier to read archive test_new?) TOP SECRETHCUMINT TOP-SECRET Fingerprints 101 0 As an example, let?s say we want to fingerprint traffic like this: E: I14 . ljlaa'ulijL-twu IE ?=ch Hes-J? tawah quILi -. gain: IE Begin REFER El Mnjahedeen Massage N2E1 Ejg1 Ern?r'EMTg 21mm thTE TUE muz?rzmeeuznnumw an magma-:1 NmFIEjlyr-J [:13ku dh'?'mUwEleHGlx'meyEDl: NjHE?fj ya I mam-1 ammo-H if Lee? whim hem ?e Hal LEI ##il End REFER El Mnjahedeen #21] Message Hi! rent I lelri=n Fuel-Ii: I TOP-SECRET Fingerprints 101 One thing that could be used to find data like this iS the string ASRAR El Mojahdeen V2.0 Message . E: I14 . IE ?=ch Hes-e tawah quILi Eegi N2E1 Ejg1 T1 DEIijdi QEDWFIMIQ :EjElegEij MTPLEN TUE cd MTf?lutEIDHle?? thj'u'h ij??nhr-JGExDTh NmFIEjlyr-J (313%!le dh?l?m NjHE?fj 3'9 1' DaTnhT?yufeEr?li?E aim?u? eEEDSHUIdUEWupz?hhgd-d an H?Mh+dynEq In an? ?injqu k1 v.15 3? Hal LEI ##il End REFER El Mnjahedeen #21] Message Hi! rent I HIrIrIa'n I TOP SECRET Fingerprints 101: Keywords 0 So let?s create a fingerprint to tag any data that contains that string ASRAR E1 Mojahdeen V2.0 Message TOP SECRETHCUMINT TOP SECRET Fingerprints 101: Keywords First we?d define the fingerprint with a name: 2 TOP SECRETHCUMINT TOP SECRET Fingerprints 101: Keywords Then, simply put the string in single quotes to denote that XKS needs to look for it as a keyword: E1 Mojahdeen V2.0 Message? TOP SECRETHCUMINT TOP SECRET Fingerprints 101: Keywords Finally, all fingerprint definitions need to end with a semi colon to tell XKS that the definition is finished El Mojahdeen V2.0 Message? TOP SECRETHCUMINT TOP SECRET Fingerprints 101: Keywords Using the fingerprint GUI on XKS Central, we can test to see if this compiles: Fingerprint llIl'ali-Ilati-n-n Suhmittal Step #1 Step #2 Step #3 itempilei Test J'l'lgainst Sessian Data Save Signature El Hgiah?ee? v2.0 I In; Results Congratulations, reur fingerprint was successfully eel-nailed! New use the Test butt-3n to run it against the designated sessien I: ata. TOP-SECRET Fingerprints 101 0 Once checked in, the fingerprint will hit on data like this: E: I14 . IE ?=le Hes-e tawah quILi Eegi Dwuwwm - - v1 21mm thTE TUE Prjuz'rzlueeuz mum-1w en magma-:1 NmFIEjlyr-J dh?r'mUwEleHGlx'meyEDl: NjHE?fj ya I Emma-1 :w-Lm-H eEEDsRUIdUEWupzuhhgd-muf if Lee? 9 55m ?e Hal LEI ##il End REFER El Mnjahedeen #21] Message Hi! rent I lelri=n Fuel-Ii: I TOP SECRET Fingerprints 101 As a second example, let?s say We want to find data like this: Using THT Farm atter Ref: June UT, Islamabad: Natienal Develapment Sampler Pic-1: Nemreet - Secter: Islamabad. AH Purchase SUBJECT QUDTATIGH AGAINST TUUR ENQUIRT REF:Purchaee cf RTU Siliccn DATED: Dear 51:; With reference tc pcur subject enquiryr we are pleased tc cur Quctaticn Hc: dated: pcur perusal. Please see the 'Terms cf Eale' attached with car ducte any further details. We hspe cur sffer suits year requirements and me lash fernard ts pear valuable purchase crder in due TOP SECRETHCUMINT TOP SECRET e; Fingerprints lOl Look for ke words that could be used to find traffic like is in the future. Using THT Farm atter Ref: June UT, Islamabad: Natianal Develspment Complex Flt-t Street mg:- Sectar: Islamabad. AH Purchase SUBJECT QUDTATIGH AGAINST TDUR ENQUIRY REF:Purchase cf RTU Siliccn DATED: Dear Sir; With reference tc pcur subject enquiryr we are pleased tc cur Quctaticn He: dated: fer pcur perusal. Please see the 'Terms cf Eale' attached with cur duste any further details. We hspe sur sffer suits year requirements and we lash fsruard ts pear valuable purchase crder in due TOP SECRETHCUMINT TOP SECRET in Fingerprints lOl <13? What if we looked for ?National Development Complex? and ?Quotation? Using THT Fern-I etter Netie?el Develepme?t Cemplex Seeter: Ielemebee. AH Purehe SUBJECT QUDTATIGH GAINST THUR ENQUIRY ef RTU Silieen DATED: Deer Sir; With referenee te yeur eubjeet enquiryr we ere te eer Quetetien He: deted: fer yeur perueel. the 'Terme er Eele' etteehed with eur quete fer any further deteile. We hepe eur effer euite yeur requirements and we leek fermerd te yeur valuable erder in due TOP SECRET Fingerprints 101: Boolean Logic Starting with these two keywords, we?d like to use Boolean Logic to create our new fingerprint 0 national development complex 0 quotation TOP SECRETHCUMINT TOP SECRET Fingerprints 101: Boolean Logic Again, step one think of a name: fingerprinthp pakistan/ agencies ndc?) TOP SECRETHCUMINT TOP SECRET Fingerprints 101: Boolean Logic 0 Step two, put single quotes around all keywords: fin gerprinthp pakistan agencies ndc?) 2 ?National Development Complex? ?quotation? TOP SECRETHCUMINT TOP SECRET Fingerprints 101: Boolean Logic 0 Use the Boolean operator and fingerprinthp pakistan/ agencies ndc?) ?National Development Complex? and ?quotation? TOP SECRETHCUMINT TOP SECRET Fingerprints 101: Boolean Logic 0 Finish the expression with the semi?colon. fingerprinthp pakistan/ agencies ndc?) ?National Development Complex? and ?quotation? TOP SECRETHCUMINT TOP SECRET fr' Fingerprints 101: Boolean Logic Use the fingerprint GUI to confirm the fingerprint definition compiles Fingerprint ?I'aIi-tlati-n-n Suhmittal Ste #2 Step #3 i Hel "ll Teet Data Sane Signalure 'netienel develepment eemplex' end 'quetetien'; HI I in}: SLI Reeults Cengratdlatidne, yeur fingerprint wee eernpiled! New use the Test butt-an td run it against the designated sessidn data. J. TOP SECRET Fingerprints 101 This fingerprint will now successfully find all sessions like this in the future! U?ng'?fouane?ar Ref: June Islamabad: Natianal lapment Cam Plat Na: Street He: Seetar: Islam Attn: AH Purchase EUEJECT QUDTATIUH AGAINST YOUR ENQUIRY REF:Purehase Bf Eiliaan DATED: Dear 511:r with reference ta raur subject enquire, we are pleased ta enalase aur Duatatian Na: dated: DTEDEJEDID, far yaur perusal. Please see the 'Terms sf Sale' attached with aur quate far an? further details. We hape aur affer suits yaur requirements and we leak farmard ta yaur valuable purchase arder in due TOP SECRETHCUMINT TOP SECRET .. . I EM. Fingerprints 101? However, how can we account for variations of how the traffic might be seen? Maybe ?National Development Complex? will be listed as Or maybe instead of a ?Quotation? it will be a ?Invoice? and etc. U?ng'?foosnoder 4 Ref: June Islamabad: National Development Complex Plot No: Street No:- Seotor: Islamabad. Mun: AH Furobase EUEJECT QUDTATIUH AGAINST YOUR ENQUIRY REF:PurEhase of Silicon DATED: Dear Eirr with referenoe to your subjeot enquiry, we are pleased to enolose our Quotation No: dated: DTEDEJEDID, for your perusal. Please see the 'Terms of Sale' attaohed with our quote for any further details. We hope our offer suits your requirements and me look forward to your valuable purchase order in due TOP SECRETHCUMINT TOP SECRET STRAPI Fingerprints 101: Boolean Logic Keywords can also be grouped together by parentheses to form more complex Boolean logic: TOP SECRETHCUMINT TOP SECRET Fingerprints 101: Boolean Logic 0 For example, we can expand on our previous fingerprint like so fin gerprinthp pakistan agencies ndc?) 2 (?National Development Complex? 01? NBC) and (?quotation? or ?invoice?) TOP SECRETHCUMINT TOP SECRET . 2/ Quick Aside 1: Context Sensitivity All keywords in are case? insensitive by default. 30 in the previous fingerprint will match on ndc, etc. TOP SECRETHCUMINT TOP SECRET a; Quick Aside 1: Context Sensitivity If you want to force a keyword to be case sensitive, simply append a after the single quotes. will only hit when NBC is found in all caps, or ?ndc?c will hit only when is found in all lower case and etc. TOP SECRETHCUMINT TOP SECRET Quick Aside 2: Keyword Scanning By default keywords in fingerprints can hit in substrings since for example ?ndc? is found within grandchildren. So this fingerprint Will hit on terms like: I grandchildren - handcard I handcuffs etc. TOP SECRETHCUMINT 455"." Wade 2: Keywor In specific cases to avoid false hits you can use the ?word? context. -- Or force there to be a space on either or both ends of the term by including them inside the single quotes So this fingerprint becomes: NDC OR: TOP SECRETHCUMINT TOP SECRET Fingerprints 101: Boolean Logic Let?s say that this fingerprint is producing good hits, but it also hitting on spam mails. (?National Development Complex? 0r and (?quotation? or ?invoice?) TOP SECRETHCUMINT TOP SECRET Fingerprints 101: Boolean Logic 0 We can use the Boolean and not to defeat unwanted traffic like below: fin gerprinthp pakistan agencies ndc?) 2 ((?National Development Complex? or NBC) and (?quotation? or ?invoice?)) and not (?Viagra? or ?herbal supplement?); TOP SECRETHCUMINT TOP SECRET Fingerprints 101: Variable Variables allow you to link to a list of keywords. For example, working with this fingerprint, we could create variables to each grouping of terms. fingerprinthp pakistan agencies ndc?) ((?National Development Complex? or and (?quotation? or ?invoice?)) and not (?Viagra? or ?herbal supplement?); TOP SECRETHCUMINT TOP SECRET rp- Fingerprints 101: Variables 9" Variables use the same syntax as fingerprints $NDC_terrns ?National Development Complex? 0r $procurement_terms ?quotation? or ?invoice?; $span1_defeats ?Viagra? or ?herbal supplement?; fingerprinthp pakistan agencies ndc?) ($NDC_terrns and $procurement_terms) and not $spam_defeats; TOP SECRETHCUMINT TOP SECRET ??ff if Fingerprints 101: Variables Variables can be re?used in multiple fingerprints. For example, we could have: fingerprinthp pakistan agencies ndc?) ($NDC_terms and $pr0curement_terms) and not $spam_defeats; $NDC_terms and (?missile launch? or ?tactical radio?); TOP SECRETHCUMINT TOP SECRET - --I .- MM. . fFingerprints 101: Variables In the future, you can modify the variable $NDC_terms and it will automatically affect both fingerprints Since they use that variable in their definition. TOP SECRETHCUMINT TOP SECRET What?s not 0 For example, take the first scenario: . f- want to look for documents from Iran that mention a banned item? 0 Just using keywords with Boolean equations, how could we restrict the term to only a document body and only coming from Iran? TOP SECRETHCUMINT TOP SECRET a: - . - ff Context Sensitive Scanning context sensitive scanning engine allows you to explicitly say where you want a term to hit. As an early example, the Tech Strings in Documents capability allowed to restrict terms to only Email, Chat or Documents Bodies The full XKS Context Sensitive Scanning engine allows for over 70 unique contexts to be used as part of an fingerprint TOP SECRETHCUMINT TOP SECRET .. .. Sensitive Sca ning For example, take the first scenario: want to look for documents from Iran that mention a banned item? 0 Using the XKS context for Country Code (based on NKB information) and the XKS context for Document Bodies, this easily becomes: cc(?ir?) and item?) TOP SECRETHCUMINT TOP SECRET .. .. Sensitive Sca ning 0 As another example, let?s say we want to tag all lphone usage 0 Using the XKS context for User Agent this easily becomes TOP SECRETHCUMINT TOP SECRET a: - i?c?u? . . - of RA Considerations XKS Fingerprints may not be USSID18 or HRA compliant if they are queried on by themselves For example, we may want to fingerprint the use of mobile web devices like the lPhone, so that attribute could be used as part of a more complex query. But querying for the lPhone ?ngerprint itself would be a USSID18 and I-IRA Violation. TOP SECRETHCUMINT TOP SECRET Considerations But if you want to look for an IPhone user from an Iranian Proxy accessing his Mail.ru account: IP address: Either [+Fingerprinte) [fullteet]: Field Builder 11me (+Fingerprinte] Field Builder i meilfweljmeilfmeilru I meilfweljrneilfmeilru AFFID Fmg?rpr'nt? meilfwebmeilfmeilrufattach breweerfcelIphenefipher'lel i I meil?weljrneilfmailrufpeet Addie Field TOP SECRETHCUMINT TOP-SECRET I . Context SenSItIve What contexts are available for use in XKS Fingerprints? TOP SECRETHCUMINT TOP SECRET HTTP Activity Contexts (1 of 2) html_title(eapr) The normalized extracted text web page titles to? and ?bomb?) http_host(expr) The ?Host? name given in the header. http_url(eapr) Every URL from HTTP GET and. POST commands. http_url_args(expr) All arguments given as part of a URL (ie. all text following the in a URL string) http_u http_referer(expr) The ?Refererz? URL given in the HTTP header http_language(expr) The normalized two letter iso?6393 language code as inferred from any and or header info or TOP SECRETHCUMINT TOP SECRET 2 HTTPActivity Conte;ts (2 0T http_c00kie(expr) The ?Cookie:? ?eld given in the header. http_server(expr) The "Server:? type name in the header. 1? 0r ?Apaehe? http_user_agent(eXpr) The ?User-Agent" ?eld given in the header. 0r ?Chmme?) web_search(expr) The normalized extracted text from web searches 0r ?plague?) x_f0marded_f0r(expr) The X?Forwarded For IP address from the HTTP Header TOP SECRETHCUMINT 455"." TOP SECRET sraspl . med Contexts 1 of 2 The source or destination IP address of the session from_ip(expr) The source IP address of the session to_ip(expr) Every URL From HTTP GET and POST commands. IP subnet in CIDR notation. The source or destination TCP or UDP port nLunber. from_port(expr) The source TCP or UDP port number. from _port(?22?) The destination TCP or UDP port number. TOP SECRETHCUMINT TDP SECRET rotocol Contexts 1 of 2 The country (either to OR from) based on IP address ee(?ir? or from_cc(expr) The souree country based on IP address or to_ee(expr) The destination country based on IP address or protocol(expt) The textual form of the 1P neat protocol. next_protocol(expr) The textual form of the next protocol. mae_address(expr) The MAC address of the target network device. 'l TOP SECRETHCUMINT ommunlcatlon Based Contexts email_body(expr) The normalized text of all email bodies. to? and ?b nild? and (?bomh? or ?weapon?? chat_body(expr) The UTF-8 normalized text of all chat bodies. to? and ?build? and (?bomb? or Weapon?) document_body(expr) The normalized text of the Omce document. O?ice documents include (but are not limited to) Microsoft O?ice, Open Office, Google Docs and Spreadsheets. to? and ?build? and (?bomb? or ?Weapon?D calendar_body(expr) The UTF-8 normalized text of all calendars. An example is Google Calendar. archive_files(expr) Matches a list of files from within an archive. For example is a ZIP ?le is transmitted, all names of?les within are passed to this context. or ?virus.doc') http_post_body(expr) The UTF-8 normalized text HTTP nrl-encoded POSTS. and ?badguy@yahoo?) TOP SECRET a Communication Based Contexts Aliases dec_email_body(expr) This eevers the email_body and decument_bedy contexts te? and ?build? and (?bemb? er ?weapen?D communicationjwed?expr) This eevere the email_bedy, deeument_bedy and ohat_bedy contexts te? and ?build? and (?bemb? er ?Weapen?D A guide to XKS contexts can be found TOP SECRETHCUMINT "f Context sensitivity Why use context?sensitive scanning? More intuitive you can say what you mean More accurate - if 'maps.google.com' is mentioned in a biog post, you don't want to try processing it as a Google Maps session Better performance for XKEYSCORE TOP SECRETHCUMINT TOP SECRET TOP-SECRET . Examples want to look for people doing web searches on Jihad from Kabul? Using the from_city() and web_search() context this becomes and TOP SECRETHCUMINT TOP-SECRET Examples 0 want to look for people using Mojahedeen Secrets from an IPhone? You can even use existing fingerprints in a fingerprint de?nition! So this becomes: and fingerprinthrowser/ cellphone /iphone?) TOP SECRETHCUMINT TOP-SECRET . I ?l Example 4 0 want to 100k for E?mails that mention words from various categories of interest to You can use multiple variables in an equation like this: and sachositions and ($acwc0untries 0r $acwbr0kers or $acwp0rts)); TOP SECRETHCUMINT TOP SECRET ml; 4 $acwitems ?machine gun? or ?grenade? 01* 47? $acwpositi0ns ?minister of defence? or ?defense minister? $acwc0untries ?somalia? or ?liberia? or ?sudan? $acwbr0kers ?south africa? 01' ?serbia? 0r ?bulgaria? CI- $acwp0rts ?I'angood? or ?albasra? or ?dar es salam? and $acwljositions and ($acwc0untries 0r $acwbr0kers 0r $acwp0rts)); TOP SECRETHCUMINT TOP SECRET 4/ a ew Fingerprint GUI 7 New XKS Fingerprint GUI allows to directly test, submit and manage fingerprints through the web Navigating-I Menu i3; Fingerprint 'u'eliclatienr' Suhmittal Fingerprints Step #1 Step #2 Sterne validate J. EUhm? I i - 5:12;: 3:72;? 1337.17.- an praise-2 F. Elm Elehal 'L-?eriehle De cleretinne a Type er: pasta any glebal DECLeaaTIens here. Signature Type er peete FINGERPRIHT definitien here- F'rese when clans editing 455"." rthUl New Fingerp 7 New XKS Fingerprint GUI allows to directly test, submit and manage fingerprints through the web 1uralluatluln Eltum ?le-p #2 Eltpi? Elebal ?u'arialsle Declaratinns Steet 'bemb' er 'mieele? er ?ied': Signature emeil_bedyi$teeti: Results Lengratulatisns. ueur ?nge'prin: was successfully eempilsd! New use the Test nutten ts run It against the Iseelgnatesl data. . lad-Ti: "Pf-'74: . . Questions? TOP SECRETHCUMINT - I I . ?j Syntax Rules The definition of the fingerprint will look like this: owner 2 2 Note the single quotes needed for the fingerprint name and owner TOP SECRETHCUMINT TOP SECRET TOP SECRET /"?fsf?fr Syntax Rules Secondly every fingerprint de?nition must be completed by a semi?colon. ?ngerprintCtest/blah/ something?, owner ?badguy?; TOP SECRETHCUMINT TOP SECRET Syntax Rules Variables also must be completed by a semi?colon. $badguy ?bomb? or ?gun? or ?weapon? owner $badguy; TOP SECRETHCUMINT TOP SECRET Syntax Rules Definitions and Variables can span multiple lines $badguy ?bomb? or ?gun? or ?weapon? ?ngerprintCtest/blah/ something?, owner -) $badguy; TOP SECRETHCUMINT