threats HUNTING (MANUAL) Proactive: Focus on new threats Vendor TI IR Alerting Hypotheses Hunting MA DF Alerting IR Vendor TI TI – threat intelligence MA – malware analysis DF – digital forensics IR – incident response
incidents without waiting for an alert : - iterative search through data - using Threat Intelligence - using knowledge about attackers’ tactics, techniques and procedures Reactive approach (alerting) Incident starts when notification comes in: - security system alert (NIDS/NIPS/AV/SIEM) - call from governmental agency - user complaint / report
join individual links into chains, ability to generate and investigate hypotheses; • OS Architecture: knowledge of native OS security mechanisms, knowledge of typical security issues of different operating systems, knowledge of specific attack vectors; • Network Architecture: understanding how computer networks work, OSI model, knowledge of TCP/IP, knowledge of basic protocols (DNS, DHCP, HTTP, SMTP, FTP, SMB); • Attack Lifecycle: understanding of typical attack stages (kill chain) and different events that happen at any given stage in an attack lifecycle; • Attack Methods/TTPs: understanding how an attacker attempts to penetrate your network, which attack vectors and tools he/she can use on different attack stages; • Log Analysis: knowledge of different log sources and event types generated by different sources, the ability to analyze logs for anomalies and pivot between data sources to see the big picture; • Network Forensics: the ability to read and understand packet capture data and determine the malicious nature of network traffic; • Tools, used for hunting and data analysis: how to use log analytics platform (e.g. Elasticsearch/Kibana), SIEM, how to use packet sniffer, how open PCAP, how to see and export logs in OS, how to collect logs from different sources and so on…
can be used Operating System logs Not all, only potentially useful. Good starting point Native OS capabilities “Real-time” processes’ activity Process start, DLL libraries loading, Process install driver, Process perform code injection, Process open port for incoming network connections, Process accept incoming network connections, Process initiate network connection, Process create/change file, Process create named pipe, Process create/change registry key/value Sysmon Auditd (Linux) Commercial EDR solutions Some of operating system events (process creation, file object access, registry key access) ASEP (Autostart Extension Points) enumeration The best event source to look for files or commands written to startup (almost all malware and attackers strive to obtain persistence in the compromised/infected systems) Autoruns/Autorunsc from Sysinternals Commercial EDR solutions Homemade scripts Periodic snapshots of different system tools output Try to find suspicious not based on real-time events, only by using periodic snapshots. E.g., periodic snapshots of active processes Arp Netstat Klist Osquery tasklist Interesting folders listing (e.g. Windows/AppData) Getting information about files in interesting folders with its metadata (hash, version info, creation time, etc) Sigtool from Sysinternals Homemade scripts Forensic artifacts Prefetch, AmCache, ShimCache, USN Journal, etc. Kansa ACE Homemade scripts
can be used Metadata of all downloaded files hash, size, name, MIME Type, Source URL, Referrer, used for downloading user-agent. Checking hashes against TI feeds Bro Suricata Proxy/NGFW logs Metadata of email headers / SMTP metadata To, From, Subject, received headers, size, used MTA, reception time, presence of an attachment Email server logs Bro Metadata of email attachments MD5 hash, size, name, MIME Type, link to the corresponding email metadata Bro Homemade scripts URL from email bodies Checking against threat intelligence feeds. Tracking emails with links to the file hostings. Checking against TI feeds Bro Homemade scripts Netflow Can be used to detect data exfiltration, worm malware activity, lateral movement, port scanning, checking remote IP-addresses against TI feeds nfcapd, nfdump … Outgoing HTTP/HTTPS Detection of communications with C2, data exfiltration, checking visited accessed URLs against TI feeds Proxy/NGFW logs Bro Outgoing DNS requests metadata Detection of DNS exfiltration, DNS tunneling. Checking requested hostnames against TI feeds Bro DNS server logs Metadata of SMB / RPC Detection of lateral movement, credentials dumping (DCSync, remote reg save), internal recon… Bro
IOC-based detection Tool-based detection TTP-based detection Attackers can use Mimikatz or similar tools to dump credentials from memory Looking for hashes (MD5, SHA1, SHA256) of Mimikatz and other similar tools executable files Looking for files with predefined names or extensions, that created by Mimikatz and other similar tools. For example, Mimikatz export dumped Kerberos tickets to the files with .kirbi extension and WCE drop DLL with predefined name wceaux.dll Looking for processes, that access lsass.exe memory or inject code to the lsass.exe memory Look for unsigned DLL libraries that loaded into lsass.exe Attackers can use PsExec/WinExe or similar tools for remote execution and lateral movement Looking for hashes (MD5, SHA1, SHA256) of PsExec/WinExe and other similar tools executable files Looking for installation of services with well-known names (PsExec installs service with name PSEXESVC, WinExe installs service with WINEXESVC) Looking for remotely installed services that spawn different processes Malicious software need to communicate with Command and Control center / Attackers use external servers for accepting back connections from compromised hosts Looking for communications with specific domains or IP- addresses, which are marked as an IOC or bad domains Looking for User-Agent specific for some hacking utility or penetration testing frameworks Looking for communications with domains generated by a certain algorithm, which is typical for a specific hacking utility or penetration testing frameworks Looking for connections with periodicity Looking for communications with randomly generated domains Looking for communications with newly registered domains
that allows to add information about the geographical location of IP addresses, based on data from the Maxmind GeoLite2 databases. This plugin is bundled with GeoLite2 City database out of the box. Commercial databases from Maxmind are also supported in this plugin. SourceIp enrichment with GeoIP DestinationIp enrichment with GeoIP In the Elasticsearch such enrichment will look like this
is for checking IP addresses in events against a list of network blocks that might contain it. Multiple addresses can be checked against multiple networks, any match succeeds. Upon success additional tags and/or fields can be added to the event. It is possible to specify subnets right in the Logstash configuration, or use external text file with the list of subnets. For non-external IP addresses enrich event with internal network zone tag: Check that IP address from the [event_data][SourceIP] field is internal or special purpose IP (127.0.0.1. multicast and so on): In the Elasticsearch such enrichment will look like this:
events with data pre-loaded from any remote database via JDBC. We can use it to get information about IP address and hostnames from CMDB database and put in right in the events. Enrich current event with information from CMDB about host, where this event is happened: We can make such enrichment for any field, where internal IP-addresses or hostnames can be presented: SourceIp, DestinationIp, computer_name, WorkstationName (from logon events) In the Elasticsearch such enrichment will look like this: Periodically get data from DB Enrich
filter allows to replace field value or add new filed to the event based on some key-value dictionary. This filter can be used to check values form different fields of the events against different IOCs feeds. Filter takes the value of a particular field from the event and checks whether it is the key in the dictionary or not. If so, the value from dictionary, is added to the event as a new field.
access mask (value of GrantedAccess field) from the event and put the list of the corresponding rights to the GrantedAccessList field. Obtaining rights based on the mask is done with Ruby filter.
https://github.com/Mark Baggett/freq freq_server.py – Mark Baggett's (SANS SEC573 Author) tool for detecting randomness using NLP techniques rather than pure entropy calculations. Uses character pair frequency analysis to determine the likelihood of tested strings of characters occurring. We can use freq_server.py to detect randomly generated service names. For communication with freq_server we will use Rest filter:
(except the fact that these are services of hacking tools) service names: Randomly generated service names (note, that frequency scores are much less):
Building information block about logon session for caching: Saving previously built information block in cache: Logstash Memcached filter allows to save some information from events in Memcached and after that use if for event enrichment. For example, we can use this filter for caching information about successful network logons.
network logon session info We can enrich Sysmon process creation events with previously cached information about network logon sessions. Such enrichment allows easily to track lateral movements.
network logon session info. Usage example Saved search “Lateral movement - process execution in network logon session”: ( event_id:1 AND source_name:*Sysmon AND event_data.LogonType:3 ) Enrichment from successful login event Get from Memcached, using LogonId as key Track processes, created in the network logon sessions
session info Some Windows events contain SubjectLogonId field. We can enrich such events with previously cached information about network logon sessions. In this example we enrich 4672 event with network logon session information. Such enrichment allows to track network logons using privileged accounts (4672 event – special privileges assigned to new logon).
to the network logon sessions info we can cache some information about created processes in Memcached for further enrichment of other Sysmon events, that are related to the created process: • Integrity Level; • User; • Command Line; • Parent Image. Building information block for caching: Saving previously built information block in cache (key is concatenation of ProcessGuid and computer_name):
information about parent process Get previously cached information about process from Memcached for enrichment of process creation events. Key is concatenation of computer_name and ParentProcessGuid:
information about parent process Set (key is concatenation of computer_name and ProcessGuid) Get (key is concatenation of computer_name and ParentProcessGuid)
about process We also can enrich different Sysmon events with additional information about process (Command Line User, Integrity Level, Parent Image), which was cached at the process creation time. It is possible because all Sysmon events have unique ProcessGuid:
real IP address from DHCP log Add to cache information about Hostname-IP combinations from the DHCP events Get from cache real IP of the workstation by its hostname. Bellow there is example of the successful network logon event in case of NTLM relay attack:
scheduled search queries (we call them ‘hunts’), that find potentially suspicious events and tag them with search name, stage of attack and if possible – MITRE Technique:
is possible to use commercial X-Pack Watcher from Elasticsearch. But also there are some open source projects. One of them is “411” – https://github.com/etsy/411:
data from the Elasticsearch and other sources, supported by 411. Filters allow to add, modify or remove Alerts from the Search pipeline. Filters are registered under Searches, with each Search having its own set of Filters. Targets allow to send generate Alerts to other services. Targets (like Filters) are registered under Searches, with each Search having its own set of Targets. It is possible to add new Target types to the 411. So, using this opportunity Target ‘Hunt’ was added.
send update request to the RabbitMQ. As tag this target uses search name. Along with tag it also send to the RabbtiMQ the array of attack stages and MITRE Technique: { "hunts":["suspicious_powershell_cmdline_downloading"], "attack_stages":["Command and Control"], "attack_ttps":["T1086: PowerShell"], "@metadata": { "_index":"logstash-2018.05.10", "_id":"d8rPg2MBVcQq8ZOpmKpK", "_type":"doc“ } } Hunt Target builds JSON like this and sends it to the RabbitMQ for further tagging of event in ES:
output for updating events with hunts Logstash RabbitMQ Input {"hunts":["suspicious_powershell_cmdline_downloading"],"attack_stages":["Command and Control"],"attack_ttps":["T1086: PowerShell"],"@metadata":{"_index":"logstash- 2018.05.10","_id":"d8rPg2MBVcQq8ZOpmKpK","_type":"doc"}} Example of data that 411 Hunt Target sends to the RabbitMQ for further event tagging
OVA format Do it after importing the VMs and configuring virtual network Step-by-Step descriptions of Lab attack scenarios How to configure your Lab virtual network
• Replace Sysmon config (file “config.xml”) on the Win2012DC VM • Replace Logstash config on the ELK VM (folder “logstash”) • Deploy hunts on the ELK VM (file “data.db”) • Import Kibana saved searches (file “kibana_all_staff.json) • If you are not going to perform attack scenarios, import test data to the Elasticsearch (file “TestEvens.rar”) How to do this is described in the instruction “!!!What you need to do before the start“.
bad DOC as an attachment • Execution via Dynamic Data Exchange (DDE) • Using different built-in tools to downaload and execute payloads (wmic, regsvr32, msiexec, bitsadmin, certutil) • Process Injection • Two-step privilege escalation: User -> NETWORK SERVICE (via weak service permissions) -> SYSTEM (via access token manipulation) • UAC Bypass, using Event Viewer • Lateral movement via service execution • Lateral movement via Task Scheduler • Credentials dumping from memory • Remote credentials dumping via DCSync • DCShadow • WMI Persistence • Task Scheduler Persistence https://yadi.sk/d/qB1PNBj_3ViWHe
examples Winword/Excel -> cmd – unusual, suspicious parent/child combination Word document with macros that launch cmd with obfuscated command line (obfuscation done by Invoke-DOSfuscation) Excel document that launch cmd via DDE -> nslookup for exfiltration of some environment information
examples Excel download, decode and execute DLL library via rundll32 (example of CSV Excel formula injection) Excel -> certutil/rundll32 – unusual, suspicious parent/child combinations Rundll32 -> cmd is also not so typical http://georgemauer.net/2017/10/07/csv-injection.html
Real-life examples Word documents with CVE-2018-0802 exploits Eqnedt32.exe -> any – unusual. Equation Editor usually doesn’t spawn any processes at all Also there are some other interesting techniques – executing file delivered via WebDAV, using mshta to execute hta from URL
of executing arbitrary code in the address space of a separate live process. • Running code in the context of another process may allow access to the process's memory, system/network resources, and possibly elevated privileges. • Execution via process injection may also evade detection from security products since the execution is masked under a legitimate process. http://struppigel.blogspot.co.uk/2017/07/process-injection-info-graphic.html Sysmon allows to detect this
about process injection techniques… https://www.endgame.com/blog/technical-blog/hunting- memory https://www.endgame.com/blog/technical-blog/ten-process- injection-techniques-technical-survey-common-and-trending- process
CreateUserThread ZwQueueApcThread Subtype 6 in our agent – code injection using CreateRemoteThread, NtCreateThreadEx or RtlCreateUserThread API Subtype 3 – code injection, using ZwQueueApcThread (old APC injection technique, Atom Bombing) Source Process Target Process
-(event_data.SourceImage:"*\\VBoxTray.exe" AND event_data.TargetImage:"*\\csrss.exe") AND -(event_data.StartFunction:EtwpNotificationThread AND event_data.SourceImage:"*\\rundll32.exe") Hunting. Parent/child process relationships Hands-on lab attack scenario 1. Meterpreter migrate command
arbitrary code There are a lot of built-in tools in Windows, that can be used to download remote payload. Some of them in addition to downloading can also be used to execute downloaded payload. Possible tools: powershell.exe, wmic.exe, regsvr32, rundll32.exe, mshta.exe, regasm.exe, regsvc.exe, odbcconf.exe, msbuild.exe, certutil.exe, bitsadmin.exe, ftp.exe... https://arno0x0x.wordpress.com/2017/11/20/windows-oneliners-to- download-remote-payload-and-execute-arbitrary-code/ https://gist.github.com/HarmJ0y/bb48307ffa663256e239
arbitrary code. Real-life examples Really suspicious ASEPs on this server Msiexec, regsvr32, ftp, and also there was WMI subscription… Another examples of msiexec in autorun
*url* *ping*) AND event_data.CommandLine:(*http* *ftp*) Hunting. Windows oneliners to download remote payload and execute arbitrary code. Hands-on lab attack scenario 1. Detection of certutil usage for downloading of Rotten Potato
(event_data.CommandLine:*bitsadmin* AND event_data.CommandLine:("*transfer* " "*addfile* " "*Add-BitsFile*" "*Start-BitsTransfer*")) OR ( event_data.CommandLine:*powershell* AND event_data.CommandLine:("*Add-BitsFile*" "*Start-BitsTransfer*")) Hunting. Windows oneliners to download remote payload and execute arbitrary code. Hands-on lab attack scenario 1. Detection of bitsadmin usage for downloading of Mimikatz
event_data.CommandLine:(*certutil*) AND event_data.CommandLine:(*decode*) Hunting. Windows oneliners to download remote payload and execute arbitrary code. Hands-on lab attack scenario 1. Detection of certutil usage for decoding of Rotten Potato and Mimikatz binaries
example Customized Mimikatz, that was downloaded using bitsadmin and decoded using certutil Bat-file, that was used to launch this Mimikatz Encoded Mimikatz binary
of an executable, legitimate or malicious, is manipulated or abused for the sake of evading defenses and observation. Several different variations of this technique have been observed. One variant is for an executable to be placed in a commonly trusted directory or given the name of a legitimate, trusted program. Alternatively, the filename given may be a close approximation of legitimate programs. Some of the often abused process file names: • svchost.exe • services.exe • winlogon.exe • csrss.exe • explorer.exe • lsass.exe • conhost.exe Look for creation, execution, or installation/presence in autorun of files with such (or similar) names.
token was changed via kernel exploitation Medium Integrity Level System Integrity Level System SID Process with Medium integrity level spawn process with System integrity level –> good sign of successful privilege escalation attack Launch whoami as System -> also good sign of successful privilege escalation attack
Integrity Level System Integrity Level Launch whoami as System -> also good sign of successful privilege escalation attack Launch sc to change service binary path as non admin -> probably attempt to escalate privileges via weak service permissions
event_data.Image:"*\\sc.exe" AND (event_data.CommandLine:(*start* *sdshow*) OR (event_data.CommandLine:*config* AND event_data.CommandLine:*binPath*)) AND event_data.IntegrityLevel:Medium Hunting. Privilege Escalation. Weak service permissions Hands-on lab attack scenario 1
functionality. Unfortunately, this is partly true. There a huge number of methods to bypass UAC, and the corresponding hacking tools for this are publicly available; • In fact, UAC can help only against low-qualified attackers; • Therefore, to limit privileges, an old trick is recommended – use separate admin accounts, instead of elevation via UAC. Hunting. UAC Bypass Protection Level Elevation Method Worse Turn off UAC Bad Automatically elevate administrators Good Run in admin-approval mode Better Run as standard user and elevate to a separate admin account. Best Run as standard user and switch user to a separate admin account instead of using UAC to elevate https://technet.microsoft.com/en-us/library/2007.09.securitywatch.aspx https://github.com/hfiref0x/UACME
( event_id:("1" "4688") AND event_data.ParentImage:"*\\eventvwr.exe" AND -event_data.Image:"*\\mmc.exe" ) OR ( event_id:13 AND event_data.TargetObject:"*\\mscfile\\shell\\op en\\command") Hunting. UAC Bypass using Event Viewer Hands-on lab attack scenario 1
login and password information from the operating system and software. • It is very common attack technique! Hunting. Credentials Dumping https://2017.zeronights.org/wp- content/uploads/materials/ZN17_Kheirkhabarov_Hunting_for_Credentials_Dumping_in_Windows_Environment.pdf
will be 4662 event corresponding to the incoming replication request DS-Replication-Get-Changes-All Non DC IP-address If the source of this replication request isn’t a DC, this is quite suspicious and can be the sign of DCSync attack
Saved search “Incoming Active Directory DB replication request from non DC”: event_id:4662 AND event_data.ObjectServer:DS AND -event_data.SubjectUserName:(*DC0*) AND event_data.ObjectType:"%{19195a5b-6da0-11d0- afd3-00c04fd930c9}" AND event_data.Properties:("{1131f6aa-9c07-11d1-f79f- 00c04fc2dcd2}" "{1131f6ad-9c07-11d1-f79f- 00c04fc2dcd2}") AND -enrich.cmdb.SourceIp.tags:dc Successful login event 4624 Enrichment from successful login event, using Logstash Memcached filter Request Memcached Get from Memcached
Real-life examples Some hacking tools and frameworks install services with random names that execute binaries form %systemroot%. Binaries, that generated by Metasploit, Impacket psexec.py, smbrelayx.py and other similar hack tools
examples Some hacking tools and frameworks install services that execute system binaries like cmd, rundll32, powershell and so on. Such services are quite suspicious: WannaCry Metasploit post-exploitation modules… Meterpreter getsystem command…
Hands-on lab attack scenario 1 Saved search “Suspicious services - executable from windows folder”: (event_id:("4697" "7045") OR (log_name:Autoruns AND event_data.Category:Services) ) AND event_data.CommandLine.keyword:/.*%[s|S][y|Y][s|S][t|T][e|E][m|M][r|R][o|O][o|O][t|T]%\\[^\\]*\.exe/ AND -event_data.CommandLine:(*paexe* *psexesvc* *winexesvc* *remcomsvc*)
hosts Hands-on lab attack scenario 1 Saved search “Access to Service Control Manager from non admin hosts”: event_id:5145 AND event_data.ShareName.keyword:*IPC* AND event_data.RelativeTargetName:svcctl AND -(enrich.cmdb.SourceIp.tags:admin OR enrich.assets.SourceIp.zone:administrativenet)
attack scenario 1 Saved search “Lateral movement via scheduled tasks”: event_id:("4698" "4702") AND event_data.LogonType:3 Enrichment from successful login event
hosts Hands-on lab attack scenario 1 Saved search “Privileged network logon from non admin host”: event_id:4672 AND event_data.LogonType:3 AND (event_data.SourceIp:* OR event_data.WorkstationName:*) AND -enrich.cmdb.SourceIp.tags:admin AND - enrich.cmdb.WorkstationName.tags:admin Enrichment from successful login events Kali VM Windows 7 (VICTIM) Lateral movement path: Kali VM -> VICTIM VM -> DC VM Get from Memcached
Saved search “WMI subscription creation”: (event_id:("19" OR "20" OR "21") AND source_name:*Sysmon) OR (event_id:"5858" AND user_data.Operation:(*EventConsumer* *EventFilter* *FilterToConsumerBinding*) )
mimikatz located in the lsadump module. It simulates the behavior of a Domain Controller (using protocols like RPC used only by DC) to inject its own data, bypassing most of the common security controls and including your SIEM. It allows to push any changes of existing AD objects without logging. Examples of possible changes: change the primary group as 519 (member of the Enterprise admin group), add the Enterprise admin group SID in the SIDHistory attribute, enable disabled account and much more… The attacks is done using the following steps: • registering the "DC" by creating 2 objects in the CN=Configuration partition and altering the SPN of the computer used; • pushing the data (triggered using DrsReplicaAdd, KCC or other internal AD events); • removing the object previously created to demote the DC. Want read more: https://www.dcshadow.com/ https://youtu.be/KILnU4FhQbc
altering the SPN of the computer used to perform attack. This action leaves a trace in the form of an appropriate Windows security event – 4742 with specific value of the “Service Principal Names” field: • the DRS service class (which has the well-known GUID E3514235–4B06–11D1-AB04– 00C04FC2DCD2); • the Global Catalog service class (which has the string “GC”). SPNs, that are required to perform DCShadow attack Computer, from which DCShadow attack is performed
creation of object in the CN=Configuration partition, in a server container. This action leaves a trace in the form of an appropriate Windows security event (4662 with specific value of Object Name and Parameter 1/2 fields). Server container of the configuration partition Created object. VICTIM is a name of the computer, from which DCShadow attack is performed
4662 event corresponding to the incoming replication request (as in case of DCSync) DS-Replication-Get-Changes-All Non DC IP-address If the source of this replication request isn’t a DC, this is quite suspicious and can be the sign of DCSync or DCShadow attack
“Possible DCShadow attack - suspicious SPN for non DC computer account”: event_id:4742 AND event_data.ServicePrincipalNames:(*E3514235* "*GC/*") Suspicious SPNs. Only real DCs should have it
“Possible DCShadow attack - suspicious DC object creation”: event_id:4662 AND event_data.ObjectServer:DS AND -event_data.SubjectUserName:(*DC0*) AND event_data.Properties:"%%7680 {bf967a92-0de6-11d0-a285-00aa003049e2}" AND event_data.AdditionalInfo:"CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration*" Enrichment from the corresponding successful network logon event, using Memcached filter
bad PDF as an attachment • Stealing NTLM hashes with a PDF from attachment • NTLM Relay attack • Remote code execution via service • Accessibility Features Backdoor • Logon (RDP) session hijacking https://yadi.sk/d/qB1PNBj_3ViWHe
search “Possible NTLM relay attack”: {"bool":{"must":[{"query_string":{"query":"event_id:4624 AND event_data.LogonType:3 AND event_data.AuthenticationPackageName:NTLM AND event_data.DhcpSourceIp:*"}},{"script":{"script":"doc['event_data.SourceIp.keyword '].value != doc['event_data.DhcpSourceIp.keyword'].value?.toLowerCase()"}}]}} SourceIp (from 4624 event) is differ from DhcpSourceIp (enrichment from DHCP log)
heirhabarov
blueb
shigashiyama
fumiyasac0921
sadnessojisan
chanyou0311
ishikawa_satoru
nikkei_engineer_recruiting
tammyeverts
asei
oracle4engineer
ryder472
4su_para
keavy
lemiorhan
addyosmani
marcduiker
paulrobertlloyd
chriscoyier
sugarenia
paigeccino
danielanewman
eileencodes
jponch
rmw
