Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Esoteric sub-domain enumeration techniques

Bharath
July 15, 2017

Esoteric sub-domain enumeration techniques

Slides from the talk "Esoteric sub-domain enumeration techniques" given at Bugcrowd Levelup virtual conference 2017.

The repository for the talk is at: https://github.com/appsecco/bugcrowd-levelup-subdomain-enumeration

The repository contains:
- Slides from the talk
- Scripts from the talk
- Cheat sheet for the enumeration techniques covered
- Common sub-domains wordlist

Bharath

July 15, 2017
Tweet

More Decks by Bharath

Other Decks in Research

Transcript

  1. ABOUT ME Bharath Kumar Security Engineer @ Offensive Security Certified

    Professional(OSCP) I enjoy good books, coffee, camping and stargazing! Appsecco
  2. DEMO ENVIRONMENT Feel free to run the DNS & DNSSEC

    attacks from the talk against the following nameserver & domain: Nameserver: ns1.insecuredns.com Domain: insecuredns.com
  3. WHAT IS SUB-DOMAIN ENUMERATION? Sub-domain enumeration is the process of

    finding sub- domains for one or more domain(s).
  4. WHY SUB-DOMAIN ENUMERATION? Finding applications running on hidden, forgotten sub-

    domains may lead to uncovering critical vulnerabilities
  5. COMMON SUB-DOMAIN ENUMERATION TECHNIQUES 1. Google dorking 2. Using specialized

    search engines 3. Dictionary based enumeration 4. Sub-domain bruteforce 5. ASN discovery
  6. TECHNIQUES WE'LL LOOK INTO 1. Certificate Transparency 2. DNSSEC zone

    walking 3. DNS zone transfer 4. Passive recon using public datasets
  7. CERTIFICATE TRANSPARENCY(CT) Under CT, a Certificate Authority(CA) will have to

    publish all SSL/TLS certificates they issue in a public log Anyone can look through the CT logs and find certificates issued for a domain Details of known CT log files: https://www.certificate-transparency.org/known- logs
  8. CT - SIDE EFFECT CT logs by design contain all

    the certificates issued by a participating CA for any given domain By looking through the logs, an attacker can gather a lot of information about an organization’s infrastructure i.e. internal domains, email addresses in a completely passive manner
  9. SEARCHING THROUGH CT LOGS There are various search engines that

    collect the CT logs and let’s anyone search through them 1. 2. 3. https://crt.sh/ https://censys.io/ https://google.com/transparencyreport/https/ct/
  10. Output of a script that searches through CT Logs for

    a given domain and extracts sub- domains & emails
  11. CT LOGS - MITIGATION Not have SSL/TLS support. This approach

    is definitely not recommended Using wildcard certificates will avoid sub-domain names being listed in CT Logs but they are a security risk
  12. CT LOGS - MITIGATION Deploy your own Public Key Infrastructure(PKI)

    project by CloudFlare helps you build an internal PKI. by Cloudflare automates certificate management using a CFSSL. Opt out of CT logs but you’ll miss out on all the security benefits that CT provides Name redaction in CT logs let's you hide your sub- domain information in a CT log CFSSL Certmgr
  13. DNSSEC DNSSEC provides a layer of security by adding cryptographic

    signatures to existing DNS records These signatures are stored alongside common record types like A, AAAA, MX etc
  14. DNSSEC - NEW RECORDS Record Purpose RRSIG Contains a cryptographic

    signature. NSEC and NSEC3 For explicit denial-of-existence of a DNS record DNSKEY Contains a public signing key DS Contains the hash of a DNSKEY record
  15. DNSSEC - AUTHENTICATED DENIAL OF EXISTENCE(RFC 7129) In DNS, when

    client queries for a non- existent domain, the server must deny the existence of that domain. It is harder to do that in DNSSEC due to cryptographic signing.
  16. PROBLEMS WITH AUTHENTICATED DENIAL OF EXISTENCE(DNSSEC) 1. NXDOMAIN responses are

    generic, attackers can spoof the responses 2. Signing the responses on the fly would mean a performance and security problem 3. Pre-signing every possible NXDOMAIN record is not possible as there will be infinite possibilities
  17. NSEC Zone entries are sorted alphabetically, and the NextSECure(NSEC) records

    point to the record after the one you looked up Basically, NSEC record says, “there are no subdomains between sub-domain X and sub- domain Y.” $ dig +dnssec @ns1.insecuredns.com firewallll.insecuredns.com ... snipped ... firewall.insecuredns.com. 604800 IN NSEC mail.insecuredns.com. A RRSIG NSEC ... snipped ...
  18. ZONE WALKING NSEC - LDNS The ldns-walk(part of ldnsutils) can

    be used to zone walk DNSSEC signed zone that uses NSEC. # zone walking with ldnsutils $ ldns-walk iana.org iana.org. iana.org. A NS SOA MX TXT AAAA RRSIG NSEC DNSKEY api.iana.org. CNAME RRSIG NSEC app.iana.org. CNAME RRSIG NSEC autodiscover.iana.org. CNAME RRSIG NSEC beta.iana.org. CNAME RRSIG NSEC data.iana.org. CNAME RRSIG NSEC dev.iana.org. CNAME RRSIG NSEC ftp.iana.org. CNAME RRSIG NSEC ^C
  19. INSTALLING LDNSUTILS # On Debian/Ubuntu $ sudo apt-get install ldnsutils

    # On Redhat/CentOS $ sudo yum install ldns # You may need to do $ sudo yum install -y epel-release
  20. ZONE WALKING NSEC - DIG You can list all the

    sub-domains by following the linked list of NSEC records of existing domains. $ dig +short NSEC api.nasa.gov apm.nasa.gov. CNAME RRSIG NSEC $ dig +short NSEC apm.nasa.gov apmcpr.nasa.gov. A RRSIG NSEC
  21. EXTRACTING THE SUB-DOMAIN FROM NSEC You can extract the specific

    sub-domain part using awk utility. $ dig +short NSEC api.nasa.gov | awk '{print $1;}' apm.nasa.gov.
  22. NSEC3 The NSEC3 record is like an NSEC record, but,

    NSEC3 provides a signed gap of hashes of domain names. Returning hashes was intended to prevent zone enumeration(or make it expensive). 231SPNAMH63428R68U7BV359PFPJI2FC.example.com. NSEC3 1 0 3 ABCDEF NKDO8UKT2STOL6EJRD1EKVD1BQ2688DM A NS SOA TXT AAAA RRSIG DNSKEY NSEC3PARAM NKDO8UKT2STOL6EJRD1EKVD1BQ2688DM.example.com. NSEC3 1 0 3 ABCDEF 231SPNAMH63428R68U7BV359PFPJI2FC A TXT AAAA RRSIG
  23. GENERATING NSEC3 HASH FOR A DOMAIN NAME ldns-nsec3-hash(part of ldnsutils)

    generates NSEC3 hash of domain name for a given salt value and number of iterations Number of iterations & salt value is available as part of NSEC3 record. $ ldns-nsec3-hash -t 3 -s ABCDEF example.com 231spnamh63428r68u7bv359pfpji2fc. $ ldns-nsec3-hash -t 3 -s ABCDEF www.example.com nkdo8ukt2stol6ejrd1ekvd1bq2688dm.
  24. ZONE WALKING NSEC3 An attacker can collect all the sub-domain

    hashes and crack the hashes offline Tools like , help us automate collecting NSEC3 hases and cracking the hashes nsec3walker nsec3map
  25. ZONE WALKING NSEC3 Zone walking NSEC3 protected zone using nsec3walker:

    # Collect NSEC3 hashes of a domain $ ./collect insecuredns.com > insecuredns.com.collect # Undo the hashing, expose the sub-domain information. $ ./unhash < insecuredns.com.collect > insecuredns.com.unhash
  26. ZONE WALKING NSEC3 # Checking the number of sucessfully cracked

    sub-domain hashes $ cat icann.org.unhash | grep "icann" | wc -l 45 # Listing only the sub-domain part from the unhashed data $ cat icann.org.unhash | grep "icann" | awk '{print $2;}' del.icann.org. access.icann.org. charts.icann.org. communications.icann.org. fellowship.icann.org. files.icann.org. forms.icann.org. mail.icann.org. maintenance.icann.org. new.icann.org. public.icann.org. research.icann.org. rs.icann.org. stream.icann.org. tally.icann.org.
  27. INSTALLING NSEC3WALKER Installation instructions are available at I used following

    commands to install nsec3walker on Ubuntu 16.04. build-essential package is a prerequisite. https://dnscurve.org/nsec3walker.html # Installing nsec3walker $ wget https://dnscurve.org/nsec3walker-20101223.tar.gz $ tar -xzf nsec3walker-20101223.tar.gz $ cd nsec3walker-20101223 $ make
  28. ZONE TRANSFER Zone transfer is a type of DNS transaction

    where a DNS server passes a copy of part of it's zone file to another DNS server.
  29. ZONE TRANSFER(ATTACK) If zone transfers are not securely configured, anyone

    can initiate a zone transfer against a nameserver and get a copy of the zone file. By design, zone file contains a lot of information about the zone and the hosts that reside in the zone.
  30. ZONE TRANSFER USING DIG $ dig AXFR @ns1.iitk.ac.in. iitk.ac.in iitk.ac.in.

    43200 IN SOA ns1.iitk.ac.in. root.ns1.iitk. iitk.ac.in. 43200 IN NS ns2.iitk.ac.in. iitk.ac.in. 43200 IN NS proxy.iitk.ac.in. home.iitk.ac.in. 43200 IN A 202.3.77.174 m3cloud.iitk.ac.in. 43200 IN A 103.246.106.161 mail.iitk.ac.in. 43200 IN A 202.3.77.162 ... snipped ... mail4.iitk.ac.in. 43200 IN A 202.3.77.189 webmail.iitk.ac.in. 43200 IN A 202.3.77.185 www.webmap.iitk.ac.in. 43200 IN A 202.3.77.74 wiki.iitk.ac.in. 43200 IN A 103.246.106.116 www.iitk.ac.in. 43200 IN A 202.3.77.184
  31. IS ZONE TRANSFER RELEVANT ANYMORE? Global zone transfers are hard

    to find on public DNS servers. It's common to find DNS servers with liberal zone transfer permissions in internal networks. Even the top level nameservers were accidentally configured to allow global DNS zone transfers. 1. 2. North Korea DNS leak Russian DNS leak
  32. ZONE TRANSFER - MITIGATION You can allow only specific IP

    addresses to initiate zone transfer against a nameserver The allow-transfer feature(in Bind) can be used to configure permissions # /etc/bind/named.conf.options has global bind settings. $ cat named.conf.options | grep "allow-transfer" allow-transfer { none; }; # /etc/bind/named.conf.local has config for individual zones zone "insecuredns.com" { type master; file "/etc/bind/zones/db.insecuredns.com.signed"; allow-transfer { 192.168.56.1; }; };
  33. EVADING IP BASED MITIGATION IP based restrictions are susceptible to

    IP address spoofing In an internal pentest, you can pretend to be the secondary nameserver, initiate a zone transfer and sniff the zone data
  34. ZONE TRANSFER - MITIGATION An added layer of security is

    to deploy DNS Transaction Signatures(TSIG) between the DNS nameservers TSIG uses shared secret keys and one-way hashing to provide a cryptographically secure means of authenticating each endpoint of a connection as being allowed to make or respond to a DNS update
  35. WHAT IS PASSIVE RECONNAISSANCE? In passive reconnaissance, an attacker gathers

    information without generating any traffic directly between him and the infrastructure managed by the target organization The objective is to be stealthy and leave low or no footprint
  36. PASSIVE RECON USING PUBLIC DATASETS and gather Internet wide scan

    data and make it available to researchers and the security community. This data includes port scans and a dump of all the DNS records that they can find. Find your needle in the haystack. scans.io Project Sonar
  37. PASSIVE RECON USING PUBLIC DATASETS Rapid7 publishes its Forward DNS

    study/dataset on scans.io project(it's a massive dataset, 20+ GB) This dataset aims to discover all domains found on the Internet The data format is a gzip-compressed JSON file so we can use jq utility to extract sub-domains of a specific domain: zcat 20170204-fdns.json.gz | \ jq -r 'if (.name | test("\\.example\\.com$")) then .name else empty end'
  38. MAKING CLOUDFLARE DO DNS ENUMERATION FOR YOU When you try

    to "Add site" to cloudflare account, cloudflare does some DNS enumeration and finds sub-domains that belong to the domain you entered
  39. DNS ENUMERATION THROUGH CLOUDFLARE 1. Login into cloudflare 2. "Add

    site" to your account 3. Provide the target domain as a site you want to add 4. Wait for cloudflare to dig through DNS data and display the results https://www.cloudflare.com/login https://www.cloudflare.com/a/add-site