OSINT:Ripping Apart Surface and Dark Web via OSINT

The basic OSINT session program will teach participants a systematical, methodological approach to design, setup, and conduct investigations through open sources. We will show a practical, hands-on training that will give the students all the skills and tools they need to do an effective Open Source Intelligence research and analysis achieving good, reliable results. Participants will return home with all the abilities they need to execute a correct Internet Search by using CLI tools and frameworks

Agenda: Tools techniques & tactics to investigate data in surface, deep and dark-web

Transcript

1. Ripping Apart Surface and Dark Web via OSINT
2. #whoami 👉 Ashwani kumar a.k.a CyberK@Lki 👉 Hobbies – Gaming & finding new places to go solo 👉 Interest area – OSINT, Network Pentest, Malware analysis, Social engineering , DevSecOps, Geopolitics & Quantum physics 👉 Professional CS GO & Fortnite Player : Nick : GodWA 👉 Certified ICS Security skills from US Homeland Security & CISA 👉 Reported Scada bugs for BSNL, Railtel & other National critical infrastructures 👉 Provide OSINT training and consultancy to govt agencies & Private entities 👉 Work closely with CyberPeace NGO helping Indian masses to stay safe in cyberspace and build next-gen cyber warriors 2 Website – hack.cyberkalki.com
3. Key takeaways World Wide Web is composed of the three layers: Surface Web, Deep Web and Darknet. • Surface Web covers only 5 % of the World Wide Web. • Deep Web includes any kind of web services and constitutes a very interesting data source for OSINT investigations. • Darknet can be accessed only by using dedicated technology, such as TOR or I2P.
4. OSINT integration with DML model
5. Surface Web Efficient OSINT analysis requires Federated Search including the following data sources: • Search engines, such as Google, Bing, Yahoo, and others. • Deep web search engines, such as DuckDuckGo. • Social media, for example Twitter, Weibo, Instagram, Facebook, or LinkedIn. • RSS feeds from websites of interest. • New Data-as-a-Service providers, e.g. for financial figures or other information.
6. Surface Web Federated search retrieves information from a variety of sources via a search application built on top of one or more search engines whereas metasearch engine is an online Information retrieval tool that take input from a user and immediately query search engines for results. Sufficient data is gathered, ranked, and presented to the users. Ref: OSINT Open Source Intelligence tools resources methods techniques
7. Deep Web Contents on the Deep Web cannot be indexed because: • it is either protected with a password, such as your cloud storage, webmail solution, digital libraries, online magazines, or newspapers. • or it is stored behind web services or APIs preventing direct access to the raw data. • There are many deep web data sources available: • Google patent database • Google academic database • EU Sanction Lists • HaveIBeenPwned The extracted deep-web links helps OSINT analyst in use cases like crypto, fraud, criminal intelligence investigation scenarios to articulate and correlate hidden data points.
8. Dark Web - The Cult of OSINT The dark web is a subset of the internet that is accessed via special means, such as a TOR browser, and not immediately available from the clear net. Another common darknet is Zeronet. Each has different access requirements or methods. Different darknet details: • TOR: https://www.torproject.org • Zeronet: https://zeronet.io/ • I2P: https://geti2p.net/en/ • Freenet: https://freenetproject.org/index.html
9. Dark Web - Illegal Services
10. Dark Web - Legal Services
11. Is TOR Really Safe ?
12. How is traffic on TOR network ? Torflow is an uncharted app which places each relay on a world map and illustrate traffic exchanged between relays as animated dots. Similarly, tor-metrics shows list of exit nodes as per geolocation tagged. Some investigators will have a requirement to identify & monitor new .onion sites as they arise. This could be to observe patterns, identify new vectors, or simply to create additional pipelines of new .onion URLs to feed into custom crawling engines for advanced users.
13. DarkWeb TTP Tools , techniques and tactics for OSINT Investigation
14. Dark Web - Marketplace to watch • General Markets • PII & PHI • Credit Cards • Digital identities • Information Trading • Remote Access • Personal Documents • Electronic Wallets • Insider Threats
15. Dark Web - Sites examples
16. Dark Web - Product examples
17. Dark Web - Automating Threat hunting
18. Dark Web - Automating Crawler
19. Dark Web - Tools to try out • Scrapy • Tor • OnionScan • Privoxy • Elastic • Redis • Torbot • OnionSearch • Darkdump / Darksearch • Maltego / Lampyre • Hunchly • Searchlight / Spectrum • OnionIngestor / Poopak : Hidden service crawler
20. Dark Web - Scrapy
21. Dark Web : Katana Scanner Katana-ds (ds for dork_scanner) is a simple python tool that automates Google Hacking/Dorking and supports Tor. It becomes a more powerful in combination with GHDB. IT supports google dorking for finding exposure points & enumeration, help finding exposed PLC and SCADA devices with tor and proxy support.
22. Dark Web : Crawler & bots TorCrawl.py is a python script to crawl and extract (regular or onion) webpages through TOR network. Similarly, TorBot is an open source intelligence tool developed in python. The main objective of this project is to collect open data from the deep web (aka dark web) and with the help of data mining algorithms, collect as much information as possible and produce an interactive tree graph. The interactive tree graph module will be able to display the relations of the collected intelligence data. On same pattern, onionscan and other tools are made and widely used with variant features and scope of data intel.
23. Dark Web OnionScan OnionScan is a free and open source tool for investigating the Dark Web. It helps to detect :- • Build a better fingerprint of your server, including php and other software versions. • Determine client IP addresses if you are co-hosting a clearnet site. • Determine your IP address if your setup allows. • Determine other sites you are co-hosting. • Determine how active your site is. • Find secret or hidden areas of your site • Open Directories • Server Fingerprint • Analytics IDs • Protocol Detection
24. Dark Web : Dumping darkweb links Tools like Onionsearch, darkdump, darksearch performs scanning of top darknet forums, websites and marketplaces against provided keywords and collate it to user for further analysis.
25. Dark Web :Monitoring hidden services
26. Q/A Session
27. Next Session Agenda • Understanding Darkweb and TOr • Planning and readiness for conducting darkweb investigation • Strategies and approaches based on use cases • Investigative workflow • Toolkit and tactics • Reporting and concerns • Challenges and workarounds
28. Follow my social media handles & website i.am.psy.ber_kalki