Exploring, understanding and monitoring macOS activity with osquery

How can osquery help with security, devops, compliance and IT?

This talk from MacDevopsYVR 2018 provides an introduction to osquery for mac administrators (and is relevant to a wider audience).

Transcript

1. Exploring, understanding and monitoring macOS ac6vity with osquery Zach Wasserman Cofounder & Principal Engineer +
2. zach @ kolide.com
 github.com / zwass
 zwass @ osquery Slack twi@er.com / thezachw
3. The Problem
4. • Sysadmins and security folks have a huge number of sources for the data relevant to their operaCons and decision-making. • How can we reliably access this data to get an understanding of the system state in the present moment, and as it changes over Cme? The Problem
5. Introducing Osquery
6. Introducing Osquery • Open-sourced by Facebook in 2014. SCll supported by a core team at FB. • 4,367+ commits, 219+ contributors • Apache 2.0 License • osquery.io
7. Osquery Goals • First class support for macOS/Linux • Enable non-developers to access and aggregate data across disparate sources • Performance/reliability to deploy across corporate and producCon infrastructure
8. Unify disparate sources of informa6on • Flat files (/etc/hosts, /etc/crontab, ~/.ssh/known_hosts, etc.) • SQLite files (/var/db/SystemPolicy [GateKeeper configuraCon], etc.) • System APIs (Apple System Log, Keychain, SMC, CoreFoundaCon, etc.) • ApplicaCon APIs (Docker, Carbon Black, etc.) • Event-based APIs (FSEvents, OpenBSM, etc.) • Filesystem (Shared folders, file hashes, permissions, etc.) • Plists (/Library/Managed\ Installs/* [Munki data], etc.) • … And more …
9. The Power of SQL
10. account_policy_data acpi_tables ad_config alf alf_exceptions alf_explicit_auths alf_services app_schemes apps apt_sources arp_cache asl augeas authorization_mechanisms authorizations authorized_keys block_devices browser_plugins carbon_black_info carves certificates chrome_extensions cpu_time cpuid crashes crontab cups_destinations cups_jobs curl curl_certificate device_file device_firmware device_hash device_partitions disk_encryption disk_events dns_resolvers docker_container_labels docker_container_mounts docker_container_networks docker_container_ports docker_container_processes docker_container_stats docker_containers docker_image_labels docker_images docker_info docker_network_labels docker_networks docker_version docker_volume_labels docker_volumes etc_hosts etc_protocols etc_services event_taps extended_attributes fan_speed_sensors file file_events firefox_addons gatekeeper gatekeeper_approved_apps groups hardware_events hash homebrew_packages intel_me_info interface_addresses interface_details iokit_devicetree iokit_registry kernel_extensions kernel_info kernel_panics keychain_acls keychain_items known_hosts last launchd launchd_overrides listening_ports load_average logged_in_users magic managed_policies mdfind memory_devices mounts nfs_shares nvram opera_extensions os_version osquery_events osquery_extensions osquery_flags osquery_info osquery_packs osquery_registry osquery_schedule package_bom package_install_history package_receipts pci_devices platform_info plist power_sensors preferences process_envs process_events process_memory_map process_open_files process_open_sockets processes prometheus_metrics python_packages quicklook_cache routes safari_extensions sandboxes shared_folders sharing_preferences shell_history signature sip_config smbios_tables smc_keys startup_items sudoers suid_bin system_controls system_info temperature_sensors time time_machine_backups time_machine_destinations uptime usb_devices user_events user_groups user_interaction_events user_ssh_keys users virtual_memory_info wifi_networks wifi_status wifi_survey xprotect_entries xprotect_meta xprotect_reports yara yara_events osquery> SELECT * FROM...
11. The Power of SQL • select * from hosts; -- /etc/hosts • select * from smc_keys; -- SMC • select * from keychain_items; -- Keychain • select * from file_events; -- FSEvents • select * from hash where path = ‘/bin/bash'; -- File hashes
12. osquery> SELECT u.username, g.gid, g.groupname FROM users u JOIN user_groups ug USING (uid) JOIN groups g ON ug.gid = g.gid WHERE uid > 500;
13. Who's using osquery?
14. Digging In
15. osqueryi • CLI and interacCve shell for execuCng queries and viewing results • Use this as a part of scripts, or for manual exploraCon • Aher iteraCng on and understanding queries in osqueryi, evolve them to create monitoring via osqueryd (more later)
16. osqueryi
17. osqueryi
18. osqueryi
19. osqueryi
20. osqueryi
21. How can we interac5vely inves5gate system ac5vity using osqueryi?
22. osqueryi
23. osqueryi
24. osqueryi
25. osqueryi
26. osqueryi
27. Get structured output for scrip5ng
28. osqueryi
29. osqueryd • Schedule queries for conCnuous results • DifferenCal engine to see how state changes over Cme • Event-based tables ensure that data is not lost even when queries run on an interval
30. osqueryd { "schedule": { "all_apps": { "query": "SELECT * FROM apps", "interval": 60 } } }
31. osqueryd
32. osqueryd { "schedule": { "hardware_events": { "query": "SELECT * FROM hardware_events", "interval": 60 } } }
33. osqueryd
34. What to do with all this power?
35. What to do with all this power? Check out the community-sourced query packs h>p:/ /bit.ly/osx_a>acks_pack
36. What to do with all this power? Implement a central management server h>ps:/ /kolide.com/fleet
37. What to do with all this power? Push logs to ELK stack for dashboards, alerCng and archiving h>p:/ /bit.ly/elk_osquery_poG
38. What to do with all this power? CondiConally install sohware using Munki h>p:/ /bit.ly/osquery_munki_groob
39. What to do with all this power? Process/Socket AudiCng, File Integrity Monitoring h>p:/ /bit.ly/advanced_osquery_clong
40. What to do with all this power? Kolide Cloud h>ps:/ /kolide.com
41. Join us in osquery Slack bit.ly/osquery_slack StackOverflow: #osquery
42. Thank you! zach @ kolide.com
 github.com / zwass
 zwass @ osquery Slack 
 twi@er.com / thezachw