Maintaining control by letting go - security and devops

Talk at Infosecurity Europe all about the intersection of security and devops, focusing on the continuous delivery pipeline as an opportunity for conversation and automated security testing.

1. Maintaining control by letting go Puppet Labs Gareth Rushgrove Security in a devops world
2. @garethr
3. Gareth Rushgrove
4. Introduction
5. We can build more secure systems, not just react to problems better Gareth Rushgrove
6. Shadow IT Gareth Rushgrove Security VS
7. Self-service Gareth Rushgrove Security VS
8. VS Rapid software development Gareth Rushgrove Security
9. VS Rapid software deployment Gareth Rushgrove Security
10. VS New tools and technologies Gareth Rushgrove Security
11. Security as a solitary gatekeeper for all changes to production doesn’t scale Gareth Rushgrove
12. Automation can help scale security expertise Gareth Rushgrove
13. Automation is also the perfect middle-ground to build better relationships between security, development and operations Gareth Rushgrove
14. This talk
15. The importance of shared tools to collaboration Gareth Rushgrove
16. Examples of applying infrastructure as code, and other software practices to security problems Gareth Rushgrove
17. What the security community can learn from the successes of Devops Gareth Rushgrove
18. WARNING This talk will feature code but is not about software development Gareth Rushgrove
19. Sharing tools and practices Bringing developers, operations and security together
20. Why share? - Common language between teams - Builds trust amongst team members - Avoid duplication of effort and data - Limits the number of systems Gareth Rushgrove
21. Example: Infrastructure as Code Gareth Rushgrove
22. Gareth Rushgrove The architecture docs Your mental model Reality The developers mental model = = = =
23. Gareth Rushgrove The architecture docs Your mental model Reality The developers mental model = = = =
24. Gareth Rushgrove
25. Gareth Rushgrove Puppet code Reality =
26. An unambiguous description of your infrastructure, stored in version control Gareth Rushgrove
27. Puppet is a domain specific language, designed to describe your infrastructure configuration
28. Gareth Rushgrove From files up to higher level constructs like applications running in IIS
29. And from host level resources to remote resources like DNS records or AWS Autoscaling Groups
30. Machines check they match the model every half an hour, and automatically remediate and report any differences Gareth Rushgrove
31. Gareth Rushgrove You can see what is changing across your infrastructure, all from the Puppet Enterprise Console
32. Shared resource between operations and development Gareth Rushgrove
33. Shared resource between operations, development, security and anyone with a stake in the infrastructure Gareth Rushgrove
34. “The constant discussion between software and infrastructure teams really helps us proactively find issues before deploying to production.”
35. All of the configuration information ends up in PuppetDB. How might that be of interest to a security team? Gareth Rushgrove
36. (The framework for this is about 20 lines of code and could be written by any good developer in an hour) Gareth Rushgrove
37. (expect installed-on-all-clients? "auditd") A one line test to check auditd is installed everywhere
38. (expect running-on-all-clients? "selinux") Because people seem to like to disable SELinux
39. (expect (every? ubuntu? (facts "operatingsystem"))) A one line test to check all machines are using the permitted Operating System
40. (expect latest-on-all-clients? "openssh") A one line test to check we’re running the latest version of Open SSH
41. Hook the tests into your operations teams monitoring system and security can get alerts when tests fail Gareth Rushgrove
42. Combine existing operations data with developers willingness to write code and the security teams knowledge of potential threats Gareth Rushgrove
43. Embrace pipelines Security testing in continuous delivery pipelines
44. Gareth Rushgrove
45. “We went from all-hands-on- deck, war-room sort of deployments to non-events” Gareth Rushgrove Jez Miller, Heartland Payment Systems, Puppet Enterprise Customer
46. Gareth Rushgrove Regular releases reduce risk
47. As applicable to changes to web applications as infrastructure, end user devices, network hardware and more Gareth Rushgrove
48. Gareth Rushgrove Login to a computer Make change
49. Gareth Rushgrove Write the code Check syntax Check style Unit tests Acceptance tests Deploy
50. Gareth Rushgrove Write the code Check syntax Check style Unit tests Acceptance tests Deploy Deploy to staging Preflight checks
51. Gareth Rushgrove Write the code Check syntax Check style Unit tests Acceptance tests Deploy Security tests
52. Automate where possible, so manual effort is focused on high risk work Gareth Rushgrove
53. Gareth Rushgrove Acceptance tests Deploy Security tests Manual review by security Automated security tests Automated security tests Is this a high risk change?
54. What is a high risk change? - Change to firewall policy? - Change to authentication code? - Change to SSH configuration? - New user added? Gareth Rushgrove
55. The important thing is the discussion with colleagues in development and operations Gareth Rushgrove
56. Turn security practices into standard processes Gareth Rushgrove
57. Turn standard processes into automated steps Gareth Rushgrove
58. Unit tests for security concerns Applying development practices to security
59. Once you know about the pipeline, you can add steps related to security Gareth Rushgrove
60. Ensure security policy is implemented in practice Gareth Rushgrove
61. Catch security problems before they hit production Gareth Rushgrove
62. Replace occasionally accessed spreadsheets with constantly running code Gareth Rushgrove
63. Useful in different contexts: - Production monitoring system - Tests in development - Smoke tests for deployment Gareth Rushgrove
64. Test your network Because everyone likes an open port Gareth Rushgrove
65. Gareth Rushgrove
66. Discovered open port 22/tcp on 45.56.74.113 Completed Connect Scan at 07:09, 3.31s elapsed (12 total ports) Nmap scan report for www.puppetlabs.com (45.56.74.113) Host is up (0.082s latency). rDNS record for 45.56.74.113: li924-113.members.linode.com PORT STATE SERVICE 20/tcp filtered ftp-data 21/tcp filtered ftp 22/tcp open ssh 23/tcp filtered telnet 25/tcp filtered smtp 80/tcp open http 110/tcp filtered pop3 443/tcp open https 512/tcp filtered exec 522/tcp filtered ulp 1080/tcp filtered socks 8080/tcp open http-proxy Standard nmap output requires manual analysis
67. Gareth Rushgrove Security tests Run network security tests
68. it 'has only a limited number of open ports' do expect(@open_ports.count).to eq(3) end it 'exposes a web server' do expect(@open_ports).to include('80/tcp') expect(@open_ports).to include('443/tcp') end it 'exposes an SSH server' do expect(@open_ports).to include('22/tcp') end it 'rejects email traffic' do expect(@closed_ports).to include('25/tcp') end Using a unit testing framework we can make explicit assertions
69. www.puppetlabs.com from 192.168.1.10 has only a limited number of open ports (FAILED - 3) exposes a web server exposes an SSH server rejects accept email traffic (FAILED - 4) Anyone can run the tests and understand what is expected and what is currently broken
70. Test your supply chain Checking packages for vulnerabilities Gareth Rushgrove
71. Old versions of third libraries might have known vulnerabilities Gareth Rushgrove
72. Gareth Rushgrove rubysec/bundler-audit
73. Gareth Rushgrove Security tests Run network security tests Check for vulnerable packages
74. source "https://rubygems.org" gem "bundler-audit" gem "rake" gem "pry" group :vulnerable do gem "actionpack", "=3.2.10" end An example Gemfile, a list of ruby packages used in an application or environment, with a vulnerable package
75. Name: actionpack Version: 3.2.10 Advisory: OSVDB-103440 Name: actionpack Version: 3.2.10 Advisory: OSVDB-89026 Criticality: High URL: http://osvdb.org/show/osvdb/89026 Description: Ruby on Rails contains a flaw in params_parser.rb of the Action Pack. The issue is triggered when a type casting error occurs during the parsing of parameters. This may allow a remote attacker to potentially execute arbitrary code. Solution: upgrade to ~> 2.3.15, ~> 3.0.19, ~> 3.1.10, >= 3.2.11 An example report from bunder-audit
76. Test your web applications Checking for XSS, SQLi and more Gareth Rushgrove
77. Gareth Rushgrove continuumsecurity/bdd-security
78. An example test for checking for SQL injection vulnerabilities Scenario: The application should not contain SQL injection vulnerabilities Meta: @id scan_sql_injection @cwe-89 Given a scanner with all policies disabled And the SQL-Injection policy is enabled And the attack strength is set to High And the alert threshold is set to Low When the scanner is run And the XML report is written to the file sql_injection.xml Then no Medium or higher risk vulnerabilities should be present
79. An example test for checking for XSS. Lots more build-in Scenario: The application should not contain XSS vulnerabilities Meta: @id scan_sql_injection @cwe-89 Given a scanner with all policies disabled And the Cross-Site-Scripting policy is enabled And the attack strength is set to High When the scanner is run Then no Medium or higher risk vulnerabilities should be present
80. Test what you deploy, not just what you write Gareth Rushgrove
81. Gareth Rushgrove Security tests Run network security tests Check for vulnerable packages Run web application security tests
82. Gareth Rushgrove Security tests Run network security tests Check for vulnerable packages Run web application security tests Static analysis Metrics analysis SSL config
83. These are just quick examples. The point is automated testing and security go together incredibly well Gareth Rushgrove
84. And embracing the pipeline means you can add small steps easily Gareth Rushgrove
85. Conclusions
86. Focus efforts on the mechanism for change, over individual changes Gareth Rushgrove
87. Focus on steps in the pipeline. No single tool will fix all problems Gareth Rushgrove
88. Favour tools (open source, in-house or commercial) with APIs that can be integrated into a pipeline Gareth Rushgrove
89. Developers have never been more interested in security, use that Gareth Rushgrove
90. Adding steps into a continuous delivery pipeline is the perfect opportunity for a conversation Gareth Rushgrove
91. Доверяй, но проверяй
92. Questions? And thanks for listening