Secure Applications, by Design (with presenter notes)

(This version of the deck includes presenter notes.)
There is a lot of good security advice in the world, but checklists like the OWASP Top 10 do not tell you how to design security into your application. Where should a developer even begin? You'll leave this session with a process for building security in depth into your application architecture, using a human-centered user experience design, threat modeling, partitioning, defense in depth, and static analysis in continuous integration. Not yet another checklist, you'll learn how to make security the foundation on which the rest of your application is built.

Transcript

1. S E C U R E A P P L I C A T I O N S , B Y D E S I G N Craig Stuntz ∈ Improving https://speakerdeck.com/craigstuntz (click) slides I have a whole hour of content; there will not be 10 minutes of dead air at the end, so Please do interrupt with questions I would be more than happy to talk after the session if you would like to chat about anything possibly not of general interest to the whole audience.
2. P R E V I E W • What does application security mean? • Some “f ixes” which don’t work • Security f rom f irst principles • Threat modeling • Application design guided by principles and threat model “OWASP training” Important: Building secure systems is hard. Checklists of best practices we can follow do not necessarily result in a secure system. Is there a methodology we can follow to do it right? I will lean heavily on Information Security Practice Principles, Indiana University’s Center for Applied Cybersecurity Research
3. – H i p p o c r a t i c O a t h ( 1 9 6 4 L o u i s L a s a g n a v e r s i o n ) “I will remember that I do not treat a fever chart, a cancerous growth, but a sick human being, whose illness may affect the person’s family and economic stability.” Let’s start at the beginning. How do you enter the medical profession? One of the oldest binding documents in human history. Many doctors start their careers with the Hippocratic Oath
4. How to you get started in construction? How to software developers start our careers? (click)
5. 1. ummm… blockchain? 2. ??? 3. prof it! hm, I like the other ones better.
6. http://www.independent.co.uk/travel/news-and-advice/air-safety-2017-best-year-safest-airline-passengers-worldwide-to70-civil-aviation-review- a8130796.html Exactly zero passengers were killed in air transportation in 2017. However, experts caution it’s unlikely this low level can be maintained because passenger laptops might explode. You can see where I’m going with this, right? Imagine saying to a 1968 person that the biggest risk to 2018 airplane passengers might be the computing industry's reckless disregard for human safety! A question for you:
7. W O U L D Y O U D E S I G N S O F T W A R E D I F F E R E N T LY I F H U M A N S A F E T Y W A S A LW A Y S T H E F I R S T C O N S I D E R A T I O N ? H O W ? https://www.flickr.com/photos/wocintechchat/25900776992/
8. – A C M C o d e o f E t h i c s a n d P r o f e s s i o n a l C o n d u c t ( p r o p o s e d ) “A computing professional should contribute to society and to human well-being, acknowledging that all people are stakeholders in computing.” The very first principle on which the ACM’s proposed updated Code of Ethics is based on protecting people, protecting human rights, and avoiding harm.
9. – A l l i s o n M i l l e r “I don't think humans are the problem, the problem is that humans are the target.” https://www.scmagazineuk.com/news-feature-google-security-interview-human-solutions--the-way-to-go/article/701976/ It’s easy to look at, say, phishing statistics and complain about human behavior. Computing today is in a place similar to where we were after the start of the industrial revolution. We’re enjoying the productivity, but starting to notice the smog and the factory accidents.
10. W H A T I S S E C U R I T Y , R E A L LY ? https://commons.wikimedia.org/wiki/File:Airport_Frankfurt_-_Fraport_-_Flughafen_Frankfurt_-_barbed_wire_and_fence_-_Stacheldraht_und_Zaun_-_05.jpg https://www.flickr.com/photos/captkodak/37054929956/ We use the word to mean different things. Let’s consider what we really mean in the context of software.
11. D O M A I N S P E C I F I C Q A Security and QA professionals have a lot in common.
12. Behavior Specification Developers, QAs, security analysts, and users are all interested in the actual behavior of software (click) Some of this will be described by a specification, formal or, more commonly, informal QAs will investigate whether the behavior (click) correctly conforms to the spec or (click) fails They usually also expand the specification as they work (click) Security analysts, on the other hand, are most interested in the areas where the behavior falls outside of the specification, but also where the specification is self- contradictory.
13. QA! Security! Another thing QA and security have in common is that they’re not something you can just shake onto some crappy app at the end of the development process and magically make everything all better. QA and security professionals will want to help you with the design of your application to push potential problems much earlier in the development process.
14. Q A : D O E S T H E S O F T W A R E D O W H A T I T S H O U L D ? To recap:
15. S E C U R I T Y : D O E S I T A L S O D O A N Y T H I N G E L S E ?
16. D o We E v e n K n o w W h a t t h e S o f t w a r e I s S u p p o s e d t o D o ? Everyone
17. M y t h s
18. “Security is good guys vs. bad guys.” https://pixabay.com/en/quietscheenten-devil-contrast-2816024/ A bank harms their customers if they leave your deposits out on the street just as much as if they get robbed. You’re as likely to harm end user security via incompetence as via direct attack.
19. “You must always choose between security and convenience.”
20. – B r u c e S c h n e i e r “The attacker just has to f ind one vulnerability — one unsecured avenue for attack — and gets to choose how and when to attack. It’s simply not a fair battle.” http://nymag.com/selectall/2017/01/the-internet-of-things-dangerous-future-bruce-schneier.html Straw man? Nope. Truism #1: “On the internet, attack is easier than defense” It’s not wrong, but it's also not so right as to deserve the moniker “truism." A complex system *can* (often!) have a large attack surface. Real-world exploits against well-designed systems rarely rely on a single vulnerability. Defenders have home turf advantage. As far as we know, right now, most sites don’t get popped.
21. “In order to write secure applications, developers must take OWASP Top 10 training.” It’s really common to find this written into software development contracts. Top 10 is a collection of mistakes you should avoid making See also proactive top 10.
22. “Nobody cares about my application’s data. It’s public anyway.” Your social networking startup might be 100% public, but that won’t stop governments from weaponizing it. Your end users are still people, so they’re still targets.
23. “In order to write secure applications, developers must • Take OWASP Top 10 training • Use Veracode • Have application pentested • Use two factor authentication on source control and hosts • Use off-the-shelf crypto libraries • Monitor production • Use memory-safe languages • Do code review • HTTPS everywhere! • Patch your inf rastructure This one is even more dangerous, because no item on this list is a bad idea.
24. T r u t h https://www.flickr.com/photos/library_of_congress/8470007173/
25. – L e s l e y C a r h a r t “Regularly rethink your threat model. Know your threat model and that of your family before making any security decision.” https://twitter.com/hacks4pancakes/status/917952052667604993 Security decisions made without context of an articulated threat model are speculation and folklore. I will elaborate on this.
26. – M a t t Ta i t “The underlying problem is folks think in terms of ‘secure’ versus ‘insecure.’ But in reality, it's ‘in/secure vs. X threat in Y threat model.’” https://twitter.com/pwnallthethings/status/922009773352120320 You can’t describe a system as “secure” or “insecure.” You can’t use the word “security” to justify a decision. Every bad password policy follows from this myth.
27. – J e s s i c a P a y n e “Bugs and exploits are not the main issue in most breeches, operational issues and technical debt are.” "Your attacker thinks like my attacker: A common threat model to create better defense"
28. “ Yo u r i m a g i n a t i o n i s f a r m o r e w o n d e r f u l t h a n a n y c o m p u t e r c o u l d e v e r b e . ” - Fred Rogers http://www.neighborhoodarchive.com/mrn/episodes/1746/index.html
29. B U I L D A R E C I P E , N O T A G R O C E R Y S T O R E Don’t blindly pull every mitigation imaginable. Assemble a suite of components which build your design.
30. B Y D E S I G N https://www.patternlanguage.com/gallery/houses.html
31. H U M A N C E N T E R E D https://www.flickr.com/photos/wocintechchat/25926671551/ What does this mean, exactly?
32. L E A R N Y O U R D O M A I N https://commons.wikimedia.org/wiki/File:Domain,_Atrium_(Hong_Kong).jpg In most domains there are non-obvious risks to human safety as well as areas in which traditional “security rules” fall flat on their face. HR example
33. https://twitter.com/slatestarcodex/status/944739157988974592 Is a spell checker security sensitive software? It is if it automatically and silently changes your prescription
34. https://www.pbs.org/newshour/science/amazon-recalls-potentially-hazardous-solar-eclipse-glasses Let’s consider some fails.
35. – S e n . R i c h a r d B u r r “You commented yesterday that your company’s goal is bringing people together. In this case, people were brought together to foment conflict, and Facebook enabled that event to happen.” https://www.texastribune.org/2017/11/01/russian-facebook-page-organized-protest-texas-different-russian-page-l/
36. iT u n e s M o n e y L a u n d e r i n g https://www.thedailybeast.com/want-to-launder-bitcoins-how-crooks-are-hacking-itunes-and-getting-paid-by-apple It wasn’t designed as a money laundering system, it was designed as a terrible music player.
37. “ I ’ m j u s t a t o a s t e r . N o b o d y w i l l e v e r t r y t o h a c k m e ! ”
38. T H R E A T M O D E L I N G Can we do better? In order to make good security decisions, we need an accurate threat model. The basic questions your threat model must answer are Who is affected by your software? What do you have? What could go wrong? Let’s expand on all of these.
39. S I X D E G R E E S Who is affected by the software you create? https://www.flickr.com/photos/wocintechchat/25388897014/ This includes a lot of people. Not just people who click on your site! Ask yourself: Who are your…
40. U s e r s https://www.flickr.com/photos/wocintechchat/25703122741/
41. C u s t o m e r s https://www.flickr.com/photos/wocintechchat/25703122741/ https://www.flickr.com/photos/wocintechchat/25926791491/ We all know that users and customers are not the same thing, right? Right?
42. Yo u r Te a m https://www.flickr.com/photos/wocintechchat/25167741264/
43. S t a k e h o l d e r s https://www.flickr.com/photos/wocintechchat/25388889234/
44. P a r t n e r s https://www.flickr.com/photos/wocintechchat/25388854424/
45. Yo u r C o m m u n i t y Exactly nobody who builds threat models for a living does this, but you have to make a list!
46. W H A T D O Y O U H A V E ? Second question: What are you building?
47. I n f r a s t r u c t u r e • Servers • Software • Clients • Gateways • Third Parties
48. D a t a • Databases • Metadata • Logs • Credentials • Files on client machines This also includes data on client machines
49. T r u s t B o u n d a r i e s • Implicit • Explicit Show “who” (or person + role/escalation) controls what.
50. W H A T C O U L D G O W R O N G ? Third question: Who might attack it? What else could go wrong?
51. D O M A I N - S P E C I F I C R I S K S
52. T a k e C a r e o f P e o p l e F i r s t https://www.flickr.com/photos/wocintechchat/25926827581/
53. L e a r n f r o m H i s t o r y https://commons.wikimedia.org/wiki/File:Maginot_line_1.jpg Talk to business analysts or red team members about past experience.
54. E x i s t e n t i a l T h r e a t s http://money.cnn.com/2012/08/09/technology/knight-expensive-computer-bug/index.html In 2012, a software bug caused Knight Capital’s high speed trading system to issue millions of bad trades over 45 minutes, costing the company $440M and effectively ending their business.
55. R e g u l a t o r y GDPR (General Data Protection Regulation) vs. event sourcing, e.g.
56. B A C K T O B A S I C S Information Security Practice Principles, Indiana University’s Center for Applied Cybersecurity Research Designed to help secure technologies and systems where “best practices” haven’t emerged.
57. C O M P R E H E N S I V I T Y Security f rom First Principles Am I covering all of my bases? Craig Jackson, Scott Russell, and Susan Sons https://upload.wikimedia.org/wikipedia/commons/7/72/Agoncillo_- _W%C3%BCrth_Rioja%2C_Museo_30_-_Christo.JPG Identify and account for all relevant systems, actors, and risks in the environment. Related Concepts: Complete Mediation, End-to-end Encryption, Reconnaissance, Inventory, Threat Modeling
58. O P P O R T U N I T Y Security f rom First Principles Am I taking advantage of my environment? https://commons.wikimedia.org/wiki/File:Amazing_Bhutan_Monastery.jpg Craig Jackson, Scott Russell, and Susan Sons 2: Take advantage of the actor relationships, material resources, and strategic opportunities available in the environment. Related Concepts: Information Sharing, White Hat Testing, Deception, Common Tools
59. R I G O R Security f rom First Principles What is correct behavior, and how am I ensuring it? https://commons.wikimedia.org/wiki/File:Turnstile_state_machine_colored.svg Craig Jackson, Scott Russell, and Susan Sons 3: Specify and enforce the expected states, behaviors, and processes governing the relevant systems and actors Related Concepts: Governance, Requirements, Monitoring, Audits Langsec. Checklists? Maybe Rigor has two phases: specification and enforcement.
60. M I N I M I Z A T I O N Security f rom First Principles Can this be a smaller target? Craig Jackson, Scott Russell, and Susan Sons 4: Minimize the size, quantity, and complexity of what is to be protected, and limit externally facing points of attack. Related Concepts: Attack Surface, Compactness, Data Minimization, Simplicity Minimize time, as well. Delete data. Remove unused code. Again, Langesec
61. C O M P A R T M E N T A L I Z A T I O N Security f rom First Principles Is this made of distinct parts with limited interactions? https://en.wikipedia.org/wiki/Bulkhead_(partition)#/media/ File:Compartments_and_watertight_subdivision_of_a_ship%27s_hull_(Seaman%27s_Pocket- Book,_1943).jpg Craig Jackson, Scott Russell, and Susan Sons 5: Totally isolate system elements, then enable and control the interactions essential for their intended purpose. Often thought of in terms of network partitioning, but network partitioning is leaky (phishing). Can compartmentalize on identity, or elevation. Related Concepts: Modularity, Forward Secrecy, Least Privilege, Air Gapping, Cryptography
62. F A U LT T O L E R A N C E Security f rom First Principles What happens if this fails? https://commons.wikimedia.org/wiki/ File:A_U.S._Soldier,_right,_looks_on_as_a_U.S._Army_Garrison_Ansbach_Junior_ROTC_cadet_negotia tes_a_high_rope_obstacle_6.jpg Craig Jackson, Scott Russell, and Susan Sons 6: Anticipate and address the potential compromise and failure of system elements and security controls. Related Concepts: Resilience, Failsafe Defaults, Defense in Depth, Revocability, Incident Response, Business Continuity and Disaster Recovery, Murphy’s Law
63. P R O P O R T I O N A L I T Y Security f rom First Principles Is this worth it? https://twitter.com/jwgoerlich/status/939268098699550720?s=09 Craig Jackson, Scott Russell, and Susan Sons Tailor security strategies to the degree of risk, accounting for the practical constraints imposed by the use case and the environment. Related Concepts: Risk Management and Acceptance, Usability
64. T H E B A S I C P R I N C I P L E S I N A C T I O N OK, these are all very interesting considerations, but how do you apply them to software architecture? How can I decide when I’m “done?”
65. B U S I N E S S P R O B L E M • A hotel chain needs to capture credit card numbers for potential incidental charges when the cardholder will not be present at check in • Example: A parent wants to authorize incidental charges for a traveling school sports team member • Current process is a paper form. Company would like to automate I think it’s obvious that there are some risks here, but let’s see if we can engineer a good solution.
66. N A Ï V E S O L U T I O N “Type a quote here.” Where do you even begin on a problem like this? There’s no shame in starting with a naïve solution, as long as you don’t end there. We’re using HTTPS because we care about security. You might see a lot of potential problems here, but you’re all very smart people. Is there a method for finding the issues?
67. N A Ï V E S O L U T I O N , R E V I S I T E D Comprehensivity “Type a quote here.” Now let’s apply the first of the seven principles, comprehensivity. Our “naïve solution is a bit oversimplified, isn’t it? Even at a purely technical level, we’ve omitted quite a bit from our original diagram.
68. N A Ï V E S O L U T I O N , R E - R E V I S I T E D Comprehensivity “Type a quote here.” Sticking with comprehensivity, let’s talk with a domain expert. You might presume the biggest risk is theft of CC#s, but they’re most concerned about site used to validate stolen CC#s. (click) We’ll add that to the diagram. Domain expert also points out there are actors missing from the threat model. (click) Not clear how they fit in just yet, but we’ll at least include them while we’re getting started.
69. N A Ï V E S O L U T I O N , R E - R E - R E V I S I T E D Comprehensivity “Type a quote here.” Important to remember that the Comprehensivity principle requires not just identifying actors, risks, etc., but also accounting for them. Threat Modeling Tool helps out with this via its analysis view. We’ll return to this later. We’re not going to account for all the threats right now because we’re not required to complete the principles in order, and we want to consider all of them before accounting for threats. We’re not done until we account for all risks. In general, you should expect to consider all seven principles twice. But we’re already off to a pretty good start!
70. D E S I G N E D I N T O P R O C E S S Comprehensivity https://jeremylong.github.io/DependencyCheck/ Let’s talk about CI. It’s more important to have a practice of security checks in CI than any specific list of checks. You may not depend on any vulnerable components today. You could build again tomorrow and have a vulnerable dependency transitively included. The hotel also has some existing security infrastructure including continuous monitoring, which we’ll consider in a moment, but I bring it up now to emphasize the point that the principle of comprehensivity applies to time as well as space and we should consider the future interactions with the system we build.
71. T R A I N I N G Comprehensivity https://twitter.com/chrisrohlf/status/925846092184477698 Pick the right training for your threat model and your team. We tend to think of training in terms of certifications and OWASP top 10. But maybe the most important topic for training is your business-specific assumptions and threat model. Training takes time; make sure you’re training on the right things.
72. O P P O R T U N I T Y For the opportunity principle, there are three factors to consider: Actor relationships: (click) The hotel’s existing infrastructure and AWS. (click) Tokenization service. Also consider information sharing with others in the same line of business. There are organizations for this. Material resources: (click) Follow AWS security best practices white paper. (click) Use managed services instead of maintaining bare metal. This is the area for checklists like OWASP proactive top 10 Strategic opportunities: Work with business users to design a process which doesn’t require creating accounts and passwords.
73. P A T C H A L L O F T H E T H I N G S Opportunity “Type a quote here.” There’s obviously a lot more we could do; I’m limited in the amount of ideas I can present at the end of a talk.
74. R I G O R We’ll depart from the threat model for a second. The threat model is in service of the principles. You don’t use the principles just to build up a threat model. Two steps: Specification and enforcement. You can specify with varying degrees of formality. A sequence diagram like you see here is more formal than a plain English description and less formal than a modeling language like TLA+. Use a model checker to require self-consistency of a specification. Enforcement of a spec: One good place to start is your language’s type system. Make incorrect state impossible to construct.
75. S T A T I C A N A LY S I S Rigor “The most important thing I have done as a programmer in recent years is to aggressively pursue static code analysis. Even more valuable than the hundreds of serious bugs I have prevented with it is the change in mindset about the way I view software reliability and code quality.” - J o h n C a r m a c k https://www.gamasutra.com/view/news/128836/InDepth_Static_Code_Analysis.php Another way to enforce rigor. Practically free!
76. We’ve gone from this “naïve” implementation of our payment system to this. Awesome, look at all those security features! More is better, right?
77. M I N I M I Z E A T T A C K S U R F A C E ( a n d e v e r y t h i n g e l s e ) https://www.owasp.org/index.php/Attack_Surface_Analysis_Cheat_Sheet The principal of minimization applies in many ways to our project: Minimize the attack surface by putting servers behind firewalls, block ssh into private network except via jump server from known IPs. Minimize tools. AWS can make this hard, feels more like a lego set sometimes. Remove user interfaces. Delete code. Minimize everything.
78. S T O R E L E S S Minimization “Limit cardholder data storage and retention time to that which is required for business, legal, and/ or regulatory purposes, as documented in your data retention policy. Purge unnecessary stored data at least quarterly.” P C I - D S S § 3 . 1 https://www.pcisecuritystandards.org/documents/PCIDSS_QRGv3_1.pdf Don’t make yourself a Target. So to speak. This is not just a good idea; it may be a compliance requirement. Minimize data collection, don’t store PII where you can avoid it. Delete old data (default of DBs is to never delete data) (except MongoDB).
79. C O M P A R T M E N T A L I Z E I T ! Three steps: Totally isolate each element on system Enable specific interactions needed Control those enabled interactions Great place to start is any place on diagram where data flows cross trust boundaries
80. D O U B L E E D G E D S W O R D Compartmentalization ““Your perimeter is not the boundary of your network it’s the boundary of your telemetry.” http://grugq.github.io/presentations/comae-blackhat-year-of-the-worm.pdf - T h e G r u g q A misconception about compartmentalization: There is a “safe” side of a perimeter. Another misconception: Compartmentalization is only about network perimeters
81. L E A S T P R I V I L E G E Compartmentalization EncryptionServiceIAMRole: Type: "AWS::IAM::Role" Properties: Path: "/" ManagedPolicyArns: - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole" AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Sid: "AllowLambdaServiceToAssumeRole" Effect: "Allow" Action: - "sts:AssumeRole" Principal: Service: - "lambda.amazonaws.com" This might at first seem to conflict with the principle of human-centered design.
82. C O M P A R T M E N T A L I Z E I T ! • Networks • Public ingress (CloudFront), WAF rules • Private ingress (Jump server) • Roles for public, hotel staff, site admin, developer, ops • Restrict data by property • Archive old data to encrypted cold storage • Use key management (KMS, HSM, etc.) for secrets So here are some decisions we might make. Again, there are many more we might arrive at if we drill down deeply
83. F A U LT T O L E R A N C E https://github.com/Xyl2k/TSA-Travel-Sentry-master-keys You must consider fault tolerance for every point of the system. I’m going to give only one example here: Compromised credentials. But you should do an error analysis for every system and every entry point, at the very least. I’m choosing an example of compromise because we often think of faults in terms of technical faults, rather than information leakage faults. Both are important.
84. F A U LT T O L E R A N C E • User safety • Stop the exf iltration • Assess the scope • Proactively prevent further damage to users • Listen • Technical • Engage DF/IR professionals to assess how it happened and how to prevent • Design system for secure storage and rotation of secrets Typical response: Free credit monitoring!
85. P R O P O R T I O N A L I T Y Having gotten this far, we have a really extensive list of risks. We want to make sure we spend time on the appropriate issues. MS Threat Modeling Tool has features to help with this. The highest priority issues are those which directly or indirectly affect human safety. Security should not exist solely for the sake of security. Security is always in furtherance of some greater organizational goals, and it should be crafted with those goals in mind. Then consider the status.
86. L A T H E R , R I N S E , R E P E A T • Plan on enumerating the f irst principles at least twice in initial app design • Following f irst principles does not mean “big design upf ront” * Because principles interact, you can’t go through them just once in an application design. The bare minimum is twice, but plan on more. * Does this approach work in an agile environment? Sure. Possibly even better if it encourages you to keep drawing water from the well.
87. F U R T H E R R E A D I N G • The Information Security Practice Principles, Center for Applied Cybersecurity Research, Indiana University • Threat Modeling, Designing for Security, by Adam Shostack
88. C R E D I T S • Some stock photography f rom wocintechchat.com, CC- BY 2.0 • Creative Commons photography credited on each slide
89. The conference organizers have asked me to ask you to please rate the session. Just open up AttendeeHub. If it crashes, open it again Click the “Session Survey” link highlighted here.
90. C O N T A C T craig.stuntz@improving.com @craigstuntz http://paperswelove.org/chapter/columbus/ https://speakerdeck.com/craigstuntz