What I've Learned Writing CTF Challenges

BSides Tampa, 2/11/2017

1. What I’ve Learned Writing CTF Challenges Vito Genovese BSides Tampa February 11, 2017
2. What is CTF?
3. What is CTF?
4. Jeopardy Style
5. Attack-Defense
21. CGC Jargon • Cyber Reasoning System • Proof of Vulnerability • Replacement Binary • Challenge Binary
22. BSTies http://bsties.notmalware.ru
23. CTF Challenges Teams attack it
24. CTF Challenges Protects something of in-game value
25. CTF Challenges Ideally: known solvable
26. CTF Challenges Also ideally: appropriately tricky
29. Solving 'em • Correct side of an if statement • Cracking a code • Using SQL injection to trick a database • Stealing the garbage file from the Gibson
30. Actually Building One
32. Original Goal
33. Original Goal
34. • Given same STDIN • Same STDOUT
 • Same clock cycles • Same syscalls
35. https://xkcd.com/1319/
36. Why Automate?
38. Thousand Cuts
39. Thousand Cuts
40. Thousand Cuts
41. Actually Building It
42. Start the First One 1. Set up CGC service-template 2. Test not-crashing 3. Make binary that doesn’t crash 4. Test crashing 5. Make binary that crashes
43. First One's Done • Ruby script to spit out 334 binaries • Vary buffer sizes • Vary stack cookie • Python script running game flow
44. First Set's Done Figure out how to make second and third sets
45. Second Set Python script to add stuff to source code
46. Third Set Python script to rearrange source code
47. All Sets Done Test the difficulty
48. Integration Testing With Gyno • Finds a bug in like 30s • Fixing the bug took 30m • (fixing the bug a week prior would've taken 5m)
49. More Integration Testing kind of weird
50. easy-prasky • single binary • Baby’s First category • Separate CRS work from first stab at CGCEF • Hacked to be first in 334 cuts
51. Deployment • Determine ops requirements • CGC kernel • Python installed • Open port
52. Deployment Puppetize it!
53. Deployment
54. Running it just kinda worked (once teams Got It)
55. CTF Challenge Process 1.Idea 2.Build 3.Test
56. CTF Challenge Framework • Configuration • Build
 • Testing • Deployment
57. CTF Challenge Framework CGC service-template
58. CTF Challenge Deployment put it on a server lol
59. It's Just Software
60. Software Goals Useful Usable Reliable
61. Useful for Competition Separate teams that solve it from teams that don't Prefer strong correlation with finals ranking
62. Useful for Education Teach new players something Get players over a hang-up
63. Usable Intended difficulty only Minimize corner cases Unintended vulnerabilities?
64. Reliable Remain vulnerable
65. Reliability is Hard Players like to attack Players like to brute Players like to complain
66. Reliability is Hard
67. Reliability is Demanding
68. Reliability is Expensive If you want it to be
69. Reliability is Cheap Build software that can be run reliably
70. Twelve Factor App • VIII. Concurrency • Scale out via the process model • IX. Disposability • Maximize robustness with fast startup and graceful shutdown
71. Twelve Factor Challenges • Limit global state • Process per connection • xinetd ftw • Make adding resources cheap
72. Adding Resources Automate!
73. Operations Automation Puppet or Chef for server provisioning
74. Operations Automation Docker is slow to spin up RunC is fast
75. Shared State • Process per connection not always feasible • Sandwich challenges in 2014 • JRuby & Celluloid slow to start • State still per connection
76. Shared State • Global state and web challenges • Persisted XSS • Tricking a global database
77. Waiting For Your Touch • 2015 web challenge • /r/thebutton knockoff • JRuby, Rails, websockets, postgres
78. Waiting For Your Touch
79. Waiting For The Challenge Slow when not slow 500s Thankfully during the day
80. Waiting For Your Touch 1. Took it down 2. Opened replacement challenge 3. Debugged for an hour
81. Debugging For Your Touch 4. Leaked and exhausted postgres connections 5. Fixed leak 6. Allocated more connections
82. Waiting For My Fix 7. Took an hour 8. Gyno forced a fifteen minute cool-down between "is it fixed" and "it is fixed" 9. Reopened
83. Most Important Part People are the most important
84. Respect People • Respect for yourself • Respect for your team • Respect for players
85. Respect for Yourself
86. Respect for Your Team
87. Respect for Your Team
88. Respect for Your team Make services easy to keep running Document!
89. Respect for Your Team Provide easy to run smoke tests "Is this exploitable?" should be a one-liner that spits out the flag
90. Respect for Players
91. Respect for Players
92. Respect for Players
93. Respect for Players
94. Building Challenges is Software Development Useful Usable Reliable
95. Reliability Imposes Constraints Limit state Limit dependencies Automate
96. Respect Is Fundamental Self Team Players
97. Thanks! Vito Genovese vito@legitbs.net @vito_lbs
 GPG B07D616143CAA77B https://legitbs.net @legitbs_ctf