Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Webhooks: Lessons (Un)learned

Webhooks: Lessons (Un)learned

Webhooks are a pillar of modern application development. They notify us of that new commit, an incoming text message, our email was delivered, and a payment was processed. Our systems can’t function without webhooks sending data seamlessly and securely across the internet.

But what happens if they’re not secure? What happens if your webhooks are intercepted, manipulated, or even replayed against your systems? What are the best ways - as both a provider and consumer - to protect our systems?

In this session, we’ll delve into the 100+ implementations we explored to build webhooks.fyi to identify the best and worst patterns to protect our systems now and in the future.

Keith Casey

July 25, 2023
Tweet

More Decks by Keith Casey

Other Decks in Technology

Transcript

  1. © ngrok. All rights reserved. Confidential Information of ngrok •

    Get out your phone • Open Postman, Insomnia, or curl (for bonus points) • Tweet at @ngrokHQ if I say something useful Requests for you
  2. © ngrok. All rights reserved. Confidential Information of ngrok Agenda

    01 Intro 02 Webhooks 101 03 The Fatal Flaw 04 Our Risks 05 What should Providers do? 06 What should Consumers do? 07 Closing / Q&A
  3. © ngrok. All rights reserved. Confidential Information of ngrok Who

    am I? https://www.youtube.com/@geekamongthetrees
  4. © ngrok. All rights reserved. Confidential Information of ngrok Why

    are you talking about this? Ref: https://webhooks.fyi/
  5. © ngrok. All rights reserved. Confidential Information of ngrok Webhooks:

    In the Beginning Ref: https://progrium.github.io/blog/2007/05/03/web-hooks-to-revolutionize-the-web/
  6. © ngrok. All rights reserved. Confidential Information of ngrok Sidebar

    on Jeff Lindsay • Coined the concept of “web hooks” • Early employee at Twilio • Created localtunnel (inspiration for ngrok) • Architect at dotCloud (which created Docker)
  7. © ngrok. All rights reserved. Confidential Information of ngrok •

    Simple protocol: HTTP • Simple payload: JSON or XML • Tech stack agnostic • Share state between systems • Building everything yourself sucks • Super easy to spoof & compromise 😈 • Er.. I mean super easy to test and mock But why webhooks?
  8. © ngrok. All rights reserved. Confidential Information of ngrok ©

    ngrok. All rights reserved. Confidential Information of ngrok Let’s play with it
  9. © ngrok. All rights reserved. Confidential Information of ngrok ©

    ngrok. All rights reserved. Confidential Information of ngrok (go to demo)
  10. © ngrok. All rights reserved. Confidential Information of ngrok ©

    ngrok. All rights reserved. Confidential Information of ngrok (606) 531-1770
  11. © ngrok. All rights reserved. Confidential Information of ngrok •

    Interception • Impersonation • Modification/Manipulation • Replay attacks • Downtime (missing requests) • Forward Compatibility • Multicast Destinations Risks & Considerations
  12. © ngrok. All rights reserved. Confidential Information of ngrok •

    Interception • Impersonation • Modification/Manipulation • Replay attacks • Downtime (missing requests) • Forward Compatibility • Multicast Destinations Risks & Considerations
  13. © ngrok. All rights reserved. Confidential Information of ngrok ©

    ngrok. All rights reserved. Confidential Information of ngrok Let’s play with it abuse it
  14. © ngrok. All rights reserved. Confidential Information of ngrok ©

    ngrok. All rights reserved. Confidential Information of ngrok (back to demo)
  15. © ngrok. All rights reserved. Confidential Information of ngrok ©

    ngrok. All rights reserved. Confidential Information of ngrok (606) 531-1770
  16. © ngrok. All rights reserved. Confidential Information of ngrok ©

    ngrok. All rights reserved. Confidential Information of ngrok Use Postman to POST to https://thatrocks.ngrok.io
  17. © ngrok. All rights reserved. Confidential Information of ngrok ©

    ngrok. All rights reserved. Confidential Information of ngrok https
  18. © ngrok. All rights reserved. Confidential Information of ngrok •

    Show the end to end process ◦ Including parameters, urls, and options • Show a payload ◦ Yes, all of them • Demonstrate verification • Embed verification in your libraries Docs
  19. © ngrok. All rights reserved. Confidential Information of ngrok •

    Mitigates replay ◦ Requires a (somewhat) sync’d clock ◦ Choose one format (unix timestamp ftw) ◦ Include it in the signature Use the timestamp (or request id) Ref: https://webhooks.fyi/security/replay-prevention
  20. © ngrok. All rights reserved. Confidential Information of ngrok Multi-version

    support Ref: https://webhooks.fyi/ops-experience/versioning • X-PagerDuty-Signature: v1=f03de6f61df6e454f3620c4d6aca17ad072d3f8bbb2760eac3b2ad 391b5e8073, v2=130dcacb53a94d983a37cf2acba98e805a1c37185309ba56fdcccb cf00d6dd8b • X-HubSpot-Signature-v2: <signed value> X-HubSpot-Signature-v3: <signed value>
  21. © ngrok. All rights reserved. Confidential Information of ngrok •

    X-PagerDuty-Signature: v1=f03de6f61df6e454f3620c4d6aca17ad072d3f8bbb2760eac3b2ad 391b5e8073, v1=130dcacb53a94d983a37cf2acba98e805a1c37185309ba56fdcccb cf00d6dd8b Zero Downtime (key rotation) Ref: https://webhooks.fyi/ops-experience/versioning
  22. © ngrok. All rights reserved. Confidential Information of ngrok ©

    ngrok. All rights reserved. Confidential Information of ngrok https
  23. © ngrok. All rights reserved. Confidential Information of ngrok ©

    ngrok. All rights reserved. Confidential Information of ngrok (back to demo) * with verification
  24. © ngrok. All rights reserved. Confidential Information of ngrok ©

    ngrok. All rights reserved. Confidential Information of ngrok (606) 531-1770
  25. © ngrok. All rights reserved. Confidential Information of ngrok ©

    ngrok. All rights reserved. Confidential Information of ngrok Use Postman to POST to https://tekrocks.ngrok.io
  26. © ngrok. All rights reserved. Confidential Information of ngrok //todo

    for providers • https or death • Doc all of it - patterns, payloads, & code • Strong authN methods • Mitigate replay attacks (timestamps) • Bonus points: Testing mechanism • Bonus points: Multi-version support Ref: https://webhooks.fyi/best-practices/webhook-providers
  27. © ngrok. All rights reserved. Confidential Information of ngrok •

    Transact over https • Verify senders • Validate payloads • Check (on critical events) • Bonus points: IP restrictions //todo for consumers Ref: https://webhooks.fyi/best-practices/webhook-consumers
  28. © ngrok. All rights reserved. Confidential Information of ngrok ©

    ngrok. All rights reserved. Confidential Information of ngrok About those bonus points for curl If you show me that you used curl, let me know today and I’ll give you a free year of ngrok Pro.