Upgrade to Pro — share decks privately, control downloads, hide ads and more …

HTTP in a Hostile World (Droidcon Toronto 2019)

Jesse Wilson
November 13, 2019

HTTP in a Hostile World (Droidcon Toronto 2019)

Video: https://youtu.be/tPA9n2mgClI

Networking on Android is difficult work. We need to display fresh data and rich media while limiting our network use. Unfortunately cookie-cutter solutions are not acceptable.

In this talk we’ll:

🥌 Verify that you’re using HTTPS securely
🥌 Acknowledge possible connectivity problems
🥌 Get performance tips
🥌 Code for maintainable APIs
🥌 Mock programmers that don’t test their networking

The body of this talk will be using HTTP effectively.

Jesse Wilson

November 13, 2019
Tweet

More Decks by Jesse Wilson

Other Decks in Programming

Transcript

  1. HTTP? • Web Pages • Web Resources (images!) • Downloads

    • Web APIs • Mobile APIs • Server APIs • Core Networking APIs (DNS) • Pretty much all networking ever
  2. POST /pizza HTTP/1.1 host: pizza.app content-length: 50 content-type: application/json; charset=UTF-8

    {x "size": "large", "toppings": ["pineapple"] }x HTTP/1.1 200 OK content-length: 21 
 content-type: application/json; charset=UTF-8 {x "time": “30m", "price": 15.00 }x Response
  3. HTTP/1.1 200 OK content-length: 21 
 content-type: application/json; charset=UTF-8 {x

    "time": “30m", "price": 15.00 }x POST /pizza HTTP/1.1 host: pizza.app content-length: 50 content-type: application/json; charset=UTF-8 {x "size": "large", "toppings": ["pineapple"] }x Status
  4. HTTP/1.1 200 OK content-length: 21 
 content-type: application/json; charset=UTF-8 {x

    "time": “30m", "price": 15.00 }x POST /pizza HTTP/1.1 host: pizza.app content-length: 50 content-type: application/json; charset=UTF-8 {x "size": "large", "toppings": ["pineapple"] }x Headers
  5. HTTP/1.1 200 OK content-length: 21 
 content-type: application/json; charset=UTF-8 {x

    "time": “30m", "price": 15.00 }x POST /pizza HTTP/1.1 host: pizza.app content-length: 50 content-type: application/json; charset=UTF-8 {x "size": "large", "toppings": ["pineapple"] }x Body
  6. HTTP/1.1 200 OK content-length: 21 
 content-type: application/json; charset=UTF-8 {x

    "time": “30m", "price": 15.00 }x POST /pizza HTTP/1.1 host: pizza.app content-length: 50 content-type: application/json; charset=UTF-8 {x "size": "large", "toppings": ["pineapple"] }x
  7. $ nc square.com 80 GET /robots.txt HTTP/1.1 host: square.com HTTP/1.1

    301 Moved Permanently location: https://square.com/robots.txt date: Thu, 07 Nov 2019 04:05:21 GMT server: envoy content-length: 0
 $
  8. HTTP/0.9 HTTP/1.0 HTTP/1.1 HTTP/2 HTTP/3 1991 1996 1997 2015 2019

    HTML only headers and status codes persistent connections multiplexing and binary encoding out-of-order packets and IP address changes
  9. HTTP in a Hostile World? • Correct • Ready •

    Secure • New stuff I’m excited about
  10. What Triggers Repeats? • Users double-tapping the send button •

    Applications retrying • Network code retrying • Gateways retrying
  11. Implementing Idempotence: Identify Intention • Generate a universally unique token

    in the UI:
 d6abe0fcd5d8c51e6c8a9159a873efcc • Write the outbound intention to a persistent queue, like your app’s SQLite database or your cloud queue service (SQS, etc.) • When processing an action, persist the identifier • The easy way: create a UNIQUE INDEX on the token
  12. API Contract v. 1 • Client sends a pizza order

    in JSON • Server sends HTTP 200 and the delivery info in JSON
  13. HTTP/1.1 200 OK { "time": "30m", "price": 15.00 } POST

    /pizza HTTP/1.1 { "size": "large", "toppings": ["pineapple"] }
  14. API Contract v. 2 • Client sends a pizza order

    in JSON • Success: Server sends HTTP 200 and the delivery info in JSON • Failure: Server sends HTTP 500 and the failure info in JSON
  15. Designing HTTP APIs • Conceptually, your client talks to your

    server • In practice, ‘invisible’ infrastructure exists • Sometimes it fails to be invisible
  16. Status Codes just a sec thumbs up! response is elsewhere

    client problem server problem 1xx 2xx 3xx 4xx 5xx
  17. Resilient to Middle Boxes • Send Content-Type on outbound requests

    • Expect Content-Type on inbound responses • HTTP 200? All Good • Other status codes? Be defensive!
  18. REST: Retrofit interface PizzaShop { @GET("/{store}/{date}/deliveries") fun deliveries( @Path("store") store:

    String, @Path("date") date: String ): Call<List<Delivery>> } square.github.io/retrofit
  19. interface PizzaShopClient : Service { @WireRpc( path = "/yum.pizza.PizzaShop/OrderFeed", requestAdapter

    = "yum.pizza.Update#ADAPTER”, responseAdapter = "yum.pizza.Order#ADAPTER" ) fun OrderFeed(): GrpcStreamingCall<Update, Order> } gRPC: Wire 3 square.github.io/wire
  20. Testing: MockWebServer github.com/square/okhttp/mockwebserver val server = MockWebServer() server.start() server.enqueue(MockResponse() .setHeader("Cookie",

    "fave_topping=pineapple") .setBody("""{"time": "30m", "price": 19.99 }""")) val serverUrl = server.url("/")
  21. Connectivity is a Lie • Bell and Rogers fail •

    Backends fail • Graceful fallbacks are possible
  22. Good-Enough Caching • It may change
 Cache-Control: max-age=0 • It

    will never change
 Cache-Control: max-age=31536000
  23. val cache = Cache( directory = File(context.cacheDir, "http"), maxSize =

    100L * 1024 * 1024 ) val client = OkHttpClient.Builder() .cache(cache) .build()
  24. Compression • Make sure gzip responses are enabled on client

    and server • Cuts size by 70% for JSON! • Consider gzip requests too • Use brotli for best compression
  25. CDNs • Content Delivery Networks • Easiest way to improve

    upon speed of light • Great for availability • These can do image resizing!
  26. Granularity • HTTP/1’s per-resource overhead is high • More speed?


    Fewer resources! • With HTTP/2 logical resources are best whatsapp.com
  27. Data is Dangerous • You’re keeping other people’s secrets •

    You don’t really know what’s secret and what’s not • Attackers are persistent
  28. Limit Exposure to Data • Are we collecting this field

    because we need it? • When will we delete it? • GDPR in Europe, CCPA in California • Differential Privacy
  29. TLS: Necessary • Transport Layer Security is the s in

    https:// • Confirms that you’re really talking to https://cash.app/ • Prevents eavesdropping • Prevents interference
  30. TLS: Not Sufficient • Doesn’t impact how https://cash.app/ secures the

    data • Doesn’t prevent URLs like https://cash.app.attacker.com/ • Old implementations of TLS had severe bugs • Nation-state attackers do nasty things
  31. Nation-State Attacks • A certificate authority is a company that

    confirms ownership of https://cash.app/ and signs our TLS certificate • By default browsers trust ~300 certificate authorities (CAs) • Nation states add themselves to the browser • Or compromise a vulnerable certificate authority
  32. CertificatePinner certificatePinner = new CertificatePinner.Builder() .add("pizza.yum", "sha256/9TxoNDzGwoGUV8oDSZI8XpGpwyPz0GY/xAyWRLLjpPc=") .add("pizza.yum", "sha256/36Bv98D5O9bYGYRXnMTZJLQRW8i8fvOiWC/TJ4jOhLw=") .add("pizza.yum",

    "sha256/++iJU8rCHLXviXyKrJ2yqBWjs/5+LAqWuP04dHVHFb0=") .build(); OkHttpClient okHttpClient = new OkHttpClient.Builder() .certificatePinner(certificatePinner) .build();
  33. TLS Evolution first widely-used release renamed from SSL to TLS

    bug fixes & new crypto bug fixes & new crypto major update with simplifications! 1996 1999 2006 2008 2018 SSLv3 TLSv1 TLSv1.1 TLSv1.2 TLSv1.3
  34. TLS Evolution first widely-used release renamed from SSL to TLS

    bug fixes & new crypto bug fixes & new crypto major update with simplifications! JAN 2020 1996 1999 2006 2008 2018 SSLv3 TLSv1 TLSv1.1 TLSv1.2 TLSv1.3
  35. TLS Evolution first widely-used release renamed from SSL to TLS

    bug fixes & new crypto bug fixes & new crypto major update with simplifications! JAN 2020 1996 1999 2006 2008 2018 Android 5+ Or Google Play Security Provider SSLv3 TLSv1 TLSv1.1 TLSv1.2 TLSv1.3
  36. Whitebox Attackers • Our apps are instruction manuals for reverse

    engineers • There are amazing tools that make this point-and-click easy
  37. Defending Against
 Whitebox Attackers • A sufficiently-determined attacker will succeed

    • All we can do is make it really annoying • Perhaps a magic HTTP header?
  38. HTTP/3 • Better worst-case latency • Change your IP address

    without restarting downloads! quicwg.org
  39. TCP streams Requests and Responses TCP packets HTTP/1.1 TCP streams

    TLS streams Requests and Responses TCP packets HTTP/1.1 with TLS
  40. TCP streams Requests and Responses TCP packets HTTP/1.1 TCP streams

    TLS streams Requests and Responses TCP packets HTTP/1.1 with TLS TCP streams TLS streams HTTP/2 frames TCP packets HTTP/2 with TLS Requests and Responses
  41. TCP streams Requests and Responses TCP packets HTTP/1.1 TCP streams

    TLS streams Requests and Responses TCP packets HTTP/1.1 with TLS TCP streams TLS streams HTTP/2 frames TCP packets HTTP/2 with TLS Requests and Responses UDP packets HTTP/3 with TLS Requests and Responses QUIC streams
  42. • What if Retrofit inspired a server framework? • gRPC

    + REST in one framework • Early days! Watch this space cashapp.github.io/misk