PyLondinium 2019: Gentle introduction to TLS, PKI, and Python's ssl module

Abstract

TLS/SSL is the most important and widely-used protocol for secure and encrypted communication. I'm going to introduce you to TLS 1.2 and 1.3, cryptographic building blocks, best-practice configuration, certificates, and public key infrastructure using Python's ssl module
Description

TLS is an ubiquitous protocol for secure communication. It's used in HTTPS, email (IMAP, POP3, SMTP), LDAP, FTP, and more. Some recent protocols like HTTP/2 are not defined for unencrypted channels. TLS offers more than just encryption with symmetric cryptography. It also ensures data integrity and strong authentication with the help of X.509 certificates and public key infrastructure (PKI).

Did you ever wonder what's the difference between SSL, TLS, and StartTLS? Or what is the meaning of cryptic terms and names like Server Name Indication, Subject Alternative Name, or TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384? What is perfect forward secrecy and why is it so important for privacy? Does HTTPS make my website slow? What is the difference between a root CA, intermediate CA and end-entity certificate? My talk explains how a client and a server establish a secure connection, why a certificate is required, and how TLS 1.3 has improved the handshake.

Some prior knowledge of cryptography and networking basics are helpful but not required to follow this talk.

Transcript

1. Gentle introduction to TLS, PKI, and Python's ssl module PyLondinium 2019 / London Christian Heimes Principal Software Engineer christian@python.org / cheimes@redhat.com @ChristianHeimes
2. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Who am I? • from Hamburg/Germany • Linux user since 1997 • Python and C developer • Fellow of the PSF • Python core developer since 2008 • maintainer of ssl and hashlib module • Python security team • Occasional contributor to OpenSSL
3. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Professional life • Principal Software Engineer at Red Hat • Security Engineering • FreeIPA Identity Management • Dogtag PKI
4. Agenda & Takeaways
5. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Agenda • high level view • ssl module • cryptography 101 • TLS handshake • certs & PKI • TLS 1.3 • books & resources
6. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Talk • Slides https://speakerdeck.com/tiran/ • Attribution-ShareAlike 4.0 International • Assumptions • Python 3.6 - 3.8 • OpenSSL 1.1.0 or 1.1.1 • TLS 1.2 and TLS 1.3 • modern cryptography, perfect forward secrecy (PFS)
7. Should I deploy TLS?
8. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Yes!
9. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Yes, you should!
10. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 TLS has exactly one performance problem: it is not used widely enough. Everything else can be optimized. https://istlsfastyet.com/
11. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Troy Hunt, I wanna go fast: HTTPS' massive speed advantage https://www.troyhunt.com/i-wanna-go-fast-https-massive-speed-advantage/
12. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Troy Hunt, I wanna go fast: HTTPS' massive speed advantage https://www.troyhunt.com/i-wanna-go-fast-https-massive-speed-advantage/
13. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0
14. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Reasons to deploy TLS • Privacy • Security (Ad / Malware injection • Performance (HTTP/2, HTTP/3) • SEO • User Experience (password warning) • Browers are deprecating HTTP support
15. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0
16. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Wikipedia definition Transport Layer Security (TLS) – and its predecessor, Secure Sockets Layer (SSL) – are cryptographic protocols that provide communications security over a computer network. The TLS protocol aims primarily to provide privacy and data integrity between two communicating computer applications.
17. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 TLS core features • encrypted transport stream • application protocol agnostic • integrity check • replay attack protection • strong authentication of server • strong authentication of client (optional) • extensible protocol
18. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 TLS standard • IETF standard (Internet engineering task force) • IANA (Internet assigned number authority) • TLS (TCP), DTLS (UDP), QUIC • ASN.1 • PKI with X.509 certificates
19. Python's ssl module
20. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 ssl module, an OpenSSL wrapper • ssl.SSLContext (SSL_CTX*) • configuration • certificates • trust anchors (root CA) • PROTOCOL_*, CERT_*, VERIFY_*, OP_*, TLSVersion.* • ... • ssl.SSLSocket (SSL*) • ssl.SSLObject • ssl.MemoryBIO (BIO*) • ssl.wrap_socket()
21. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Example import socket import ssl HOSTNAME = 'pylondinium.org' ctx = ssl.create_default_context() sock = socket.create_connection((HOSTNAME, 443)) ssock = ctx.wrap_socket(sock, server_hostname=HOSTNAME) import socket import ssl HOSTNAME = 'pylondinium.org' ctx = ssl.create_default_context() sock = socket.create_connection((HOSTNAME, 443)) ssock = ctx.wrap_socket(sock, server_hostname=HOSTNAME)
22. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Example (2) ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) ctx.minimum_version = ssl.TLSVersion.TLSv1_2 ctx.maximum_version = ssl.TLSVersion.TLSv1_2 # TLSv1_3 ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_default_certs() ctx.check_hostname = True ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) ctx.minimum_version = ssl.TLSVersion.TLSv1_2 ctx.maximum_version = ssl.TLSVersion.TLSv1_2 # TLSv1_3 ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_default_certs() ctx.check_hostname = True ssock = ctx.wrap_socket(sock, server_hostname=HOSTNAME) ssock = ctx.wrap_socket(sock, server_hostname=HOSTNAME) print(ssock.cipher()) ('ECDHE-ECDSA-CHACHA20-POLY1305', 'TLSv1.2', 256) ('ECDHE-RSA-AES128-GCM-SHA256', 'TLSv1.2', 128) ('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256) print(ssock.cipher()) ('ECDHE-ECDSA-CHACHA20-POLY1305', 'TLSv1.2', 256) ('ECDHE-RSA-AES128-GCM-SHA256', 'TLSv1.2', 128) ('TLS_AES_256_GCM_SHA384', 'TLSv1.3', 256) sock = socket.create_connection((HOSTNAME, 443)) sock = socket.create_connection((HOSTNAME, 443))
23. Crypto 101 (in 3 minutes)
24. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Secure hash functions Hash functions for MAC, signatures, and more. • MD5 • SHA (SHA1) • SHA2 • SHA-256 • SHA-384
25. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Symmetric-key algorithm (bulk encryption) Same key for encryption and decryption. • DES • Triple DES (3DES) • RC4 • AES (AES-128, AES-256) • CHACHA20
26. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0
27. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0
28. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Symmetric-key algorithm (2) • Padding • Mode of operation • CBC • GCM • Authenticated encryption • CBC with MtE • AEAD (GCM, Poly1305)
29. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Asymmetric cryptographic algorithms public / private key cryptography • asymmetric encryption • signatures • key agreement
30. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Asymmetric encryption public key encrypts, private key decrypts • ElGamal encryption • RSA encryption (PKCS#1) • RSAES-PKCS1-v1.5 • RSAES-OAEP
31. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Signatures private key signs hash of message message, public key verifies • RSA signature (PKCS#1) • RSASSA-PKCS1-v1.5 • RSAES-PSS • DSS (DSA) • ECDSA (secp256r1, secp384r1, …) • EdDSA (Edward Curve25519, …)
32. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Key agreement protocol own private key + peer's public = key • finite field Diffie-Hellman (DH) • elliptic curve Diffie-Hellman (ECDH) • ephemeral DH / ECDH Common secret = = Secret colours + + (assume that mixture separation is expensive) Public transport = = Secret colours + + Common paint Alice Bob
33. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Cryptographic building blocks • key agreement / exchange • authentication algorithm • bulk encryption (symmetric) • cipher mode • one-way function for message authentication (MAC)
34. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 SSL_RSA_WITH_NULL_MD5 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-ECDSA-CHACH0-POLY1305 TLS_AES_256_GCM_SHA384
35. TLS handshake
36. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 TLS handshake • exchange parameters • version, ciphers, server name (virtual host) • verify identity of server • server signs with its private key • certificate, trust chain, hostname • agree on pre-master secret • derive secrets • verify handshake ssock = ctx.wrap_socket(sock, server_hostname=HOSTNAME) ssock = ctx.wrap_socket(sock, server_hostname=HOSTNAME)
37. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 TLS 1.2 handshake with RSA key exchange ClientHello Supported cipher suites max version, client random, ... ServerHello select cipher suite version, server random, ... Certificate Chain ServerHelloDone Finish MAC of handshake message ChangeCipherSpec ClientKeyChange RSA encrypted pre-master secret Finish MAC of handshake message ChangeCipherSpec HTTP GET (verify mac)
38. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 TLS 1.2 handshake with Diffie-Hellman ClientHello supported cipher suites max version, client random, server name indicator (SNI), ... Finish MAC of handshake message ChangeCipherSpec ClientKeyChange Diffie-Hellman server params Finish MAC of handshake message ChangeCipherSpec HTTP GET (verify mac) ServerHello select cipher suite version, server random, ... Certificate Chain ServerHelloDone ServerKeyExchange Diffie-Hellman server params Signature
39. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Message callback (Python 3.8) def msg_cb(conn, direction, version, content_type, msg_type, data): if content_type is not ssl._TLSContentType.HEADER: print(f"{direction:<5} {version._name_:<7} " f"{msg_type._name_:<20} {len(data)}") ctx._msg_callback = msg_cb def msg_cb(conn, direction, version, content_type, msg_type, data): if content_type is not ssl._TLSContentType.HEADER: print(f"{direction:<5} {version._name_:<7} " f"{msg_type._name_:<20} {len(data)}") ctx._msg_callback = msg_cb write TLSv1_2 CLIENT_HELLO 213 read TLSv1_2 SERVER_HELLO 63 read TLSv1_2 CERTIFICATE 4616 read TLSv1_2 SERVER_KEY_EXCHANGE 115 read TLSv1_2 SERVER_DONE 4 write TLSv1_2 CLIENT_KEY_EXCHANGE 37 write TLSv1_2 CHANGE_CIPHER_SPEC 1 write TLSv1_2 FINISHED 16 read TLSv1_2 NEWSESSION_TICKET 202 read TLSv1_2 FINISHED 16 write TLSv1_2 CLIENT_HELLO 213 read TLSv1_2 SERVER_HELLO 63 read TLSv1_2 CERTIFICATE 4616 read TLSv1_2 SERVER_KEY_EXCHANGE 115 read TLSv1_2 SERVER_DONE 4 write TLSv1_2 CLIENT_KEY_EXCHANGE 37 write TLSv1_2 CHANGE_CIPHER_SPEC 1 write TLSv1_2 FINISHED 16 read TLSv1_2 NEWSESSION_TICKET 202 read TLSv1_2 FINISHED 16
40. Certificates & Public Key Infrastructure
41. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Certificate validation ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) ctx.minimum_version = ssl.TLSVersion.TLSv1_2 ctx.maximum_version = ssl.TLSVersion.TLSv1_2 ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_default_certs() ctx.check_hostname = True ssock = ctx.wrap_socket(sock, server_hostname=HOSTNAME) ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) ctx.minimum_version = ssl.TLSVersion.TLSv1_2 ctx.maximum_version = ssl.TLSVersion.TLSv1_2 ctx.verify_mode = ssl.CERT_REQUIRED ctx.load_default_certs() ctx.check_hostname = True ssock = ctx.wrap_socket(sock, server_hostname=HOSTNAME)
42. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 X.509 certificates • ASN.1 • CER/DER: binary ASN.1 • PEM: base64 encoded ASN.1 + header/footer • P12, PFX: PKCS#12 safe bags • cert / private key pair • content • public key • metadata • extensions • issuer signature
43. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Fields • Serial number • Subject • Issuer • Validity • notBefore • notAfter • Subject Public Key Information • algorithm • public key • X509v3 extensions
44. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 X509v3 extensions • Basic Constraints • Key Usage • Extended Key Usage • Subject Alternative Name • Subject Key ID & Authority Key ID • CRL distribution point • Authority Information Access (OCSP, parent CA) • Certificate Policy & Naming Policy • SCT (Certificate Transparency) • ...
45. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Certificate types • trust anchors (root CA certs) • intermediate CA certs • end-entity certs • server • client • code signing • email • CRL/OCSP signing • ... root CA self-signs intermediate CA 1 intermediate CA 2 signs end-entity cert signs signs
46. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Trust store for root CAs • Linux/BSD: PKCS#11 crypto token dumped to file → • /etc/pki/tls/certs/ca-bundle.crt • /etc/ssl/certs/ca-certificates.crt • /etc/ssl/ca-bundle.pem • /etc/ssl/cert.pem • /usr/local/etc/ssl/cert.pem • /usr/local/share/certs/ca-root-nss.crt • /etc/ssl/certs/ • /system/etc/security/cacerts • Windows: CryptoAPI (registry), SChannel API • macOS: keychain, SecureTransport API
47. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 pylondinium.org X.509 certificate Certificate: Data: Version: 3 (0x2) Serial Number: 2f:90:ae:04:83:e7:4d:41:7c:c7:1e:ff:66:71:ee:11 Signature Algorithm: ecdsa-with-SHA256 Issuer: C = GB, ST = Greater Manchester, L = Salford, O = ... Validity Not Before: Feb 11 00:00:00 2019 GMT Not After : Aug 20 23:59:59 2019 GMT Subject: OU = Domain Control Validated, OU = PositiveSSL Multi- Domain, CN = sni216722.cloudflaressl.com ... X509v3 extensions: X509v3 Subject Alternative Name: DNS:sni216722.cloudflaressl.com, DNS:*.cazinovulkan-platinum.com, DNS:*.club-vulcan-casino.net, ..., DNS:*.pylondinium.org, DNS:*.resident- slot.com, ..., DNS:pylondinium.org, ... Certificate: Data: Version: 3 (0x2) Serial Number: 2f:90:ae:04:83:e7:4d:41:7c:c7:1e:ff:66:71:ee:11 Signature Algorithm: ecdsa-with-SHA256 Issuer: C = GB, ST = Greater Manchester, L = Salford, O = ... Validity Not Before: Feb 11 00:00:00 2019 GMT Not After : Aug 20 23:59:59 2019 GMT Subject: OU = Domain Control Validated, OU = PositiveSSL Multi- Domain, CN = sni216722.cloudflaressl.com ... X509v3 extensions: X509v3 Subject Alternative Name: DNS:sni216722.cloudflaressl.com, DNS:*.cazinovulkan-platinum.com, DNS:*.club-vulcan-casino.net, ..., DNS:*.pylondinium.org, DNS:*.resident- slot.com, ..., DNS:pylondinium.org, ...
48. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Don't roll your own verification • CN/SAN • wildcards • internationalized domain names (IDNA) • > 6 7 bugs in Python's hostname verification code • CVE-2013-2099, bpo-12000, bpo-17997, bpo-17305, bpo-30141 • Python 3.7 uses X509_VERIFY_PARAM_set1_host() OpenSSL 1.0.2+ / LibreSSL 2.7.0
49. TLS 1.3
50. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Kill all the bad crypto! • RC4, 3DES, AES-CBC • MD5, SHA1 • NULL ciphers • arbitrary DH groups and curves • static RSA authentication • renegotiation • compression • PKCS#1 v1.5 • MAC then Encrypt
51. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 New modern crypto • Elliptic Curve Crypto • Edwards Curve (Ed25519) • Curve25519, X25519 • authenticated encryption (AEAD) • AES-GCM • CHACHA20-Poly1305 • mandatory Diffie-Hellman PFS → • no “out-of-band TLS decryption” • RSAES-PSS signatures
52. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Protocol improvements • Cipher negotiation protected by Finish MAC (LogJam) • Separation of key agreement, ciphers, authentication • Cipher suites • TLS_AES_128_GCM_SHA256, TLS_CHACHA20_POLY1305_SHA256 • TLS extensions • signature_algorithms, cert_signature_algorithms, elliptic_curves • Session resumption with PSK-ECDHE • after Finish, TLS encrypted, next master key • Encrypted Post-Handshake auth (client cert)
53. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 TLS 1.3 with 1-RTT handshake ClientHello Supported ciphers, groups, signatures supported versions SNI Key Share ServerHello select cipher, group, signature Key Share Certificate Chain Finish MAC & Signature ChangeCipherSpec Finish MAC HTTP GET ChangeCipherSpec
54. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 TLS 1.3 with OpenSSL 1.1.1 ctx.minimum_version = ssl.TLSVersion.TLSv1_3 ctx.maximum_version = ssl.TLSVersion.TLSv1_3 ctx.minimum_version = ssl.TLSVersion.TLSv1_3 ctx.maximum_version = ssl.TLSVersion.TLSv1_3 write TLSv1_3 CLIENT_HELLO 512 read TLSv1_3 SERVER_HELLO 122 read TLSv1_3 CERTIFICATE_STATUS 1 read TLSv1_3 ENCRYPTED_EXTENSIONS 6 read TLSv1_3 CERTIFICATE 4623 read TLSv1_3 CERTIFICATE_VERIFY 78 read TLSv1_3 FINISHED 52 write TLSv1_3 CHANGE_CIPHER_SPEC 1 write TLSv1_3 CERTIFICATE_STATUS 1 write TLSv1_3 FINISHED 52 write TLSv1_3 CLIENT_HELLO 512 read TLSv1_3 SERVER_HELLO 122 read TLSv1_3 CERTIFICATE_STATUS 1 read TLSv1_3 ENCRYPTED_EXTENSIONS 6 read TLSv1_3 CERTIFICATE 4623 read TLSv1_3 CERTIFICATE_VERIFY 78 read TLSv1_3 FINISHED 52 write TLSv1_3 CHANGE_CIPHER_SPEC 1 write TLSv1_3 CERTIFICATE_STATUS 1 write TLSv1_3 FINISHED 52
55. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 TLS 1.3 with 2-RTT retry handshake ClientHello Supported ciphers, groups, signatures supported versions Key Share ServerHello select cipher, group, signature Key Share Certificate Chain Finish MAC HTTP GET Finish MAC & Signature ChangeCipherSpec ChangeCipherSpec HelloRetryRequest select cipher, group, signature Cookie ClientHello Cookie New Key Share
56. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 TLS 1.3 with 0-RTT early data ClientHello PSK + PSK mode Key Share ServerHello PSK Key Share Finish MAC & Signature NewSessionTicket PSK Early data HTTP GET Early data HTTP Response Finish MAC TLS Alert close_notify
57. Summary
58. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Use HTTPS everywhere!
59. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 from __future__ import ??? • Update to OpenSSL 1.1.1 • 1.02 and 1.1.0 EOL • PEP 543 – A Unified TLS API for Python • Replacement for certifi package “Ceterum censeo certifinem esse delendam” • HTTP/3 (UDP/QUIC)
60. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Books
61. TLS/SSL, PyLondinium 2019, @ChristianHeimes, CC BY-SA 4.0 Resources • https://www.ssllabs.com/ssltest/ • https://istlsfastyet.com/ • The Illustrated TLS Connection https://tls.ulfheim.net/ • Deploying TLS 1.3: the great, the good and the bad (33c3) https://www.youtube.com/watch?v=0opakLwtPWk
62. THANK YOU plus.google.com/+RedHat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHatNews linkedin.com/company/red-hat