Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Adventures in the Dungeons of OpenSSL

December 15, 2023

Adventures in the Dungeons of OpenSSL

12/15/2023 @ RubyConf Taiwan 2023 https://2023.rubyconf.tw/


(original description from timetable)

As part of implementing Hybrid Public Key Encryption (HPKE; RFC 9180) in Ruby, I had a chance to send a patch into Ruby's OpenSSL gem. Missing functionalities? Major version upgrade of the OpenSSL backend? There is a deep dungeon behind one of Ruby's default gem, and I will talk about one adventure through this dungeon.


December 15, 2023

More Decks by sylph01

Other Decks in Programming


  1. Adventures in the Dungeons of OpenSSL Ryo Kajiwara/ 梶原 龍

    (sylph01) 2023/12/15 @ RubyConf Taiwan 2023 1
  2. I do stuff Play rhythm games (especially DanceDanceRevolution) Play the

    bassoon/contrabassoon Ride a lot of trains (Rails!) (travelled on 99% of JR) Build keyboards if anything catches your interest let's talk! 4
  3. And I do stuff that is more relevant to this

    talk: Freelance web developer focused on Digital Identity and Security Mainly working at codeTakt, developing ID platforms for schools Worked/ing on writing/editing and implementing standards HTTPS in Local Network CG / Web of Things WG @ W3C, OAuth / Messaging Layer Security WG @ IETF Worked as an Officer of Internet Society Japan Chapter (2020-23) 5
  4. I'm from the Forgotten Realm of Japan called Shikoku image

    https:/ /twitter.com/Mitchan_599/status/994221711942971392 6
  5. 8

  6. Caution Cryptographic API can be very easy to misuse What's

    different from dungeons like parse.y -vania is that you can actually hurt yourself. Do you like security breaches? I've done my research, but I don't consider myself a cryptography expert I don't have a PhD/Master in this field, so yeah... If you're not sure, please have your system audited by a security expert before going to production 12
  7. Caution The level of production-readiness in this talk: The hpke

    gem: Almost ready Minor changes in the gem API are possible Mostly uses pretty safe stuff But not audited by an external security expert OpenSSL extension version of HPKE: Definitely not 13
  8. Past Related Work Do Pure Ruby Dream of Encrypted Binary

    Protocol? / Yusuke Nakamura @ RubyKaigi 2021 Implementing QUIC in Ruby Talks about the pain of handling hex-encoded and raw strings in Ruby 14
  9. 15

  10. Hello Alice! Alice's private key Encrypt 6EB69570 08E03CE4 Hello Alice!

    Decrypt Alice's public key Alice Bob Recap: Public Key Encrypytion Alice generates key pair Bob encrypts with Public Key Alice can decrypt with Private Key https:/ /commons.wikimedia.org/wiki/File:Public_key_encryption.svg 17
  11. What is HPKE In the past: Encrypt a session key

    using Public Key Cryptography like RSA Then send your messages using symmetric ciphers like AES You don't do everything with PKC because it's costly 18
  12. What is HPKE The problem: There is lots of misuse

    and implementation bugs in cryptography PKCS#1 padding in RSA leading to Bleichenbacher's Oracle Attack Nonce reuse DSA/ECDSA: leading to leakage of private key in PlayStation 3's code signing AES's initialization vector re-use is everywhere! 19
  13. What is HPKE You (typically) don't encrypt/decrypt with Elliptic Curve

    Cryptography You get key exchange (ECDH) and digital signatures (ECDSA) so the "encrypt with Public Key" trick doesn't work 20
  14. What is HPKE HPKE solves this by: Using standardized/reviewed protocols

    to exchange keys and encrypt/decrypt Using best known cipher suites possible combination of cryptographic algorithms Using high-level APIs that prevent misuse 21
  15. What is HPKE HPKE uses Agreement of symmetric keys using

    a Key Encapsulation Mechanism (KEM) that internally uses a Key Derivation Function (KDF) Then use the symmetric keys to perform Authenticated Encryption with Associated Data (AEAD) 22
  16. How to use HPKE HPKE is a one-way protocol from

    Sender to Receiver Agreement of symmetric key using secret exporter is possible Receiver creates a public/private key pair, publishes public key Sender uses public key to encapsulate session key, then uses the session key to encrypt message Sender sends the encapsulation and the message Receiver uses the encapsulation to decapsulate session key, uses the session key to decrypt the message 23
  17. What is needed for HPKE? Key Encapsulation Mechanism Diffie-Hellman Key

    Exchange use of Elliptic Curves: P-256, P-384, P-521, X25519, X448 Key Derivation Function HMAC-based Extract-and-Expand Key Derivation Function (HKDF) HMAC-SHA256, HMAC-SHA512 Authenticated Encryption with Associated Data AES-128-GCM, AES-256-GCM, ChaCha20-Poly1305 25
  18. What we had readily available AES-128-GCM, AES-256-GCM, ChaCha20-Poly1305 test/openssl/test_cipher.rb ChaCha20-Poly1305

    isn't in this test code, but uses the same API as AES-GCM HMAC-SHA256, HMAC-SHA512 Even with SHA256 and SHA512 only, I can implement HMAC Actually we have HKDF itself but we need to use some parts of HKDF and customize it 28
  19. Really? These seem to be undocumented Some APIs in OpenSSL

    are intentionally left undocumented to avoid misuses by people who are not well-versed in cryptography. Yes, misuse in cryptography can hurt yourself. 31
  20. OpenSSL::PKey Set of classes that handle everything Public Key Cryptography,

    including RSA, DSA, Diffie-Hellman, and Elliptic Curve Cryptography. Wraps OpenSSL's EVP_PKEY struct. Note that X25519/X448 are actually ECC but does not use OpenSSL::PKey::EC . 32
  21. Raw public/private key support for X25519/X448 There was a pull

    request that worked on this But it was 3 years old and not working on some platforms So I started working on this by Fixing errors on the CI matrix Addressing unfixed review comments 34
  22. So you can't do this pkey = OpenSSL::PKey::EC.new('prime256v1') pkey.private_key =

    SecureRandom.random_bytes(32) Note: A private key of P-256 elliptic curve consists of a 32 byte number (=scalar). 38
  23. In OpenSSL 3.0, we were not just missing APIs to

    make X25519/X448 key pairs, we didn't even have APIs to make EC key pairs! 39
  24. This was available der = # write something in ASN.1

    DER format OpenSSL::PKey.read(der) 41
  25. Enter ASN.1 Abstract Syntax Notation One Set of binary notation

    rules for encoding data structures DER (Distinguished Encoding Rules) is the ruleset to describe data structures in exactly one way, so as to ensure digital signatures produce a unique output PEM files encode certificates private keys written in DER with Base64 43
  26. PKey -> DER -> ASN.1 Sequence pk = OpenSSL::PKey::EC.generate('prime256v1') pk.private_to_der.unpack1('H*')

    => "308187020100301306072a8648ce3d020106082a8648 ce3d030107046d306b02010104203baadea1f85b96b3 3b8a895ab2c44a4b72e827ff1b1ac23e7b756daef2f0 892ba14403420004b9f0779b47f432fd7df13b67cf21 0c58db276653a13b862db2eb6d47f3eabe9ac4317181 8e19e78f37fe5e86a7439d471d0f3442a2b727f9560a ff417d432391" (formatting just for visibility purposes) 44
  27. ASN.1 Sequence -> DER -> PKey::EC Do it backwards! def

    create_key_pair_from_secret(secret) asn1_seq = OpenSSL::ASN1.Sequence([ OpenSSL::ASN1.Integer(1), OpenSSL::ASN1.OctetString(secret), OpenSSL::ASN1.ObjectId(curve_name, 0, :EXPLICIT) ]) OpenSSL::PKey.read(asn1_seq.to_der) end 46
  28. I did this in the HPKE gem for X25519/X448 too

    def create_key_pair_from_secret(secret) asn1_seq = OpenSSL::ASN1.Sequence([ OpenSSL::ASN1.Integer(0), OpenSSL::ASN1.Sequence([ OpenSSL::ASN1.ObjectId(asn1_oid) ]), OpenSSL::ASN1.OctetString("\x04\x20" + secret) ]) OpenSSL::PKey.read(asn1_seq.to_der) end 47
  29. Now we have HPKE in Ruby GH: sylph01/hpke-rb gem install

    hpke now! note: Still in beta. I will likely tinker around with the API of the gem 50
  30. 51

  31. Well actually, we had HPKE itself in OpenSSL 3.2 blog

    post in OpenSSL Blog: https:/ /www.openssl.org/blog/blog/2023/10/18/ossl-hpke/ 52
  32. so I did. GH: sylph01/openssl, hpke branch (note: this is

    super experimental territory, DO NOT use in production!) 54
  33. How to develop C extensions GH: ruby/ruby/doc/extension.rdoc is your friend.

    You want to know how to: Convert C values into Ruby values and vice versa Wrap a C struct into a Ruby object 55
  34. Handling HPKE Context OSSL_HPKE_CTX *OSSL_HPKE_CTX_new(int mode, OSSL_HPKE_SUITE suite, int role,

    OSSL_LIB_CTX *libctx, const char *propq); int OSSL_HPKE_encap(OSSL_HPKE_CTX *ctx, unsigned char *enc, size_t *enclen, const unsigned char *pub, size_t publen, const unsigned char *info, size_t infolen); int OSSL_HPKE_seal(OSSL_HPKE_CTX *ctx, unsigned char *ct, size_t *ctlen, const unsigned char *aad, size_t aadlen, const unsigned char *pt, size_t ptlen); https:/ /www.openssl.org/docs/manmaster/man3/OSSL_HPKE_CTX_new.html 56
  35. Handling HPKE Context static void ossl_hpke_ctx_free(void *ptr) { OSSL_HPKE_CTX_free(ptr); }

    const rb_data_type_t ossl_hpke_ctx_type = { "OpenSSL/HPKE_CTX", { 0, ossl_hpke_ctx_free, }, 0, 0, RUBY_TYPED_FREE_IMMEDIATELY }; 57
  36. Handling HPKE Context static VALUE hpke_ctx_new0(VALUE arg){ OSSL_HPKE_CTX *ctx =

    (OSSL_HPKE_CTX *)arg; VALUE obj; obj = rb_obj_alloc(cContext); RTYPEDDATA_DATA(obj) = ctx; return obj; } VALUE ossl_hpke_ctx_new(OSSL_HPKE_CTX *ctx){ VALUE obj; int status; obj = rb_protect(hpke_ctx_new0, (VALUE)ctx, &status); if (status) { OSSL_HPKE_CTX_free(ctx); rb_jump_tag(status); } return obj; } 58
  37. Handling HPKE Context static VALUE ossl_hpke_ctx_alloc(VALUE klass) { return TypedData_Wrap_Struct(klass,

    &ossl_hpke_ctx_type, NULL); } void Init_ossl_hpke_ctx(void) { mHPKE = rb_define_module_under(mOSSL, "HPKE"); cContext = rb_define_class_under(mHPKE, "Context", rb_cObject); ... rb_define_alloc_func(cContext, ossl_hpke_ctx_alloc); } 59
  38. Generating the key int OSSL_HPKE_keygen(OSSL_HPKE_SUITE suite, unsigned char *pub, size_t

    *publen, EVP_PKEY **priv, const unsigned char *ikm, size_t ikmlen, OSSL_LIB_CTX *libctx, const char *propq); priv is an EVP_PKEY so we can wrap that into OpenSSL::PKey pub is an unsigned char * = String, but which format? 60
  39. Generating the key Apparently: for EC keys it's the equivalent

    of priv.public_key.to_octet_string(:uncompressed) for X25519/X448 keys it's the equivalent of priv.raw_public_key 61
  40. Encapsulating the key VALUE ossl_hpke_encap(VALUE self, VALUE pub, VALUE info)

    { (snip definitions) GetHpkeCtx(self, sctx); // extract C pointer to context from object enclen = sizeof(enc); if (OSSL_HPKE_encap( sctx, enc, &enclen, (unsigned char*)RSTRING_PTR(pub), RSTRING_LEN(pub), (unsigned char*)RSTRING_PTR(info), RSTRING_LEN(info)) != 1) { ossl_raise(eHPKEError, "could not encap"); } enc_obj = rb_str_new((char *)enc, enclen); return enc_obj; } 62
  41. How to develop (OpenSSL) ruby/openssl's CONTRIBUTING.md has a "Testing: With

    different versions of OpenSSL" section: git checkout openssl-3.2 OPENSSL_DIR=$HOME/.openssl/openssl-something ./Configure --prefix=$OPENSSL_DIR --libdir=lib enable- fips enable-trace '-Wl,-rpath,$(LIBRPATH)' -O0 -g3 -ggdb3 -gdwarf-5 make -j4 , make install 63
  42. How to develop (OpenSSL) I wanted to peek inside the

    OSSL_HPKE_CTX , but I got an invalid use of incomplete typedef error. This is because the full typedef is not exposed to the public header files. I moved the internal definitions to public header file <openssl/hpke.h> then recompiled OpenSSL. 64
  43. How to develop (OpenSSL gem) Then to build with that

    version: bundle exec rake clean bundle exec rake compile -- --with-openssl- dir=$OPENSSL_DIR irb -I lib -r openssl 65
  44. How to develop (debugging) I wanted to see the C

    string in hex printed into the console, so: void rbdebug_print_hex(const unsigned char *str, size_t len) { VALUE rbstr; rbstr = rb_str_new((char *)str, len); rb_p(rb_funcall(rbstr, rb_intern("unpack1"), 1, rb_str_new_cstr("H*"))); } rb_p is your friend. Also rb_funcall is nice too. 66
  45. Example irb(main):001:0> suite = OpenSSL::HPKE::Suite.new_with_names( :dhkem_p256_hkdf_sha256, :hkdf_sha256, :aes_128_gcm) => #<OpenSSL::HPKE::Suite:0x00007f5460d55b68

    @aead_id=1, @kdf_id=1, @kem_id=16> irb(main):002:0> priv = OpenSSL::HPKE.keygen_with_suite(suite) irb(main):003:0> pub = priv.public_key => #<OpenSSL::PKey::EC::Point:0x00007f5460d67520 @group=#<OpenSSL::PKey::EC::Group:0x00007f5460d674d0>> Generate a public/private key pair 67
  46. Example # sender irb(main):004:0> sctx = OpenSSL::HPKE::Context::Sender.new(:base, suite) => #<OpenSSL::HPKE::Context::Sender:0x00007f5460d22768

    @aead_id=1, @kdf_id=1, @kem_id=16> irb(main):005:0> enc = sctx.encap(pub.to_octet_string(:uncompressed), "Some info") irb(main):006:0> ct = sctx.seal("\x01\x02\x03\x04\x05\x06\x07\x08", "a message not in a bottle") => "\x93\xC1\xDF7\x87\xA8\xBER\xC9&\xA4\xD1\xB7\x10\x8E..." Create sender context Encapsulate key into the public key of the receiver Using the generated shared secret, seal message before commit 4548dd0e it uses a different style of initialization API 68
  47. Example # receiver irb(main):007:0> rctx = OpenSSL::HPKE::Context::Receiver.new(:base, suite) => #<OpenSSL::HPKE::Context::Receiver:0x00007f5460d64e88

    @aead_id=1, @kdf_id=1, @kem_id=16> irb(main):008:0> rctx.decap(enc, priv, "Some info") irb(main):009:0> pt = rctx.open("\x01\x02\x03\x04\x05\x06\x07\x08", ct) => "a message not in a bottle" Receiver gets the encapsulation enc and sealed message ct Create receiver context Decapsulate key from enc with private key and derive shared secret Then using the derived shared secret, open message 69
  48. By the way... Is anyone familiar with Ruby's encoding? I

    don't want the force_encoding but I kinda need it now irb(main):010:0> sctx = OpenSSL::HPKE::Context::Sender.new(:base, suite) irb(main):011:0> rctx = OpenSSL::HPKE::Context::Receiver.new(:base, suite) => #<OpenSSL::HPKE::Context::Receiver:0x00007f5460d2d8c0 @aead_id=1, @kdf_id=1, @kem_id=16> irb(main):012:0> enc = sctx.encap(pub.to_octet_string(:uncompressed), "Some info") => "\x04\x17M\b\xE0\x9D\xEEg\xE2..." irb(main):013:0> ct = sctx.seal("\x01\x02\x03\x04\x05\x06\x07\x08", " ニッホンゴー") => "E\x80\xEFP\x95Z9N\x1F\t..." irb(main):014:0> irb(main):015:0> rctx.decap(enc, priv, "Some info") => true irb(main):016:0> pt = rctx.open("\x01\x02\x03\x04\x05\x06\x07\x08", ct) irb(main):017:0> puts pt.force_encoding('UTF-8') ニッホンゴー => nil 70
  49. Will this go into actual OpenSSL gem? This still needs

    a lot of work: This is limited to OpenSSL 3.2, so needs guards against older versions Currently supports Base mode only Hardcoded length values need to be fixed Is the C coding actually safe? 71
  50. 72

  51. Why HPKE in Ruby? It's the building block for modern

    security/privacy protocols TLS Encrypted ClientHello Oblivious HTTP Messaging Layer Security 75
  52. Why HPKE in Ruby? If we don't have the building

    block for modern networking protocols, people would not implement them ... and just go to Python / Go / Rust / whatever because they have the building blocks readily available. 76
  53. The adventure continues... HPKE is just a building block for

    other protocols I am working on implementing protocols that rely on HPKE Also now that I came back from the OpenSSL dungeon alive, I might continue digging into this "Forgotten Realm" 79
  54. Shoutouts (@ mentions are in GitHub ID) Ruby OpenSSL maintainers,

    esp. @rhenium Past RubyKaigi speakers, esp. @unasuke and @shioimm HPKE implementers, esp. @dajiaji And to the organizers of RubyConf Taiwan 2023! 80