Share
Embed
Adventures in the Dungeons of OpenSSL
12/15/2023 @ RubyConf Taiwan 2023 https://2023.rubyconf.tw/
----
(original description from timetable)
As part of implementing Hybrid Public Key Encryption (HPKE; RFC 9180) in Ruby, I had a chance to send a patch into Ruby's OpenSSL gem. Missing functionalities? Major version upgrade of the OpenSSL backend? There is a deep dungeon behind one of Ruby's default gem, and I will talk about one adventure through this dungeon.
sylph01
December 15, 2023
Transcript
81 slides
-
Adventures in the Dungeons of OpenSSL Ryo Kajiwara/ 梶原 龍
(sylph01) 2023/12/15 @ RubyConf Taiwan 2023 1 -
I do stuff Play rhythm games (especially DanceDanceRevolution) Play the
bassoon/contrabassoon Ride a lot of trains (Rails!) (travelled on 99% of JR) Build keyboards if anything catches your interest let's talk! 4 -
And I do stuff that is more relevant to this
talk: Freelance web developer focused on Digital Identity and Security Mainly working at codeTakt, developing ID platforms for schools Worked/ing on writing/editing and implementing standards HTTPS in Local Network CG / Web of Things WG @ W3C, OAuth / Messaging Layer Security WG @ IETF Worked as an Officer of Internet Society Japan Chapter (2020-23) 5 -
I'm from the Forgotten Realm of Japan called Shikoku image
https:/ /twitter.com/Mitchan_599/status/994221711942971392 6 -
My ticket this time was from 松山 (Matsuyama) to 松山
(Songshan) 7 -
Caution Cryptographic API can be very easy to misuse What's
different from dungeons like parse.y -vania is that you can actually hurt yourself. Do you like security breaches? I've done my research, but I don't consider myself a cryptography expert I don't have a PhD/Master in this field, so yeah... If you're not sure, please have your system audited by a security expert before going to production 12 -
Caution The level of production-readiness in this talk: The hpke
gem: Almost ready Minor changes in the gem API are possible Mostly uses pretty safe stuff But not audited by an external security expert OpenSSL extension version of HPKE: Definitely not 13 -
Past Related Work Do Pure Ruby Dream of Encrypted Binary
Protocol? / Yusuke Nakamura @ RubyKaigi 2021 Implementing QUIC in Ruby Talks about the pain of handling hex-encoded and raw strings in Ruby 14 -
Hello Alice! Alice's private key Encrypt 6EB69570 08E03CE4 Hello Alice!
Decrypt Alice's public key Alice Bob Recap: Public Key Encrypytion Alice generates key pair Bob encrypts with Public Key Alice can decrypt with Private Key https:/ /commons.wikimedia.org/wiki/File:Public_key_encryption.svg 17 -
What is HPKE In the past: Encrypt a session key
using Public Key Cryptography like RSA Then send your messages using symmetric ciphers like AES You don't do everything with PKC because it's costly 18 -
What is HPKE The problem: There is lots of misuse
and implementation bugs in cryptography PKCS#1 padding in RSA leading to Bleichenbacher's Oracle Attack Nonce reuse DSA/ECDSA: leading to leakage of private key in PlayStation 3's code signing AES's initialization vector re-use is everywhere! 19 -
What is HPKE You (typically) don't encrypt/decrypt with Elliptic Curve
Cryptography You get key exchange (ECDH) and digital signatures (ECDSA) so the "encrypt with Public Key" trick doesn't work 20 -
What is HPKE HPKE solves this by: Using standardized/reviewed protocols
to exchange keys and encrypt/decrypt Using best known cipher suites possible combination of cryptographic algorithms Using high-level APIs that prevent misuse 21 -
What is HPKE HPKE uses Agreement of symmetric keys using
a Key Encapsulation Mechanism (KEM) that internally uses a Key Derivation Function (KDF) Then use the symmetric keys to perform Authenticated Encryption with Associated Data (AEAD) 22 -
How to use HPKE HPKE is a one-way protocol from
Sender to Receiver Agreement of symmetric key using secret exporter is possible Receiver creates a public/private key pair, publishes public key Sender uses public key to encapsulate session key, then uses the session key to encrypt message Sender sends the encapsulation and the message Receiver uses the encapsulation to decapsulate session key, uses the session key to decrypt the message 23 -
What is needed for HPKE? Key Encapsulation Mechanism Diffie-Hellman Key
Exchange use of Elliptic Curves: P-256, P-384, P-521, X25519, X448 Key Derivation Function HMAC-based Extract-and-Expand Key Derivation Function (HKDF) HMAC-SHA256, HMAC-SHA512 Authenticated Encryption with Associated Data AES-128-GCM, AES-256-GCM, ChaCha20-Poly1305 25 -
What we had readily available AES-128-GCM, AES-256-GCM, ChaCha20-Poly1305 test/openssl/test_cipher.rb ChaCha20-Poly1305
isn't in this test code, but uses the same API as AES-GCM HMAC-SHA256, HMAC-SHA512 Even with SHA256 and SHA512 only, I can implement HMAC Actually we have HKDF itself but we need to use some parts of HKDF and customize it 28 -
What we 'kinda' had Elliptic curves P-256, P-384, P-521 test/openssl/test_pkey_ec.rb
X25519, X448 test/openssl/test_pkey.rb 29 -
Really? These seem to be undocumented Some APIs in OpenSSL
are intentionally left undocumented to avoid misuses by people who are not well-versed in cryptography. Yes, misuse in cryptography can hurt yourself. 31 -
OpenSSL::PKey Set of classes that handle everything Public Key Cryptography,
including RSA, DSA, Diffie-Hellman, and Elliptic Curve Cryptography. Wraps OpenSSL's EVP_PKEY struct. Note that X25519/X448 are actually ECC but does not use OpenSSL::PKey::EC . 32 -
Raw public/private key support for X25519/X448 Raw public/private key support
for EC "kinda" existed 33 -
Raw public/private key support for X25519/X448 There was a pull
request that worked on this But it was 3 years old and not working on some platforms So I started working on this by Fixing errors on the CI matrix Addressing unfixed review comments 34 -
So you can't do this pkey = OpenSSL::PKey::EC.new('prime256v1') pkey.private_key =
SecureRandom.random_bytes(32) Note: A private key of P-256 elliptic curve consists of a 32 byte number (=scalar). 38 -
In OpenSSL 3.0, we were not just missing APIs to
make X25519/X448 key pairs, we didn't even have APIs to make EC key pairs! 39 -
How can we create a key pair with a specified
private key? 40 -
This was available der = # write something in ASN.1
DER format OpenSSL::PKey.read(der) 41 -
ASN.1 Sequence -> DER -> PKey::EC it's even in Ruby
3.3 preview1! 42 -
Enter ASN.1 Abstract Syntax Notation One Set of binary notation
rules for encoding data structures DER (Distinguished Encoding Rules) is the ruleset to describe data structures in exactly one way, so as to ensure digital signatures produce a unique output PEM files encode certificates private keys written in DER with Base64 43 -
PKey -> DER -> ASN.1 Sequence pk = OpenSSL::PKey::EC.generate('prime256v1') pk.private_to_der.unpack1('H*')
=> "308187020100301306072a8648ce3d020106082a8648 ce3d030107046d306b02010104203baadea1f85b96b3 3b8a895ab2c44a4b72e827ff1b1ac23e7b756daef2f0 892ba14403420004b9f0779b47f432fd7df13b67cf21 0c58db276653a13b862db2eb6d47f3eabe9ac4317181 8e19e78f37fe5e86a7439d471d0f3442a2b727f9560a ff417d432391" (formatting just for visibility purposes) 44 -
ASN.1 Sequence -> DER -> PKey::EC Do it backwards! def
create_key_pair_from_secret(secret) asn1_seq = OpenSSL::ASN1.Sequence([ OpenSSL::ASN1.Integer(1), OpenSSL::ASN1.OctetString(secret), OpenSSL::ASN1.ObjectId(curve_name, 0, :EXPLICIT) ]) OpenSSL::PKey.read(asn1_seq.to_der) end 46 -
I did this in the HPKE gem for X25519/X448 too
def create_key_pair_from_secret(secret) asn1_seq = OpenSSL::ASN1.Sequence([ OpenSSL::ASN1.Integer(0), OpenSSL::ASN1.Sequence([ OpenSSL::ASN1.ObjectId(asn1_oid) ]), OpenSSL::ASN1.OctetString("\x04\x20" + secret) ]) OpenSSL::PKey.read(asn1_seq.to_der) end 47 -
API to generate PKeys with a private key is being
worked on @ Issue #555 49 -
Now we have HPKE in Ruby GH: sylph01/hpke-rb gem install
hpke now! note: Still in beta. I will likely tinker around with the API of the gem 50 -
Well actually, we had HPKE itself in OpenSSL 3.2 blog
post in OpenSSL Blog: https:/ /www.openssl.org/blog/blog/2023/10/18/ossl-hpke/ 52 -
so I did. GH: sylph01/openssl, hpke branch (note: this is
super experimental territory, DO NOT use in production!) 54 -
How to develop C extensions GH: ruby/ruby/doc/extension.rdoc is your friend.
You want to know how to: Convert C values into Ruby values and vice versa Wrap a C struct into a Ruby object 55 -
Handling HPKE Context OSSL_HPKE_CTX *OSSL_HPKE_CTX_new(int mode, OSSL_HPKE_SUITE suite, int role,
OSSL_LIB_CTX *libctx, const char *propq); int OSSL_HPKE_encap(OSSL_HPKE_CTX *ctx, unsigned char *enc, size_t *enclen, const unsigned char *pub, size_t publen, const unsigned char *info, size_t infolen); int OSSL_HPKE_seal(OSSL_HPKE_CTX *ctx, unsigned char *ct, size_t *ctlen, const unsigned char *aad, size_t aadlen, const unsigned char *pt, size_t ptlen); https:/ /www.openssl.org/docs/manmaster/man3/OSSL_HPKE_CTX_new.html 56 -
Handling HPKE Context static void ossl_hpke_ctx_free(void *ptr) { OSSL_HPKE_CTX_free(ptr); }
const rb_data_type_t ossl_hpke_ctx_type = { "OpenSSL/HPKE_CTX", { 0, ossl_hpke_ctx_free, }, 0, 0, RUBY_TYPED_FREE_IMMEDIATELY }; 57 -
Handling HPKE Context static VALUE hpke_ctx_new0(VALUE arg){ OSSL_HPKE_CTX *ctx =
(OSSL_HPKE_CTX *)arg; VALUE obj; obj = rb_obj_alloc(cContext); RTYPEDDATA_DATA(obj) = ctx; return obj; } VALUE ossl_hpke_ctx_new(OSSL_HPKE_CTX *ctx){ VALUE obj; int status; obj = rb_protect(hpke_ctx_new0, (VALUE)ctx, &status); if (status) { OSSL_HPKE_CTX_free(ctx); rb_jump_tag(status); } return obj; } 58 -
Handling HPKE Context static VALUE ossl_hpke_ctx_alloc(VALUE klass) { return TypedData_Wrap_Struct(klass,
&ossl_hpke_ctx_type, NULL); } void Init_ossl_hpke_ctx(void) { mHPKE = rb_define_module_under(mOSSL, "HPKE"); cContext = rb_define_class_under(mHPKE, "Context", rb_cObject); ... rb_define_alloc_func(cContext, ossl_hpke_ctx_alloc); } 59 -
Generating the key int OSSL_HPKE_keygen(OSSL_HPKE_SUITE suite, unsigned char *pub, size_t
*publen, EVP_PKEY **priv, const unsigned char *ikm, size_t ikmlen, OSSL_LIB_CTX *libctx, const char *propq); priv is an EVP_PKEY so we can wrap that into OpenSSL::PKey pub is an unsigned char * = String, but which format? 60 -
Generating the key Apparently: for EC keys it's the equivalent
of priv.public_key.to_octet_string(:uncompressed) for X25519/X448 keys it's the equivalent of priv.raw_public_key 61 -
Encapsulating the key VALUE ossl_hpke_encap(VALUE self, VALUE pub, VALUE info)
{ (snip definitions) GetHpkeCtx(self, sctx); // extract C pointer to context from object enclen = sizeof(enc); if (OSSL_HPKE_encap( sctx, enc, &enclen, (unsigned char*)RSTRING_PTR(pub), RSTRING_LEN(pub), (unsigned char*)RSTRING_PTR(info), RSTRING_LEN(info)) != 1) { ossl_raise(eHPKEError, "could not encap"); } enc_obj = rb_str_new((char *)enc, enclen); return enc_obj; } 62 -
How to develop (OpenSSL) ruby/openssl's CONTRIBUTING.md has a "Testing: With
different versions of OpenSSL" section: git checkout openssl-3.2 OPENSSL_DIR=$HOME/.openssl/openssl-something ./Configure --prefix=$OPENSSL_DIR --libdir=lib enable- fips enable-trace '-Wl,-rpath,$(LIBRPATH)' -O0 -g3 -ggdb3 -gdwarf-5 make -j4 , make install 63 -
How to develop (OpenSSL) I wanted to peek inside the
OSSL_HPKE_CTX , but I got an invalid use of incomplete typedef error. This is because the full typedef is not exposed to the public header files. I moved the internal definitions to public header file <openssl/hpke.h> then recompiled OpenSSL. 64 -
How to develop (OpenSSL gem) Then to build with that
version: bundle exec rake clean bundle exec rake compile -- --with-openssl- dir=$OPENSSL_DIR irb -I lib -r openssl 65 -
How to develop (debugging) I wanted to see the C
string in hex printed into the console, so: void rbdebug_print_hex(const unsigned char *str, size_t len) { VALUE rbstr; rbstr = rb_str_new((char *)str, len); rb_p(rb_funcall(rbstr, rb_intern("unpack1"), 1, rb_str_new_cstr("H*"))); } rb_p is your friend. Also rb_funcall is nice too. 66 -
Example irb(main):001:0> suite = OpenSSL::HPKE::Suite.new_with_names( :dhkem_p256_hkdf_sha256, :hkdf_sha256, :aes_128_gcm) => #<OpenSSL::HPKE::Suite:0x00007f5460d55b68
@aead_id=1, @kdf_id=1, @kem_id=16> irb(main):002:0> priv = OpenSSL::HPKE.keygen_with_suite(suite) irb(main):003:0> pub = priv.public_key => #<OpenSSL::PKey::EC::Point:0x00007f5460d67520 @group=#<OpenSSL::PKey::EC::Group:0x00007f5460d674d0>> Generate a public/private key pair 67 -
Example # sender irb(main):004:0> sctx = OpenSSL::HPKE::Context::Sender.new(:base, suite) => #<OpenSSL::HPKE::Context::Sender:0x00007f5460d22768
@aead_id=1, @kdf_id=1, @kem_id=16> irb(main):005:0> enc = sctx.encap(pub.to_octet_string(:uncompressed), "Some info") irb(main):006:0> ct = sctx.seal("\x01\x02\x03\x04\x05\x06\x07\x08", "a message not in a bottle") => "\x93\xC1\xDF7\x87\xA8\xBER\xC9&\xA4\xD1\xB7\x10\x8E..." Create sender context Encapsulate key into the public key of the receiver Using the generated shared secret, seal message before commit 4548dd0e it uses a different style of initialization API 68 -
Example # receiver irb(main):007:0> rctx = OpenSSL::HPKE::Context::Receiver.new(:base, suite) => #<OpenSSL::HPKE::Context::Receiver:0x00007f5460d64e88
@aead_id=1, @kdf_id=1, @kem_id=16> irb(main):008:0> rctx.decap(enc, priv, "Some info") irb(main):009:0> pt = rctx.open("\x01\x02\x03\x04\x05\x06\x07\x08", ct) => "a message not in a bottle" Receiver gets the encapsulation enc and sealed message ct Create receiver context Decapsulate key from enc with private key and derive shared secret Then using the derived shared secret, open message 69 -
By the way... Is anyone familiar with Ruby's encoding? I
don't want the force_encoding but I kinda need it now irb(main):010:0> sctx = OpenSSL::HPKE::Context::Sender.new(:base, suite) irb(main):011:0> rctx = OpenSSL::HPKE::Context::Receiver.new(:base, suite) => #<OpenSSL::HPKE::Context::Receiver:0x00007f5460d2d8c0 @aead_id=1, @kdf_id=1, @kem_id=16> irb(main):012:0> enc = sctx.encap(pub.to_octet_string(:uncompressed), "Some info") => "\x04\x17M\b\xE0\x9D\xEEg\xE2..." irb(main):013:0> ct = sctx.seal("\x01\x02\x03\x04\x05\x06\x07\x08", " ニッホンゴー") => "E\x80\xEFP\x95Z9N\x1F\t..." irb(main):014:0> irb(main):015:0> rctx.decap(enc, priv, "Some info") => true irb(main):016:0> pt = rctx.open("\x01\x02\x03\x04\x05\x06\x07\x08", ct) irb(main):017:0> puts pt.force_encoding('UTF-8') ニッホンゴー => nil 70 -
Will this go into actual OpenSSL gem? This still needs
a lot of work: This is limited to OpenSSL 3.2, so needs guards against older versions Currently supports Base mode only Hardcoded length values need to be fixed Is the C coding actually safe? 71 -
Why HPKE in Ruby? It's the building block for modern
security/privacy protocols TLS Encrypted ClientHello Oblivious HTTP Messaging Layer Security 75 -
Why HPKE in Ruby? If we don't have the building
block for modern networking protocols, people would not implement them ... and just go to Python / Go / Rust / whatever because they have the building blocks readily available. 76 -
The adventure continues... HPKE is just a building block for
other protocols I am working on implementing protocols that rely on HPKE Also now that I came back from the OpenSSL dungeon alive, I might continue digging into this "Forgotten Realm" 79 -
Shoutouts (@ mentions are in GitHub ID) Ruby OpenSSL maintainers,
esp. @rhenium Past RubyKaigi speakers, esp. @unasuke and @shioimm HPKE implementers, esp. @dajiaji And to the organizers of RubyConf Taiwan 2023! 80