How to Interchange Ed25519 Keys

[This fragment is available in an audio version.]

Herewith pointers to Java 15 and Go code that converts Ed25119 public keys back and forth between short text strings and key objects you can use to verify signatures. The code isn’t big or complicated, but it took me quite a bit of work and time to figure out, and led down surprisingly dusty and ancient pathways. Posted to help others who need to do this and perhaps provide mild entertainment.
[Update 04/23: “agwa” over at YCombinator showed how to simplify the Go with x509.MarshalPKIXPublicKey and x509.ParsePKIXPublicKey.]

They call modern crypto “public-key”; because keys are public, people can post them on the Internet so other people’s code can use them, for example to verify signatures.

How, exactly, I wondered, would you go about doing that? The good news is that, in Go and Java 15 at least, there is good core library support for doing the necessary incantations, with no need to take any external dependencies.

If you don’t care about the history and weirdness, here’s the Go code and here’s the Java..

Now, on to the back story. But first…

Should I even do this? · “Don’t write your own crypto!” they say. And I’ve never been tempted. But I wonder if there’s a variant form that says “Don’t even use the professionally-crafted crypto libraries and fool around with keypairs unless you know what you’re doing!”

Because as I stumbled through the undergrowth figuring this stuff out in the usual way, via Google and StackOverflow, I did not convince myself that the path I was following was the right one. I wondered if there was a gnostic body of asymmetric-crypto lore and if I’d ever studied it I’d know the right way to do what I was trying do. And since I hadn’t studied it, I should just leave this stuff to the people who had. I’m not being ironic or sarcastic here, I really do wonder that.

Let’s give it a try · I’m building bits and pieces of software related to my @bluesky Identity scheme. I want a person to be able to post a “Zero-Knowledge Proof” which includes a public key and a signature so other people can check the signature — go check that link for details. I started exploring using the Go programming language, which has a nice generic crypto package and then a boatload of other packages for different key flavors, with names like aes and rsa and ed25519 that even a crypto peasant like me recognizes.

Ed25519 · This is the new-ish hotness in public-key tech. It’s fast and very safe and produces small keys and signatures. It’s also blessedly free of necessary knobs to twist, for example you don’t have to think about hash functions. It’s not quantum-safe but for this particular application that’s a complete non-issue. I’ve come to like it a lot.

Anyhow, a bit of poking around the usual parts of the Internet suggested that what a good little Gopher wants is crypto/ed25519. For a while I was wondering if I might need to support other algorithms too; let’s push that on the stack for now.

Base64 and PEM and ASN.1, oh my! · I remembered that these things are usually in base64, wrapped in lines that look like -----BEGIN PUBLIC KEY----- and -----END PUBLIC KEY-----. A bit of Googling reminds me that this is PEM format, developed starting in 1985 as “Privacy-Enhanced Mail”. Suddenly I feel young! OK then, let’s respect the past.

Go has a pem package to deal with these things, works about as you’d expect. And what, I wondered, might I find inside?

Abstract Syntax Notation One · Everyone just says ASN.1. It’s even older than PEM, dating from 1984. I have a bit of a relationship with ASN.1, where by “a bit of a relationship” I mean that back in the day I hated it intensely. It was in the era before Open Source, meaning that if you wanted to process ASN.1-encoded data, you had to buy an expensive, slow, buggy, parser with a hostile API. I guess I’m disrespecting the past. And it doesn’t help that I have a conflict of interest; when I was in the cabal that cooked up XML, ASN.1 was clearly seen as Part Of The Problem.

Be that as it may, when you have a PEM you have ASN.1 and if you’re a Gopher you have a perfectly-sane asn1 package that’s fast and free. Yay!

Five Oh Nine · It turns out you can’t parse or generate ASN.1 without having a schema. For public keys (and for certs and lots of other things), the schemas come from X.509 which is I guess relatively modern, having been launched in 1988. I know little about it, but have observed that among security geeks, discussion of X.509 (later standardized in the IETF as PKIX) is often accompanied by head-shaking and sad faces.

Key flavors · OK, I’ve de-PEM-ified the data and fed the results to the ASN.1 reader, and now I have what seems to me like a simple and essential question: What flavor of key does this represent? Maybe because I want to toss anything that isn’t ed25519. Maybe because I want to dispatch to the right crypto package. Maybe because I just want to freaking know what this bag of bits claims to be, a question the Internet should have an answer for.

Ladies, gentlemen, and others, when you ask the Internet “How do you determine what kind of public key this is?” you come up empty. Or at least I did. Eventually I stumbled across Signing JWTs with Go's crypto/ed25519 by Blain Smith, to whom I owe a debt of thanks, because it doesn’t assume you know what’s going on, it just shows you step-by-step how to unpack a PEM of an ASN.1 of an ed25519 public key.

It turns out that what you need to do is dig into that ASN.1 data and pull out an “Object Identifier”. At which point my face brightened up because do I ever like self-describing data. So I typed “ASN.1 Object Identifier” into Google and, well, unbrightening set in.

We must go deeper · At which point I wrote a little Go program whose inputs were a random RSA-key PEM I found somewhere and the ed25519 example from Blain Smith’s blog. I extracted an Object Identifier from each and discovered that Object Identifiers are arrays of numbers; for the RSA key, 1.2.840.113549.1.1.1, and for the elliptic-curve key, 1.3.101.112.

So I googled those strings and “1.3.101.112” led me to Appendix A of RFC8420, which has a nice simple definition and a note that the real normative reference is RFC8410, whose Section 3 discusses the question but does not actually include the actual strings encoded by the ASN.1.

“Oh,” I thought, “there must be a helpful registry of these values!” There sort of is, which I found by pasting the string values into Google: The informal but reasonably complete OID Repository. Which, frankly, doesn’t look like an interface you want to bet your future on. But did confirm that 1.2.840.113549.1.1.1 means RSA and 1.3.101.112 means ed25119.

So I guess I could write code based on those findings. But first, I decided to write this. Because the journey was not exactly confidence-inspiring. After all, public-key cryptography and its infrastructure (usually abbreviated “PKI”) is fucking foundational to fucking Internet Security, by which these days I mean banking and payments and privacy and generally truth.

And I’m left hoping the trail written in fairy dust on cobwebs that I just finished following is a best practice.

Then I woke up · At this point I talked to a friend who is crypto-savvy and asked “Would it be OK to require just ed25519, hardwire that into the protocol and refuse to consider anything else?” They said: “Yep, because first of all, these are short-lived signatures and second, if ed25519 fails, it’ll fail slowly and there’ll be time to migrate to something else.” Which considerably simplified the problem.

And by this time I understood that the conventional way to interchange these things is as base64’ed encoding of ASN.1 serializations of PKIX-specified data structures. It would have been nice if one of my initial searches had turned up a page saying just that. And it turns out that there are libraries to do these things, and that they’re built into modern programming languages so you don’t have to take dependencies.

G, meet J · So what I did was write two little libraries, one each in Go and Java, to translate public keys back and forth between native progam objects and base64, either encoded in the BEGIN/END cruft or not.

First let’s go from in-program data objects to base64 strings. In Go you do like so:

1. Use the x509 package’s MarshalPKIXPublicKey method to turn the key into the ASN.1-serialized bytes.

2. Base64 the bytes. You’re done!

In Java it’s like this:

1. Use the getEncoded method of PublicKey to get the ASN.1-serialization byte sequence.

2. Base64 the bytes, you’re done.

By the way, the base64 is only sixty characters long, gotta love that ed25519.

The next task is to read that textually-encoded public key into its native form as a programming-language object that you can use to verify signatures. In Go:

1. De-base64 the string into bytes.

2. Use the x509 package’s ParsePKIXPublicKey method to do the ASN.1 voodoo and extract the public key.

3. The key comes back as an interface{} so you have to do some typecasting to get the ed25519 key, but then you can just return it.

For Java:

1. Discard the BEGIN and END crap if it’s there.

2. Decode the remaining Base64 to yield a stream of bytes, which is the ASN.1 serialization of the data structure.

3. Now you need an ed25519-specialized KeyFactory instance, for which there’s a getInstance method.

4. Now you make a new X509EncodedKeySpec by feeding the ASN.1 serialized bytes to its constructor.

5. Now your KeyFactory can generate a public key if you feed the X509 thing to its generatePublic method. You’re done!

Other languages? · It might be of service to the community for someone else to bash equivalent tools out in Python and Rust and JS and whatever else, which would be a good thing because public keys are a good thing and ed25519 is a good thing.

References · Useful blogs & articles:

1. Java EdDSA (Ed25519 / Ed448) Example

2. How to Read PEM File to Get Public and Private Keys

3. Export & Import PEM files in Go

4. Signing JWTs with Go’s crypto/ed25519

5. Ed25519 in JDK 15, Parse public key from byte array and verify. I think this one is a hot mess, but it had high Google Juice, so consider this pointer cautionary. I ended up not having to do any of this stuff.

6. ASN.1 key structures in DER and PEM

RFCs:

1. 8419: Use of Edwards-Curve Digital Signature Algorithm (EdDSA) Signatures in the Cryptographic Message Syntax (CMS)

2. 8032: Edwards-Curve Digital Signature Algorithm (EdDSA)

3. 7468: Textual Encodings of PKIX, PKCS, and CMS Structures

4. 8410: Algorithm Identifiers for Ed25519, Ed448, X25519, and X448 for Use in the Internet X.509 Public Key Infrastructure

5. 8420: Using the Edwards-Curve Digital Signature Algorithm (EdDSA) in the Internet Key Exchange Protocol Version 2 (IKEv2)

---

---

Contributions