----- Original Message ----- > Hi Endi, > > Just want to quickly discuss certificate IDs. > > Currently on ACMEBackend interface we have > > public BigInteger issueCertificate(String csr); > > I think this is a bit of a problem. e.g. Dogtag currently supports > multiple issuers (LWCAs). It is incidental that serial numbers do > not collide. This might not hold for other backends. Yet we need > the certificate ID to uniquely identify the certificate, so that we > can retrieve it, revoke it, etc. > > I suggest changing the return value to a string (which is how it > gets stored in the ACMEOrder object anyway). > > I'd further suggest that by convention, where possible, the string > be a representation of issuer+serial, which is a bit nicer for > humans looking at the stored objects than a base64url-encoded > big-endian bigint. > > What do you think? > > Cheers, > Fraser Hi Fraser, I agree there is a problem, but I'm not sure about using issuer+serial as certificate ID. What do we use as "issuer", is it the issuer DN or the authority ID?

Is issuer DN unique enough?

How do we join with the serial number? What format do we use for serial number?

What if we need to add another field in the future? It seems there's going to be many questions/issues with this solution.

How about this instead? 1. Change issueCertificate() to return the full cert chain. 2. Store the cert chain in a "certs" table in ACME database. 3. Autogenerate the cert ID for each cert record. 4. Store the account ID in the cert record. 5. Store the cert ID in the order record. So a copy of the cert will be stored in ACME database. The cert ID will be unique for that particular ACME server. We don't need to include the issuer DN/ID. The cert serial number will not matter either. We can also use the certs table to authorize revocation requests.

The cert ID is not meant to be human readable anyway (as shown in RFC 8555).

----- Original Message ----- > > > Currently on ACMEBackend interface we have > > > > > > public BigInteger issueCertificate(String csr); > > > > > > I think this is a bit of a problem. e.g. Dogtag currently supports > > > multiple issuers (LWCAs). It is incidental that serial numbers do > > > not collide. This might not hold for other backends. Yet we need > > > the certificate ID to uniquely identify the certificate, so that we > > > can retrieve it, revoke it, etc. > > > > > > I suggest changing the return value to a string (which is how it > > > gets stored in the ACMEOrder object anyway). > > > > > > I'd further suggest that by convention, where possible, the string > > > be a representation of issuer+serial, which is a bit nicer for > > > humans looking at the stored objects than a base64url-encoded > > > big-endian bigint. > > > > I agree there is a problem, but I'm not sure about using issuer+serial > > as certificate ID. What do we use as "issuer", is it the issuer DN > > or the authority ID? > > The issuer DN. > > > Is issuer DN unique enough? > > (issuer, serial) pair must be globally unique. > > > How do we join with the serial number? > > What format do we use for serial number? > > Doesn't really matter as long as it is unambiguous. For example, > serial as decimal number, followed by ';', followed by string > representation of Issuer DN. > > > What if we need to add another field in the future? It seems there's going > > to > > be many questions/issues with this solution. > > It is up to the ACMEBackend to produce a certificate ID. I'm simply > proposing this because a backend could contain multiple CAs with > separate serial number domains, hence deriving certificate ID from > serial number alone would not be unique. The idea of > (issuer,serial) pair is just a suggested convention. Some backends > e.g. might prefer UUIDs or whatever makes it easy to retrieve a > certificate/chain. > > > How about this instead? > > > > 1. Change issueCertificate() to return the full cert chain. > > 2. Store the cert chain in a "certs" table in ACME database. > > 3. Autogenerate the cert ID for each cert record. > > 4. Store the account ID in the cert record. > > 5. Store the cert ID in the order record. > > > > So a copy of the cert will be stored in ACME database. The cert > > ID will be unique for that particular ACME server. We don't need > > to include the issuer DN/ID. The cert serial number will not matter > > either. We can also use the certs table to authorize revocation > > requests. > > I thought about this a little while back, and I prefer the current > approach of storing an identifier as a "handle" to retrieve the cert > from the backend. Cert objects will increase the size of > records/objects significantly. For the LDAP backend it could be a > problem, both for disk usage but in particular for replication. > > I'm OK with the idea of *optional* certificate/chain storage in the > ACME database, e.g. for backends that do not support retrieval. But > I don't think we need that with the current backends (certainly not > with the PKIBackend). > > > The cert ID is not meant to be human readable anyway (as > > shown in RFC 8555). > > But it doesn't matter if it is human readable. Either way, storing > only the serial number is not enough IMO. Let me backtrack a little bit. Is there a plan to modify Dogtag to eventually support different serial number domains? If not, this is not an issue for Dogtag.

If there is such plan, will the issuer DN be unique across LWCAs? If issuer DN will be unique, it's something to consider. If not, (issuer DN, serial number) will not be unique either, so we need to use something else such as authority ID. Or is there another backend with multiple issuers that we want to support in the future? The cert ID will have to be something like (issuer ID, serial number) where the issuer ID is unique for the backend. If the issuer DN is unique, it can be used as the issuer ID. Otherwise, it needs to be a backend-specific unique ID similar to authority ID in Dogtag.

We need to consider these possibilities before changing the cert ID. On the other hand, I'm still not sure it's actually necessary to include these information into cert ID. Let's look at the code. For cert enrollment (ACMEFinalizeOrderService) we convert the serial number that we get from ACMEBackend into cert ID: BigInteger serialNumber = backend.issueCertificate(csr); String certID = Base64.encodeBase64URLSafeString(serialNumber.toByteArray()); We can change it so ACMEBackend can generate the cert ID like this: String certID = backend.issueCertificate(csr);

If the cert ID is (issuer DN, serial number), we can generate the cert ID from the new cert. But does the backend return the new cert or just the serial number?

If the serial number is not unique, the backend might need to be changed to return the cert itself so we can get the issuer DN. If the cert ID is (authority ID, serial number), how do we get the authority ID since it's not included in the cert? The backend might need to be changed to return the authority ID along with the new cert, or to provide a way to look up the authority ID using a cert.

For cert retrieval (ACMECertificateService) we're passing the cert ID to ACMEBackend: String certChain = backend.getCertificateChain(certID); The ACMEBackend can extract the issuer DN or authority ID from the cert ID so it can retrieve the cert from the backend again. Since we get the cert during enrollment anyway, we can actually store it into ACME database like this: String certChain = backend.issueCertificate(csr); String certID = database.addCert(certChain, orderID, accountID, expirationTime); Later we can simply retrieve it from the database instead of calling the backend again: String certChain = database.getCertificateChain(certID);

Here the cert ID can simply be a unique ID generated by the database. Unlike earlier, the backend doesn't need to know about cert ID at all. For cert revocation (ACMERevokeCertificateService) the client will only provide the cert binaries. It doesn't provide the cert ID.

Currently the ACMEEngine.validateRevocation() will generate the cert ID from the serial number so it can find the order that generated the cert (so we can authorize the account): String certID = Base64.encodeBase64URLSafeString(serialNumber.toByteArray()); ACMEOrder order = database.getOrderByCertificate(certID); We can changed it so ACMEBackend can generate the cert ID like this: String certID = backend.getCertID(certBytes); ACMEOrder order = database.getOrderByCertificate(certID); If the cert ID is (issuer DN, serial number), we can generate the cert ID from the provided cert binaries. But if the cert ID is (authority ID, serial number), how do we get the authority ID? Do we call the lookup operation above again to get the authority ID? Instead of that we can do this: String certID = database.getCertID(certBytes); ACMEOrder order = database.getOrderByCertificate(certID); which doesn't involve the backend at all. So backend-issued cert ID might work if we use a backend that already provides the required functionality above. Otherwise we may need to modify the backend, which is not always an option. The database-issued cert ID is a solution that doesn't require modifications to the backend, so I think it should be the default option. The certs stored in ACME database should be considered a cache. The server can purge it so it doesn't grow too large if that's a concern. Note that regardless of cert ID, the above revocation mechanism relies on order, authorization, or cert records in ACME database, which may not be available depending on the server's purging policy. If someone needs to have a reliable revocation mechanism they need to revoke using the private key.

----- Original Message ----- > > Let me backtrack a little bit. Is there a plan to modify Dogtag to > > eventually support different serial number domains? If not, this is > > not an issue for Dogtag. > > There is no plan to do so. It is not an issue for Dogtag. But > still, I feel basing certificate ID on only serial number is not a > robust approach in general. > > > If there is such plan, will the issuer DN > > be unique across LWCAs? If issuer DN will be unique, it's something > > to consider. If not, (issuer DN, serial number) will not be unique > > either, so we need to use something else such as authority ID. > > > > Or is there another backend with multiple issuers that we want to > > support in the future? The cert ID will have to be something like > > (issuer ID, serial number) where the issuer ID is unique for the > > backend. If the issuer DN is unique, it can be used as the issuer > > ID. Otherwise, it needs to be a backend-specific unique ID similar > > to authority ID in Dogtag. > > All certs must have unique (issuer,serial). This is implied by the > requirement that all certs from a given issuer must have different > serial numbers. > > > We need to consider these possibilities before changing the cert ID. > > On the other hand, I'm still not sure it's actually necessary to > > include these information into cert ID. > > > > Let's look at the code. For cert enrollment (ACMEFinalizeOrderService) > > we convert the serial number that we get from ACMEBackend into cert ID: > > > > BigInteger serialNumber = backend.issueCertificate(csr); > > String certID = > > Base64.encodeBase64URLSafeString(serialNumber.toByteArray()); > > > > We can change it so ACMEBackend can generate the cert ID like this: > > > > String certID = backend.issueCertificate(csr); > > > > I strongly agree with pushing the certID generation into the > ACMEBackend. Stepping away from the whole (issuer,serial) > discussion, say (for example) the only "handle" a backend has for > accessing a cert is a UUID. Then storing the serial number is no > good - you cannot derive the UUID handle from the serial number. > > So the backend must generate the (String) certID that is appropriate > for that backend. > > > If the cert ID is (issuer DN, serial number), we can generate the > > cert ID from the new cert. But does the backend return the new cert > > or just the serial number? > > > Yeah, good question; of course you must be able to retrieve the > cert (and therefore you can learn the Issuer DN) but this could mean > another round-trip to Dogtag. Which is the next thing you said :) > > > If the serial number is not unique, the > > backend might need to be changed to return the cert itself so we > > can get the issuer DN. > > > > If the cert ID is (authority ID, serial number), how do we get the > > authority ID since it's not included in the cert? The backend might > > need to be changed to return the authority ID along with the new > > cert, or to provide a way to look up the authority ID using a cert. > > > I am not suggesting to use the authority ID. But FWIW Dogtag does > enforce that Issuer DN <-> Authority ID is a bijection. > > > > > For cert retrieval (ACMECertificateService) we're passing the cert ID > > to ACMEBackend: > > > > String certChain = backend.getCertificateChain(certID); > > > > The ACMEBackend can extract the issuer DN or authority ID from the > > cert ID so it can retrieve the cert from the backend again. > > > > Since we get the cert during enrollment anyway, we can actually store > > it into ACME database like this: > > > > String certChain = backend.issueCertificate(csr); > > String certID = database.addCert(certChain, orderID, accountID, > > expirationTime); > > > > Later we can simply retrieve it from the database instead of calling > > the backend again: > > > > String certChain = database.getCertificateChain(certID); > > > As I said in previous email, I am opposed to storing the cert > (chain) in the ACME database. If some backend requires it e.g. > because the backend itself does not store the cert, then it can be > optional. But we do not need that now. > > > Here the cert ID can simply be a unique ID generated by the database. > > Unlike earlier, the backend doesn't need to know about cert ID at all. > > > > For cert revocation (ACMERevokeCertificateService) the client will > > only provide the cert binaries. It doesn't provide the cert ID. > > > And the ACMEBackend implementation receives the cert, and must work > out what to do with it. How it tells the backend system to revoke > the certificate, and whether that process even involves a string > CertID handle, or just a serial number, or the (issuer,serial) pair, > or whatever, depends on the backend system. But I think that the > current interface: > > public void revokeCert(ACMERevocation revocation) ... > > ... is suitable. > > > Currently the ACMEEngine.validateRevocation() will generate the > > cert ID from the serial number so it can find the order that > > generated the cert (so we can authorize the account): > > > > String certID = > > Base64.encodeBase64URLSafeString(serialNumber.toByteArray()); > > ACMEOrder order = database.getOrderByCertificate(certID); > > > > We can changed it so ACMEBackend can generate the cert ID like this: > > > > String certID = backend.getCertID(certBytes); > > ACMEOrder order = database.getOrderByCertificate(certID); > > > > If the cert ID is (issuer DN, serial number), we can generate the > > cert ID from the provided cert binaries. But if the cert ID is > > (authority ID, serial number), how do we get the authority ID? Do > > we call the lookup operation above again to get the authority ID? > > > > Instead of that we can do this: > > > > String certID = database.getCertID(certBytes); > > ACMEOrder order = database.getOrderByCertificate(certID); > > > > which doesn't involve the backend at all. > > > > So backend-issued cert ID might work if we use a backend that > > already provides the required functionality above. Otherwise we > > may need to modify the backend, which is not always an option. > > > > The database-issued cert ID is a solution that doesn't require > > modifications to the backend, so I think it should be the default > > option. The certs stored in ACME database should be considered > > a cache. The server can purge it so it doesn't grow too large if > > that's a concern. > > > > Note that regardless of cert ID, the above revocation mechanism > > relies on order, authorization, or cert records in ACME database, > > which may not be available depending on the server's purging > > policy. If someone needs to have a reliable revocation mechanism > > they need to revoke using the private key. > > > Let us put aside the discussion about whether for the PKIBackend we > use only (serial) as certID, or (issuer,serial) pair. I think we > *should* switch to something derived from (issuer,serial), but we do > not *need* to. So we can leave that discussion for now. > > The main change we need is for ACMEBackend.issuerCertificate to > return String certID, i.e.: > > public String issuerCertificate(String csr) ... > > because BigInteger (i.e. serial) may not be an appropriate "handle" > for all backends. Hence we should require each ACMEBackend > implementation to produce the appropriate certIDs. > > Do you agree? > > Cheers, > Fraser > Hi Fraser, Please take a look at this PR: https://github.com/dogtagpki/pki/pull/350 So the PKIBackend will continue to work like before, but these changes should allow us to support: - both single-authority and multi-authority backends - both one-step and two-step enrollments One thing though, I think we discussed before about using UUID to generate the serial number which has a very low chance of collision. If a backend uses UUID for all of its certificate authorities, can the serial number by itself be considered unique?