Florian, I don't have a whole lot of time at this point, but I could offer a little info from my experience with LunaSA in case of private key unwrapping (your case seems to be sym key unwrapping though). For LunaSA private key unwrapping the isSensitive flag needs to be false and isExtractable needs to be true, and the key needs to be "temporary". In CRSEnrollment.java where it fails to decrypt PKCS10 blob, kw.unwrapSymmetric() is being called to decrypt the symmetric key and failed. My guess is that the usage or flags (internally CK_FLAGS) might need to be manipulated similar to what I've done with private key unwrapping in JSS PK11KeyWrapper.c:Java_org_mozilla_jss_pkcs11_PK11KeyWrapper_nativeUnwrapPrivWithSym() (search for "isLunasa"). Of course without spending actual time investigating, this is just one guess. Another guess is that in the following call, "keylength" needs to e specified: (given the symptom you reported, this might be a likely case) sk = kw.unwrapSymmetric(req.getWrappedKey(), skt, SymmetricKey.Usage.DECRYPT, 0); // keylength is ignored If you are a customer, feel free to escalate the BZ. And if you are not, you could try to vote in on https://pagure.io/dogtagpki/issue/442. regards, Christina On 10/04/2017 05:40 AM, Supper Florian 6342 sIT wrote:

Dear dogtag team,

We’ve been using dogtag CAs (the RHEL packages in server-rpm repo) together with Safenet HSMs for some years for handling SCEP requests.

We’re running into an issue again which we also had in the past: when using the HSM (a requirement here), only SCEP requests using DES for the encryption

can be decoded. When DES3 is used, dogtag throws an error with “could not unwrap PKCS10 blob”. With no HSM, both algorithms work.

However, the DES3 requests themselves are OK: we can unpack the inner pkcs#7, and decrypt the payload using ‘cmsutil’ (pointed at the nss db of the CA instance)

and read the pkcs#10 request within. So the HSM itself has no problem decrypting.

We also encountered this issue in the past with RHEL6 / DogTag 9, and it is still present with RHEL7 / Dogtag 10. At that time, we were able to configure the clients

to use DES to avoid the issue, but we can’t always dictate which algorithm the clients use,

and DES is nevertheless very weak.

It may still be related the old BZ: https://bugzilla.redhat.com/show_bug.cgi?id=825887 and be an issue with the FIPS-2 mode (which we are using)

It appears to be an issue with Dogtag. If someone has a suggestion or idea, we would appreciate hearing it.

Thanks in advances

BR

Florian

Below you can find all needed parameters and config which we used.

- CA is an subca - (But dont matter, because the same issue occures also on an root ca)

- SCEP enrollment works with DES encryption (HSM attached)

- SCEP enrollment with DES3 works when __NO__ hsm is used

- SCEP requests (DES + 3DES) can be decoded when using cmsutil direct against the HSM libary. (cmsutil -d /var/lib/pki/pkit04/alias -D -i inner_pkcs7_request.p7 -o request_des3.der )

- HSM client logs can be provided if needed

- For us is seems to be a problem in dogtag.

#Installed packages

pki-base-java-10.3.3-19.el7_3.noarch

pki-ca-10.3.3-19.el7_3.noarch

pki-base-10.3.3-19.el7_3.noarch

pki-tools-10.3.3-19.el7_3.x86_64

pki-kra-10.3.3-19.el7_3.noarch

pki-symkey-10.3.3-19.el7_3.x86_64

pki-core-debuginfo-10.3.3-19.el7_3.x86_64

pki-server-10.3.3-19.el7_3.noarch

pki-javadoc-10.3.3-19.el7_3.noarch

#OS

Red Hat Enterprise Linux Server release 7.3 (Maipo)

#Java

java-1.8.0-openjdk-1.8.0.131-3.b12.el7_3.x86_64

#SafeNet LunaClient

Version : 5.4.1

Release : 2

#SCEP enrollment profile:

caRouterCert.cfg (default)

#SCEP config in CS.cfg

ca.scep.allowedEncryptionAlgorithms=DES3,DES

ca.scep.allowedHashAlgorithms=SHA1,SHA256,SHA512

ca.scep.enable=true

ca.scep.encryptionAlgorithm=DES3

ca.scep.hashAlgorithm=SHA1

ca.scep.nonceSizeLimit=16

#SSECP call

./sscep enroll -u http : / / pkit04 . eb . lan . at : 8080 / ca / cgi-bin / pkiclient . exe -c pkit04-ca.crt -k local.key -r local.csr -l cert.crt -S sha1 -E 3des

#debug log

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:263:init() CRSEnrollment: init: SCEP support is enabled.

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:264:init() CRSEnrollment: init: SCEP nickname: pkit04:caSigningCert cert-pkit04 CA

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:265:init() CRSEnrollment: init: CA nickname: pkit04:caSigningCert cert-pkit04 CA

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:266:init() CRSEnrollment: init: Token name: pkit04

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:267:init() CRSEnrollment: init: Is SCEP using CA keys: true

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:268:init() CRSEnrollment: init: mNonceSizeLimit: 16

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:269:init() CRSEnrollment: init: mHashAlgorithm: SHA1

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:270:init() CRSEnrollment: init: mHashAlgorithmList: SHA1,SHA256,SHA512

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:273:init() CRSEnrollment: init: mAllowedHashAlgorithm[0]=SHA1

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:273:init() CRSEnrollment: init: mAllowedHashAlgorithm[1]=SHA256

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:273:init() CRSEnrollment: init: mAllowedHashAlgorithm[2]=SHA512

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:275:init() CRSEnrollment: init: mEncryptionAlgorithm: DES3

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:276:init() CRSEnrollment: init: mEncryptionAlgorithmList: DES3,DES

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:279:init() CRSEnrollment: init: mAllowedEncryptionAlgorithm[0]=DES3

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:279:init() CRSEnrollment: init: mAllowedEncryptionAlgorithm[1]=DES

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:285:init() CRSEnrollment: init: mProfileId=caRouterCert

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:349:service() operation=PKIOperation

[03/Oct/2017:07:35:52][http-bio-8080-exec-1]: CRSEnrollment.java:351:service() message=MIIKywYJKoZIhvcNAQcCoIIKvDCCCrgCAQExCzAJBgUrDgMCGgUAMIIFnwYJKoZI

-...snip..

t3fqG6FkBAh3L1saONZJ0pfzOnnY5CZ4aJuf5ql3XA==

[03/Oct/2017:07:35:53][http-bio-8080-exec-1]: CRSEnrollment.java:920:handlePKIOperation() Processing PKCSReq

[03/Oct/2017:07:35:53][http-bio-8080-exec-1]: LdapBoundConnFactory.java:324:getConn() In LdapBoundConnFactory::getConn()

[03/Oct/2017:07:35:53][http-bio-8080-exec-1]: LdapBoundConnFactory.java:326:getConn() masterConn is connected: true

[03/Oct/2017:07:35:53][http-bio-8080-exec-1]: LdapBoundConnFactory.java:368:getConn() getConn: conn is connected true

[03/Oct/2017:07:35:53][http-bio-8080-exec-1]: LdapBoundConnFactory.java:398:getConn() getConn: mNumConns now 5

[03/Oct/2017:07:35:53][http-bio-8080-exec-1]: LdapBoundConnFactory.java:444:returnConn() returnConn: mNumConns now 6

[03/Oct/2017:07:35:53][http-bio-8080-exec-1]: CRSEnrollment.java:1164:unwrapPKCS10() failed to unwrap PKCS10 org.mozilla.jss.crypto.SymmetricKey$NotExtractableException

[03/Oct/2017:07:35:53][http-bio-8080-exec-1]: CRSEnrollment.java:385:service() ServletException javax.servlet.ServletException: Couldn't handle CEP request (PKCSReq) - Could not unwrap PKCS10 blob: null

#Error in localhost_access log

10.10.10.10 - - [02/Oct/2017:11:09:27 +0200] "GET / ca / cgi-bin / pkiclient . exe ? operation = PKIOperation & message = MIIKzgYJKoZIhvcNAQcCoIIKvz...snip.. HTTP/1.0" 500 3071

#Error in localhost log

SEVERE: Servlet.service() for servlet [caSCEP] in context with path [/ca] threw exception [Couldn't handle CEP request (PKCSReq) - Could not unwrap PKCS10 blob: null] with root cause

javax.servlet.ServletException: Couldn't handle CEP request (PKCSReq) - Could not unwrap PKCS10 blob: null

at com.netscape.cms.servlet.cert.scep.CRSEnrollment.service(CRSEnrollment.java:386)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)

at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)

at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)

at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:297)

at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)

at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)

at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)

at java.security.AccessController.doPrivileged(Native Method)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)

at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)

at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)

at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)

at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:260)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)

at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:55)

at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:191)

at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:187)

at java.security.AccessController.doPrivileged(Native Method)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:186)

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)

at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)

at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)

at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)

at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)

at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)

at java.lang.Thread.run(Thread.java:748)

_______________________________________________ Pki-users mailing list Pki-users@redhat.com https://www.redhat.com/mailman/listinfo/pki-users