Hello All,

I thing something is wrong with dogtag packages included in the new CentOS 7 release. Once CentOS 7.4.1708 arrived in the distro repositories we got our systems updated. But when we rebooted the PKI infrastructure server nodes we realized that pki-tomcat somehow cannot load the certificates and some of the other settings.

We started analyzing the problem by presuming that we made some mistake in the configuration but when we tried to create from scratch CA subsystem on freshly installed system (CentOS 7.4.1708, 389 server, and the pki-* packages installed), we failed:

Subsystem (CA/KRA/OCSP/TKS/TPS) [CA]:

Tomcat:
  Instance [pki-tomcat]:
  HTTP port [8080]:
  Secure HTTP port [8443]:
  AJP port [8009]:
  Management port [8005]:

Administrator:
  Username [caadmin]:
  Password:
  Verify password:
  Import certificate (Yes/No) [N]?
  Export certificate to [/root/.dogtag/pki-tomcat/ca_admin.cert]:

Directory Server:
  Hostname [ds.example.com]:
  Use a secure LDAPS connection (Yes/No/Quit) [N]?
  LDAP Port [389]:
  Bind DN [cn=Directory Manager]:
  Password:
  Base DN [o=pki-tomcat-CA]:

Security Domain:
  Name [example.com Security Domain]:

Begin installation (Yes/No/Quit)? Yes

Log file: /var/log/pki/pki-ca-spawn.20170925074602.log
Installing CA into /var/lib/pki/pki-tomcat.
Storing deployment configuration into /etc/sysconfig/pki/tomcat/pki-tomcat/ca/deployment.cfg.
pkispawn    : ERROR    ....... server failed to restart

Installation failed: server failed to restart

Note that it is a fresh installation. No any customization. 389 server is running and it got tested before starting with the CA subsystem installation procedure. All DNS records matching the machine address are available.

I checked the spawn log file (/var/log/pki/pki-ca-spawn.20170925074602.log). Most of the entries there seem absolutely fine. The only records that show some problems are:

2017-09-25 06:32:28 pkispawn    : INFO     ....... executing 'systemctl daemon-reload'
2017-09-25 06:32:28 pkispawn    : INFO     ....... executing 'systemctl start pki-tomcatd@pki-tomcat.service'
2017-09-25 06:32:29 pkispawn    : DEBUG    ........... No connection - server may still be down
2017-09-25 06:32:29 pkispawn    : DEBUG    ........... No connection - exception thrown: 404 Client Error: Not Found
2017-09-25 06:32:30 pkispawn    : DEBUG    ........... No connection - server may still be down
2017-09-25 06:32:30 pkispawn    : DEBUG    ........... No connection - exception thrown: 404 Client Error: Not Found
2017-09-25 06:32:31 pkispawn    : DEBUG    ........... No connection - server may still be down
2017-09-25 06:32:31 pkispawn    : DEBUG    ........... No connection - exception thrown: 404 Client Error: Not Found
...

2017-09-25 06:33:30 pkispawn    : ERROR    ....... server failed to restart
2017-09-25 06:33:30 pkispawn    : DEBUG    ....... Error Type: Exception
2017-09-25 06:33:30 pkispawn    : DEBUG    ....... Error Message: server failed to restart
2017-09-25 06:33:30 pkispawn    : DEBUG    .......   File "/sbin/pkispawn", line 533, in main

Since that piece of information is not very particular on what exactly happens, I checked the debug log in /var/log/pki/pki-tomcat/ca/debug and found these pieces of suspicious info:

[25/Sep/2017:07:46:56][localhost-startStop-1]: CMSEngine: restart at autoShutdown? false
[25/Sep/2017:07:46:56][localhost-startStop-1]: CMSEngine: autoShutdown crumb file path? /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
[25/Sep/2017:07:46:56][localhost-startStop-1]: CMSEngine: about to look for cert for auto-shutdown support:auditSigningCert cert-pki-tomcat
[25/Sep/2017:07:46:56][localhost-startStop-1]: CMSEngine: cert not found:auditSigningCert cert-pki-tomcat
[25/Sep/2017:07:46:56][localhost-startStop-1]: CMSEngine: Exception:org.mozilla.jss.crypto.ObjectNotFoundException

...

Property internaldb.ldapconn.port missing value

...

I know that "Property internaldb.ldapconn.port missing value" error is explained here http://pki.fedoraproject.org/wiki/Troubleshooting as something that could be ignored, but the spawn process does not create any new LDAP data base (it is supposed to create o=pki-tomcat-CA). Moreover, except from the cn=Directory Manager dn password validation, there is no even a single attempt to connect to the 389 directory server running on the same machine.

Anyone with experience in that?

Best,

Veselin