In general, I think we all agree that putting profiles on ldap is a good idea. However, in practice, I think we want to allow admins to make changes to local files as they see fit. Please see my in-line response below. On 07/02/2014 10:17 AM, Ade Lee wrote:
On Tue, 2014-07-01 at 12:42 +1000, Fraser Tweedale wrote:
On Mon, Jun 30, 2014 at 02:28:09PM -0500, Endi Sukma Dewata wrote:
Hi, I have some comments about the design: http://pki.fedoraproject.org/wiki/LDAP_Profile_Storage
I think all system/default profiles should remain file-based and all custom profiles should be LDAP-based. It will make a clean separation: system profiles are owned by us (Dogtag developers), custom profiles are owned by the admin.
The system profiles will be read-only. This way we will be able to update the system profiles without writing any upgrade scripts because the files will be updated automatically by RPM. Just one requirement, all server instances must be upgraded to the same version.
If the admin wants to change a system profile, they can clone it into a custom profile and make the changes there. The custom profiles cannot have the same names as the system profiles, so there's won't be any conflict/confusion, and no need to support a "restore" command. In general we won't need to write upgrade scripts for custom profiles except if we change the LDAP schema.
This was my original proposal, with the exception of making system/default profiles read-only and cloning them to make modifications. I think this is a good modification to that proposal.
Ade was in favour of moving to LDAP only, but perhaps this variation makes using a mix of LDAP and file-based profiles more palatable? It would be good to hear from more people on this part of the design - to see if there's a clear winner.
Perhaps we can view files v.s. ldap as "localized" v.s. "centralized" instead of "system" v.s. "customized". Again, the benefit of putting profiles in ldap is so that it can be configured once and utilized by many. My comments below are based on such view.
The problem with the above proposal is one of migration. Customers are used to both creating new profiles and modifying existing profiles to suit their needs. If we go along with this mechanism, we would have to tell these customers that they would need to rename their existing customized profiles (and update all the relevant clients etc.)
There is another problem, and that is that it is not clear that we want updates to the default profiles to be propagated to existing instances. I have looked at the profiles and there have been only a handful of changes over the last 7 years. Those changes include things like updating the default signing algorithms or the default validity. More likely than not, admins would prefer that we not change the behavior of profiles in existing instances underneath them.
The changes that I have found are all behavioral - and therefore things that admin can opt out of -- or would prefer to do on their own schedule. There have been no structural changes.
If there are structural changes, then we need to (and can) provide an upgrade script which would run with the automatic upgrade. An example of this would be a schema upgrade as we sort out how to represent profiles in LDAP.
So, I think we should do the following: 1. Continue to provide the system profiles in files. These files will be parsed and stored in LDAP when an instance is created. if the ldap copies are the "centralized" ones, then we don't want to override them at each installation of an instance. We could give admins an option to override what's stored in ldap during configuration. When a profile with the same id exists in both ldap and local system, then by default the one on ldap takes precedence. Although we could add "per profile" config param for each instance to change the view (remember the unix ns/yp config where you can set the order of the source? so if "file" is 1st in the order, then the local ones have priority, etc.) With this, a site that has carefully crafted their profiles and stored in ldap to share among all ca instances don't have to bother changing anything for each ca instance created (because they are default to the ldap). Upgrade will also not affect the ldap copies inadvertently.
2. All profiles for an instance should live in LDAP. This makes it simple - no need to check to see if a profile is in LDAP or files or both, and which has priority etc. Tools will be provided to manage/ create/delete profiles etc. It might be simple, but we need to think about what's the value of them existing in ldap and not being shared by others.
3. Updates to system profile files will not affect the existing LDAP profiles. We can provide update scripts or manual instructions for admins to run when they opt to do so. This will be for behavioral changes. so, in the view I specified above, update to the "local" profiles will only affect the instance where they exist, however, we can allow option for admins to "publish" to ldap.
4. Structural changes will be done using upgrade scripts using the database upgrade mechanism. This framework is something that we had planned to do already in 10.3. We already have a model on how to do this in our current upgrade framework.
About the schema itself, I suppose we want to have something that resembles the actual Profile data structure (see ProfileData Java class). There should be an LDAP attribute for each single-valued Java attribute (e.g. name, description, enabled, visible). This way the profile is more manageable and can be queried based on these attributes. For collection attributes (e.g. inputs, outputs, policySets) we can use child LDAP entries to represent them.
About the names, right now we use "profile" to refer to both cert profile in CA and token profile in TPS. We probably should use separate terms to distinguish them (e.g. CertProfile and TokenProfile). This applies to Java/Python class names, REST interface, CLI, LDAP schema, etc.
Good point. I've s/profile/certProfile/ in the schema.
So the LDAP entries for a profile may look like this:
dn: cn=MyUserCert,ou=Profiles,ou=CA,{suffix} objectClass: certProfile cn: MyUserCert displayName: Manual User Dual-Use Certificate Enrollment description: This certificate profile is for enrolling user certificates. visible: true enabled: true enabledBy: admin
dn: ou=Inputs,cn=MyUserCert,ou=Profiles,ou=CA,{suffix} objectClass: organizationalUnit ou: Inputs
dn: cn=i1,ou=Inputs,cn=MyUserCert,ou=Profiles,ou=CA,o=pki-tomcat objectClass: pkiCertProfileInput cn: i1 classId: keyGenInputImpl
Introducing this schema (and the more fine-grained functionality it would support) could happen later. I see the benefits but I think it's better to just get the profiles in LDAP as soon as possible, then look at how we can break them down to enable richer features.
I would prefer to see if we could design something more fine grained at first, rather than assuming an upgrade. If nothing else, you will need to separate out whether the profile is visible and enabled and by whom.
Also, if we retain file-based profiles as you suggest, whatever queries LDAP makes easier, we would still need to be able to do those same queries across the file-based profiles. So while file-based profiles remain, a finer-grained LDAP profile schema doesn't buy us anything in terms of reducing complexity or the amount of code to write.
About the REST interface & CLI, since this will be the primary way to edit profiles, we might want to have more granular commands to modify parts of the profile. Right now with ca-profile-mod command you need to send the entire profile in a file. It would be nice to be able to specify some parameters to change certain attributes only, or use separate commands to manage the inputs/outputs.
We'll also need an interface to find existing cert records that use a certain profile and bulk modify them to use a different profile. This will be useful when you create a clone to change the system profile.
Again, I agree with the benefits here, but feel that this may be out of scope for the initial changeover to LDAP profiles, given how much we need that for FreeIPA.
Agreed. I think this can be a later development.
I'm not sure if we should support 10.2 -> 10.3 cloning. When we release 10.3 the 10.2 will still be fairly new so it might be reasonable to require all clones to be upgraded. It will reduce the amount of testing requirement too.
I'll incorporate this comment (and all your others) in the design proposal; you should see the updated document soon.
Cheers,
Fraser
-- Endi S. Dewata
On 6/18/2014 2:44 AM, Fraser Tweedale wrote:
Hi all,
A requirement from the FreeIPA side is the ability to add and customise CA profiles. Dogtag's current profile creation behaviour writes the new profile to the filesystem beside the standard profiles (as well as making the appropriate update to the registry, etc.)
There does not seem to be a mechanism to distribute new/modified profiles to replicas - though perhaps I have missed something.
Because this behaviour is required, unless I have overlooked something or there is a better way (in which case please shout out), I think it makes sense to begin a design proposal for an LDAP-based profile store.
Finally, a brief mention of some tickets related to profile storage that could be good to tackle simultaneously should the proposed change go ahead:
- https://fedorahosted.org/pki/ticket/778 - https://fedorahosted.org/freeipa/ticket/4002
_______________________________________________ Pki-devel mailing list Pki-devel@redhat.com https://www.redhat.com/mailman/listinfo/pki-devel
_______________________________________________ Pki-devel mailing list Pki-devel@redhat.com https://www.redhat.com/mailman/listinfo/pki-devel