2:41 p.m.
Hi all,
Attached is a patch with some initial work for the Python client
library, specifically focused on the interactions between the DRM and
the python client, and the functionality exposed in the
KeyRequestResource and KeyResource REST APIs.
The easiest way to see this code in action is to run the main function
in the KRAClient module. I usually run this in Eclipse.
You will need to set up a few things first though:
1. Install a CA/KRA. It this is not on the default ports, you will
need to modify the connection information in KRAClient.__main__
2. The python code uses python-requests to talk to the server, and
requests uses openssl. That means you need to export your DRM admin
cert to a PEM file, so that it can be used for client auth. I did this
as follows:
openssl pkcs12 -in ~/.dogtag/pki-tomcat/ca_admin_cert.p12 -out temp4.pem -nodes
Without any changes, the code in KRAClient.__main__ assumes this file
will be in /tmp/temp4.pem.
3. We do some crypto functions using NSS commands (like generating a
symmetric key or wrapping using the transport cert). Therefore, we need
to create an NSS database and populate it with the transport cert. The
code expects it to be at /tmp/drmtest/certdb
I did this as follows:
mkdir /tmp/drmtest/certdb
certutil -N -d /tmp/drmtest/certdb
chmod +r /tmp/drmtest/certdb/*
certutil -L -d /var/lib/pki/pki-tomcat/alias/ -n "transportCert cert-pki-tomcat
KRA" -a > transport_cert.txt
certutil -A -d /tmp/drmtest/certdb/ -n "kra transport cert" -i
./transport_cert.txt -a -t "u,u,u"
4. Then just run kraclient.__main__ with no arguments.
The story is still somewhat incomplete, but its a lot of code - so I
want to start getting some eyes on this.
Here is what I plan to add in subsequent patches:
1. Extract the NSS functionality into a separate module nssutil.py, and
import that module only if python-nss is actually installed. Similarly,
only define those functions that need nss if python-nss is installed.
The idea here is that the client library does not necessarily require
NSS. If NSS (and python-nss) is not installed, then the client needs to
provide things like symmetric keys and wrapping etc.
This will almost certainly be the case for Barbican, where they will
generate their own keys, and do their own wrapping.
2. Complete the generate_pki_archive_options() function, and then test
archival.
3. Add logic to handle exceptions. This will almost certainly take the
form of decorator classes that handle the various exceptions and return
codes expected.
4. The new modules pass most of the PEP8 conventions as detected by
pylint. One thing that pylint does not like is having attribute names
that are camelCased. The reason those are there is because they
represent the actual names/attributes as passed over in JSON by the
server. We populate the relevant objects by doing something like this:
def from_dict(self, attr_list):
''' Return KeyInfo from JSON dict '''
for key in attr_list:
setattr(self, key, attr_list[key])
return self
I plan to add a small conversion function that convert camelCased
attribute names to camel_cased to conform with PEP8.
Please review,
Ade
11:42 a.m.
On 2/12/2014 2:41 PM, Ade Lee wrote:
Hi all,
Attached is a patch with some initial work for the Python client
library, specifically focused on the interactions between the DRM and
the python client, and the functionality exposed in the
KeyRequestResource and KeyResource REST APIs.
The easiest way to see this code in action is to run the main function
in the KRAClient module. I usually run this in Eclipse.
You will need to set up a few things first though:
1. Install a CA/KRA. It this is not on the default ports, you will
need to modify the connection information in KRAClient.__main__
2. The python code uses python-requests to talk to the server, and
requests uses openssl. That means you need to export your DRM admin
cert to a PEM file, so that it can be used for client auth. I did this
as follows:
openssl pkcs12 -in ~/.dogtag/pki-tomcat/ca_admin_cert.p12 -out temp4.pem -nodes
Without any changes, the code in KRAClient.__main__ assumes this file
will be in /tmp/temp4.pem.
3. We do some crypto functions using NSS commands (like generating a
symmetric key or wrapping using the transport cert). Therefore, we need
to create an NSS database and populate it with the transport cert. The
code expects it to be at /tmp/drmtest/certdb
I did this as follows:
mkdir /tmp/drmtest/certdb
certutil -N -d /tmp/drmtest/certdb
chmod +r /tmp/drmtest/certdb/*
certutil -L -d /var/lib/pki/pki-tomcat/alias/ -n "transportCert cert-pki-tomcat
KRA" -a > transport_cert.txt
certutil -A -d /tmp/drmtest/certdb/ -n "kra transport cert" -i
./transport_cert.txt -a -t "u,u,u"
4. Then just run kraclient.__main__ with no arguments.
The story is still somewhat incomplete, but its a lot of code - so I
want to start getting some eyes on this.
Here is what I plan to add in subsequent patches:
1. Extract the NSS functionality into a separate module nssutil.py, and
import that module only if python-nss is actually installed. Similarly,
only define those functions that need nss if python-nss is installed.
The idea here is that the client library does not necessarily require
NSS. If NSS (and python-nss) is not installed, then the client needs to
provide things like symmetric keys and wrapping etc.
This will almost certainly be the case for Barbican, where they will
generate their own keys, and do their own wrapping.
2. Complete the generate_pki_archive_options() function, and then test
archival.
3. Add logic to handle exceptions. This will almost certainly take the
form of decorator classes that handle the various exceptions and return
codes expected.
4. The new modules pass most of the PEP8 conventions as detected by
pylint. One thing that pylint does not like is having attribute names
that are camelCased. The reason those are there is because they
represent the actual names/attributes as passed over in JSON by the
server. We populate the relevant objects by doing something like this:
def from_dict(self, attr_list):
''' Return KeyInfo from JSON dict '''
for key in attr_list:
setattr(self, key, attr_list[key])
return self
I plan to add a small conversion function that convert camelCased
attribute names to camel_cased to conform with PEP8.
Please review,
Ade
Some comments:
1. The 'created' date in cert.py should be updated.
2. The CertId.__init__() doesn't parse the hex number.
3. To be consistent with the Java API, all from_dict() methods probably
should be a @staticmethod or @classmethod.
4. To be consistent the decode_from_json() can be called from_json(). Is
there any difference between this method and from_dict()?
5. The CertResource should be called CertClient for consistency with
Java API. Same thing with KeyResource and KeyRequestResource. The
*Resource classes in Java are internal implementation. The classes
actually exposed to the client applications are the *Client classes.
6. The following line is executed each time getCert() is called:
e.TYPES['CertData'] = CertData()
It should be executed just once. See system.py.
7. The TYPES list in #6 holds references to classes, not instances. This
is probably related to #3 as well.
8. The listCerts() should have passed the status as a query parameter.
9. As in #6, the following line in searchCerts() should be executed just
once:
e.TYPES['CertSearchRequest'] = CertSearchRequest
10. I have not tested the code, but searchCerts() doesn't seem to work
with the new Jackson-style JSON format. See system.py.
11. The get_transport_cert() is defined in CertResource. In the Java API
getTransportCert() is defined in SystemCertResource and used by
KRAClient. There's no SystemCertClient yet, but this method should be
defined there.
12. KeyRequestResponse.decode_from_json() does not populate keyData
attribute.
13. As in #6, in KeyResource.retrieve_key() the e.NOTYPES registration
should only be executed once.
14. The key.py's main program is a sample of a client application using
the Python library, but it does the e.NOTYPES registrations. Although
the registrations are only executed once, any client application should
not need to do these registrations. The Python library itself should do
the registrations automatically. See system.py.
15. It would be better to move the main programs in these Python library
into a separate test folder because they become functional tests and
contain hard-coded test configuration (e.g. hostname, port, path,
request ID, client ID). Please include the instruction to setup the test
environment as mentioned in the above email.
16. From client application's perspective, it would be better if the
kraclient.generate_sym_key() can take a list of usages, instead of
requiring the client app to join the usages manually.
17. Ideally the Key/KeyRequest-specific methods in KRAClient should be
moved into KeyClient/KeyRequestClient classes to avoid cluttering up the
KRAClient class. In the Java client library user-specific methods are
grouped into UserClient under KRAClient.
18. Instead of adding barbican_encode() which basically wraps
generate_sym_key(), the KeyRequestResponse could provide get_key_id()
which will get the key ID from the requestInfo so the client app can
call generate_sym_key() directly:
response = kraclient.generate_sym_key(...)
key_id = response.get_key_id()
19. Instead of adding barbican_decode(), the KRAClient could provide a
more generic method that can be used by other client applications.
Something like this:
class KRAClient(object):
def retrieve_key(self, key_id):
response = request_recovery(key_id)
approve_request(response.get_request_id())
transport_cert = get_transport_cert()
session_key = generate_session_key(...)
wrapped_session_key = wrap_session_key(...)
request = KeyRecoveryRequest(key_id, ...)
response = key_client.retrieve_key(request)
return unwrap_key(response.wrappedPrivateData, ...))
Client apps such as Barbican would be able to subclass and override the
generate_session_key() to provide its own key, possibly using other
crypto library.
--
Endi S. Dewata
11:36 a.m.
See inline below:
On Thu, 2014-02-13 at 11:42 -0600, Endi Sukma Dewata wrote:
On 2/12/2014 2:41 PM, Ade Lee wrote:
> Hi all,
>
> Attached is a patch with some initial work for the Python client
> library, specifically focused on the interactions between the DRM and
> the python client, and the functionality exposed in the
> KeyRequestResource and KeyResource REST APIs.
>
> The easiest way to see this code in action is to run the main function
> in the KRAClient module. I usually run this in Eclipse.
>
> You will need to set up a few things first though:
> 1. Install a CA/KRA. It this is not on the default ports, you will
> need to modify the connection information in KRAClient.__main__
>
> 2. The python code uses python-requests to talk to the server, and
> requests uses openssl. That means you need to export your DRM admin
> cert to a PEM file, so that it can be used for client auth. I did this
> as follows:
>
> openssl pkcs12 -in ~/.dogtag/pki-tomcat/ca_admin_cert.p12 -out temp4.pem -nodes
>
> Without any changes, the code in KRAClient.__main__ assumes this file
> will be in /tmp/temp4.pem.
>
> 3. We do some crypto functions using NSS commands (like generating a
> symmetric key or wrapping using the transport cert). Therefore, we need
> to create an NSS database and populate it with the transport cert. The
> code expects it to be at /tmp/drmtest/certdb
>
> I did this as follows:
> mkdir /tmp/drmtest/certdb
> certutil -N -d /tmp/drmtest/certdb
> chmod +r /tmp/drmtest/certdb/*
>
> certutil -L -d /var/lib/pki/pki-tomcat/alias/ -n "transportCert cert-pki-tomcat
KRA" -a > transport_cert.txt
> certutil -A -d /tmp/drmtest/certdb/ -n "kra transport cert" -i
./transport_cert.txt -a -t "u,u,u"
>
> 4. Then just run kraclient.__main__ with no arguments.
>
> The story is still somewhat incomplete, but its a lot of code - so I
> want to start getting some eyes on this.
>
> Here is what I plan to add in subsequent patches:
>
> 1. Extract the NSS functionality into a separate module nssutil.py, and
> import that module only if python-nss is actually installed. Similarly,
> only define those functions that need nss if python-nss is installed.
>
> The idea here is that the client library does not necessarily require
> NSS. If NSS (and python-nss) is not installed, then the client needs to
> provide things like symmetric keys and wrapping etc.
>
> This will almost certainly be the case for Barbican, where they will
> generate their own keys, and do their own wrapping.
>
> 2. Complete the generate_pki_archive_options() function, and then test
> archival.
>
> 3. Add logic to handle exceptions. This will almost certainly take the
> form of decorator classes that handle the various exceptions and return
> codes expected.
>
> 4. The new modules pass most of the PEP8 conventions as detected by
> pylint. One thing that pylint does not like is having attribute names
> that are camelCased. The reason those are there is because they
> represent the actual names/attributes as passed over in JSON by the
> server. We populate the relevant objects by doing something like this:
>
> def from_dict(self, attr_list):
> ''' Return KeyInfo from JSON dict '''
> for key in attr_list:
> setattr(self, key, attr_list[key])
> return self
>
> I plan to add a small conversion function that convert camelCased
> attribute names to camel_cased to conform with PEP8.
>
> Please review,
> Ade
Some comments:
1. The 'created' date in cert.py should be updated.
Done - also added a note that the implementation in cert.py is
incomplete and needs to be tested.
2. The CertId.__init__() doesn't parse the hex number.
Will be fixed in subsequent patch.
3. To be consistent with the Java API, all from_dict() methods
probably
should be a @staticmethod or @classmethod.
Done.
4. To be consistent the decode_from_json() can be called from_json().
Is
there any difference between this method and from_dict()?
Done.
5. The CertResource should be called CertClient for consistency with
Java API. Same thing with KeyResource and KeyRequestResource. The
*Resource classes in Java are internal implementation. The classes
actually exposed to the client applications are the *Client classes.
Done.
6. The following line is executed each time getCert() is called:
e.TYPES['CertData'] = CertData()
It should be executed just once. See system.py.
Done.
7. The TYPES list in #6 holds references to classes, not instances.
This
is probably related to #3 as well.
Done.
8. The listCerts() should have passed the status as a query
parameter.
Will be fixed in subsequent patch when certs are addressed.
9. As in #6, the following line in searchCerts() should be executed
just
once:
e.TYPES['CertSearchRequest'] = CertSearchRequest
Done.
10. I have not tested the code, but searchCerts() doesn't seem to
work
with the new Jackson-style JSON format. See system.py.
Will be fixed in subsequent patch when cert.py is fixed.
11. The get_transport_cert() is defined in CertResource. In the Java
API
getTransportCert() is defined in SystemCertResource and used by
KRAClient. There's no SystemCertClient yet, but this method should be
defined there.
Done
12. KeyRequestResponse.decode_from_json() does not populate keyData
attribute.
Fixed.
13. As in #6, in KeyResource.retrieve_key() the e.NOTYPES
registration
should only be executed once.
Fixed.
14. The key.py's main program is a sample of a client application
using
the Python library, but it does the e.NOTYPES registrations. Although
the registrations are only executed once, any client application should
not need to do these registrations. The Python library itself should do
the registrations automatically. See system.py.
Fixed.
15. It would be better to move the main programs in these Python
library
into a separate test folder because they become functional tests and
contain hard-coded test configuration (e.g. hostname, port, path,
request ID, client ID). Please include the instruction to setup the test
environment as mentioned in the above email.
Done - put into kra/functional/drmtest.py and added drmtest.readme.txt.
Also changed the name of drmclient.py and its readme to
drmclient_deprecated.py. We will remove these files as soon as we have
extracted all the useful code out of them.
16. From client application's perspective, it would be better if
the
kraclient.generate_sym_key() can take a list of usages, instead of
requiring the client app to join the usages manually.
Done
17. Ideally the Key/KeyRequest-specific methods in KRAClient should
be
moved into KeyClient/KeyRequestClient classes to avoid cluttering up the
KRAClient class. In the Java client library user-specific methods are
grouped into UserClient under KRAClient.
18. Instead of adding barbican_encode() which basically wraps
generate_sym_key(), the KeyRequestResponse could provide get_key_id()
which will get the key ID from the requestInfo so the client app can
call generate_sym_key() directly:
response = kraclient.generate_sym_key(...)
key_id = response.get_key_id()
Done
19. Instead of adding barbican_decode(), the KRAClient could provide
a
more generic method that can be used by other client applications.
Something like this:
class KRAClient(object):
def retrieve_key(self, key_id):
response = request_recovery(key_id)
approve_request(response.get_request_id())
transport_cert = get_transport_cert()
session_key = generate_session_key(...)
wrapped_session_key = wrap_session_key(...)
request = KeyRecoveryRequest(key_id, ...)
response = key_client.retrieve_key(request)
return unwrap_key(response.wrappedPrivateData, ...))
Client apps such as Barbican would be able to subclass and override the
generate_session_key() to provide its own key, possibly using other
crypto library.
OK - there have been a bunch of changes to address this. I have also
extracted the crypto functionality. See the commit message.
Attached is a new patch. Please apply this on top of the previous
patch.
12:02 p.m.
On 2/17/2014 11:36 AM, Ade Lee wrote:
> 4. To be consistent the decode_from_json() can be called
from_json(). Is
> there any difference between this method and from_dict()?
>
Done.
For consistency should they all be called from_json()?
> 16. From client application's perspective, it would be better
if the
> kraclient.generate_sym_key() can take a list of usages, instead of
> requiring the client app to join the usages manually.
Done
In key.py, the SymKeyGenerationRequest constructor takes a list of
key_usages string to be concatenated later, but in the main program it's
taking an already concatenated list of usages.
> 17. Ideally the Key/KeyRequest-specific methods in KRAClient
should be
> moved into KeyClient/KeyRequestClient classes to avoid cluttering up the
> KRAClient class. In the Java client library user-specific methods are
> grouped into UserClient under KRAClient.
The KRAClient still has key-specific operations such as retrieve_key(),
generate_sym_key(), etc. Should they be moved into KeyClient as well?
20. As mentioned on IRC, the drmtest setup requires manually importing
the transport cert. Also the KRAClient constructor takes the transport
cert nickname. Shouldn't the KRAClient get the transport cert directly
from the server and import it to the client database?
21. Also already discussed on IRC, the NSSCryptoUtil contains hardcoded
nonce_iv default value. It probably should be random.
22. The symmetric_unwrap() takes base-64-encoded data and nonce_iv
parameters. I think usually the caller would have to provide undecoded data.
--
Endi S. Dewata
12:26 p.m.
So we can see the status of things, pushed to master.
Will address issues below in a separate patch.
Also, will address jmagne suggestion to make the functions in CrytoUtil
more generic - in particular generate_symmetric_key
Ade
On Tue, 2014-02-18 at 12:02 -0600, Endi Sukma Dewata wrote:
On 2/17/2014 11:36 AM, Ade Lee wrote:
>> 4. To be consistent the decode_from_json() can be called from_json(). Is
>> there any difference between this method and from_dict()?
>>
> Done.
For consistency should they all be called from_json()?
>> 16. From client application's perspective, it would be better if the
>> kraclient.generate_sym_key() can take a list of usages, instead of
>> requiring the client app to join the usages manually.
>
> Done
In key.py, the SymKeyGenerationRequest constructor takes a list of
key_usages string to be concatenated later, but in the main program it's
taking an already concatenated list of usages.
>> 17. Ideally the Key/KeyRequest-specific methods in KRAClient should be
>> moved into KeyClient/KeyRequestClient classes to avoid cluttering up the
>> KRAClient class. In the Java client library user-specific methods are
>> grouped into UserClient under KRAClient.
The KRAClient still has key-specific operations such as retrieve_key(),
generate_sym_key(), etc. Should they be moved into KeyClient as well?
20. As mentioned on IRC, the drmtest setup requires manually importing
the transport cert. Also the KRAClient constructor takes the transport
cert nickname. Shouldn't the KRAClient get the transport cert directly
from the server and import it to the client database?
21. Also already discussed on IRC, the NSSCryptoUtil contains hardcoded
nonce_iv default value. It probably should be random.
22. The symmetric_unwrap() takes base-64-encoded data and nonce_iv
parameters. I think usually the caller would have to provide undecoded data.
3959
days inactive
3966
days old
4 comments
2 participants
participants (2)
-
Ade Lee -
Endi Sukma Dewata