[PATCH 022] Modernize for Python 3 support
One big step for Python 3 support. With this patch and manual installation of pyldap and nss, pki can be imported under Python 3.4. The code doesn't run properly yet, though. Run python-modernize fixers: libmodernize.fixes.fix_import libmodernize.fixes.fix_print libmodernize.fixes.fix_input_six libmodernize.fixes.fix_xrange_six lib2to3.fixes.fix_execfile libmodernize.fixes.fix_metaclass libmodernize.fixes.fix_unicode_type libmodernize.fixes.fix_dict_six Add more from __future__ import absolute_import Manually fix import problems with either six.moves or manual try/except ImportError blocks. Remove Exception.message attribute access. Add a workaround for policycoreutils-python3. It lacks sepolgen on Fedora 22. Test pep8 and lint on Python 3 with tox.
And now with attachment. I also attached a 2nd patch that fixes the one unit test under Python 3. Christian
Went through the 15 patches. Looks pretty straightforward and acks for the first 12 and the last patch, which the following comment: 1. Please add comments to describe some of the changes being made so that we have a record for later on. This is particularly true for things like adding absolute_import (which doesn't have a reference in the code and someone might be tempted to remove in future). On patch 13 - Manually fix import problems and access to Exception.message, I have the following question: 2. In base/server/python/pki/server/deployment/pkiparser.py and in base/common/python/pki/key.py, you solve the import problem for urllib by trying to import the Python2 first and then Python 3 (or visa versa). Why doesn't importing the relevant six module work instead? And on patch 14 (sepolgen): 3. Could you explain what the effect of your sepolgen change is? Please co-ordinate checkins with Matt. Ade On Fri, 2015-08-14 at 17:03 +0200, Christian Heimes wrote:
And now with attachment.
I also attached a 2nd patch that fixes the one unit test under Python 3.
Christian
_______________________________________________ Pki-devel mailing list Pki-devel@redhat.com https://www.redhat.com/mailman/listinfo/pki-devel
Ade, thanks for your review of my gigantic patch set. On 2015-08-14 21:21, Ade Lee wrote:
Went through the 15 patches. Looks pretty straightforward and acks for the first 12 and the last patch, which the following comment:
1. Please add comments to describe some of the changes being made so that we have a record for later on.
This is particularly true for things like adding absolute_import (which doesn't have a reference in the code and someone might be tempted to remove in future).
Your and Endi's review have shown me that several small patches are easier to understand and better to read than a large, squashed patch bomb. I will post a series of smaller patches later. Each patch will a long description
On patch 13 - Manually fix import problems and access to Exception.message, I have the following question:
2. In base/server/python/pki/server/deployment/pkiparser.py and in base/common/python/pki/key.py, you solve the import problem for urllib by trying to import the Python2 first and then Python 3 (or visa versa). Why doesn't importing the relevant six module work instead?
I don't mind to use six.moves here, too. I guess I'm used to manual imports with try/except ImportError. I've to disable pylint warnings anyway as pylint doesn't understand six.moves magic.
And on patch 14 (sepolgen):
3. Could you explain what the effect of your sepolgen change is?
It's a temporary workaround for an incomplete Python 3 package. It disables the SELinux policy generator in pkispawn until the package is fixed. As a consequence you have to disable SELinux enforcement -- but only for pkispawn and pkidestroy on Python 3. It doesn't affect the regular Python 2.7 code. Christian
On 8/14/2015 10:03 AM, Christian Heimes wrote:
And now with attachment.
I also attached a 2nd patch that fixes the one unit test under Python 3.
Christian
Some comments: 1. Build works on F22, but fails on F21: ************* Module pylint-build-scan I: 33, 0: Locally disabling import-error (F0401) (locally-disabled) ************* Module pki I: 33, 0: Locally disabling redefined-builtin (W0622) (locally-disabled) I:127, 0: Locally disabling too-few-public-methods (R0903) (locally-disabled) I:141, 0: Locally disabling too-few-public-methods (R0903) (locally-disabled) I:149, 0: Locally disabling invalid-name (C0103) (locally-disabled) I:161, 0: Locally disabling invalid-name (C0103) (locally-disabled) F: 33, 0: Unable to import 'six.moves' (import-error) ************* Module pki.encoder I: 36, 0: Locally disabling method-hidden (E0202) (locally-disabled) ************* Module pki.profile I:1361, 0: Locally disabling broad-except (W0703) (locally-disabled) I:1398, 0: Locally disabling broad-except (W0703) (locally-disabled) ************* Module pki.key I: 34, 0: Locally disabling no-name-in-module (E0611) (locally-disabled) I: 36, 0: Locally disabling no-name-in-module (E0611) (locally-disabled) I: 43, 0: Locally disabling too-few-public-methods (R0903) (locally-disabled) I: 54, 0: Locally disabling too-few-public-methods (R0903) (locally-disabled) I: 66, 0: Locally disabling invalid-name (C0103) (locally-disabled) I:118, 0: Locally disabling invalid-name (C0103) (locally-disabled) I:150, 0: Locally disabling too-few-public-methods (R0903) (locally-disabled) I:186, 0: Locally disabling invalid-name (C0103) (locally-disabled) I:222, 0: Locally disabling too-few-public-methods (R0903) (locally-disabled) I:258, 0: Locally disabling invalid-name (C0103) (locally-disabled) F: 34, 4: Unable to import 'urllib.parse' (import-error) ************* Module pki.upgrade I:545, 0: Locally disabling exec-used (W0122) (locally-disabled) I:632, 0: Locally disabling broad-except (W0703) (locally-disabled) I:699, 0: Locally disabling broad-except (W0703) (locally-disabled) ************* Module pki.server.cli.migrate I:166, 0: Locally disabling protected-access (W0212) (locally-disabled) I:279, 0: Locally disabling protected-access (W0212) (locally-disabled) ************* Module pki.server.cli.instance I:327, 0: Locally disabling no-member (E1101) (locally-disabled) I:327, 0: Locally disabling maybe-no-member (E1103) (locally-disabled) I:383, 0: Locally disabling no-member (E1101) (locally-disabled) I:383, 0: Locally disabling maybe-no-member (E1103) (locally-disabled) I:439, 0: Locally disabling no-member (E1101) (locally-disabled) I:439, 0: Locally disabling maybe-no-member (E1103) (locally-disabled) ************* Module pki.server.deployment.pkiparser I: 36, 0: Locally disabling no-name-in-module (E0611) (locally-disabled) I: 38, 0: Locally disabling import-error (F0401) (locally-disabled) I: 40, 0: Locally disabling redefined-builtin (W0622) (locally-disabled) I:362, 0: Locally disabling no-member (E1101) (locally-disabled) I:377, 0: Locally disabling no-member (E1101) (locally-disabled) F: 36, 4: Unable to import 'urllib.parse' (import-error) F: 40, 0: Unable to import 'six.moves' (import-error) F: 41, 0: Unable to import 'six.moves' (import-error) ************* Module pki.server.deployment.pkiscriptlet I: 36, 0: Locally disabling unused-argument (W0613) (locally-disabled) W: 29, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.pkihelper I: 35, 0: Locally disabling no-name-in-module (E0611) (locally-disabled) I:1091, 0: Locally disabling broad-except (W0703) (locally-disabled) I:2978, 0: Locally disabling broad-except (W0703) (locally-disabled) I:2999, 0: Locally disabling broad-except (W0703) (locally-disabled) I:3903, 0: Locally disabling no-member (E1101) (locally-disabled) ************* Module pki.server.deployment.pkiconfig I: 25, 0: Locally disabling redefined-builtin (W0622) (locally-disabled) F: 25, 0: Unable to import 'six.moves' (import-error) ************* Module pki.server.deployment.scriptlets.security_databases W: 31, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.webapp_deployment W: 33, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.instance_layout W: 33, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.finalization W: 32, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.infrastructure_layout W: 32, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.subsystem_layout W: 31, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.selinux_setup W: 36, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.slot_substitution W: 31, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.initialization W: 31, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.configuration W: 34, 0: Class has no __init__ method (no-init) I: 83, 0: Locally disabling unused-argument (W0613) (locally-disabled) ************* Module pkispawn I: 58, 0: Locally disabling unused-argument (W0613) (locally-disabled) I:602, 0: Locally disabling broad-except (W0703) (locally-disabled) ************* Module pkidestroy I: 55, 0: Locally disabling unused-argument (W0613) (locally-disabled) ************* Module pki-upgrade I: 31, 0: Locally disabling unused-argument (W0613) (locally-disabled) ************* Module pki-server-upgrade I: 33, 0: Locally disabling unused-argument (W0613) (locally-disabled) ************* Module 01-RemoveInaccessableURLsFromServerXML I: 34, 0: Locally disabling anomalous-backslash-in-string (W1401) (locally-disabled) ************* Module 01-AddKraAuditEvents I: 48, 0: Locally disabling unused-argument (W0613) (locally-disabled) 2. In "Manually fix import problems and access to Exception.message" you replaced e.message['desc'] with e.args[0]['desc']. I suppose the code works fine, but it's kind of strange that we'd have to use a more cryptic code. Are there other alternatives? Everything else looks fine. Thanks for the clarification about list(). -- Endi S. Dewata
On 2015-08-15 00:36, Endi Sukma Dewata wrote:
On 8/14/2015 10:03 AM, Christian Heimes wrote:
And now with attachment.
I also attached a 2nd patch that fixes the one unit test under Python 3.
Christian
Some comments:
1. Build works on F22, but fails on F21:
************* Module pylint-build-scan I: 33, 0: Locally disabling import-error (F0401) (locally-disabled) ************* Module pki I: 33, 0: Locally disabling redefined-builtin (W0622) (locally-disabled) I:127, 0: Locally disabling too-few-public-methods (R0903) (locally-disabled) I:141, 0: Locally disabling too-few-public-methods (R0903) (locally-disabled) I:149, 0: Locally disabling invalid-name (C0103) (locally-disabled) I:161, 0: Locally disabling invalid-name (C0103) (locally-disabled) F: 33, 0: Unable to import 'six.moves' (import-error) ************* Module pki.encoder I: 36, 0: Locally disabling method-hidden (E0202) (locally-disabled) ************* Module pki.profile I:1361, 0: Locally disabling broad-except (W0703) (locally-disabled) I:1398, 0: Locally disabling broad-except (W0703) (locally-disabled) ************* Module pki.key I: 34, 0: Locally disabling no-name-in-module (E0611) (locally-disabled) I: 36, 0: Locally disabling no-name-in-module (E0611) (locally-disabled) I: 43, 0: Locally disabling too-few-public-methods (R0903) (locally-disabled) I: 54, 0: Locally disabling too-few-public-methods (R0903) (locally-disabled) I: 66, 0: Locally disabling invalid-name (C0103) (locally-disabled) I:118, 0: Locally disabling invalid-name (C0103) (locally-disabled) I:150, 0: Locally disabling too-few-public-methods (R0903) (locally-disabled) I:186, 0: Locally disabling invalid-name (C0103) (locally-disabled) I:222, 0: Locally disabling too-few-public-methods (R0903) (locally-disabled) I:258, 0: Locally disabling invalid-name (C0103) (locally-disabled) F: 34, 4: Unable to import 'urllib.parse' (import-error) ************* Module pki.upgrade I:545, 0: Locally disabling exec-used (W0122) (locally-disabled) I:632, 0: Locally disabling broad-except (W0703) (locally-disabled) I:699, 0: Locally disabling broad-except (W0703) (locally-disabled) ************* Module pki.server.cli.migrate I:166, 0: Locally disabling protected-access (W0212) (locally-disabled) I:279, 0: Locally disabling protected-access (W0212) (locally-disabled) ************* Module pki.server.cli.instance I:327, 0: Locally disabling no-member (E1101) (locally-disabled) I:327, 0: Locally disabling maybe-no-member (E1103) (locally-disabled) I:383, 0: Locally disabling no-member (E1101) (locally-disabled) I:383, 0: Locally disabling maybe-no-member (E1103) (locally-disabled) I:439, 0: Locally disabling no-member (E1101) (locally-disabled) I:439, 0: Locally disabling maybe-no-member (E1103) (locally-disabled) ************* Module pki.server.deployment.pkiparser I: 36, 0: Locally disabling no-name-in-module (E0611) (locally-disabled) I: 38, 0: Locally disabling import-error (F0401) (locally-disabled) I: 40, 0: Locally disabling redefined-builtin (W0622) (locally-disabled) I:362, 0: Locally disabling no-member (E1101) (locally-disabled) I:377, 0: Locally disabling no-member (E1101) (locally-disabled) F: 36, 4: Unable to import 'urllib.parse' (import-error) F: 40, 0: Unable to import 'six.moves' (import-error) F: 41, 0: Unable to import 'six.moves' (import-error) ************* Module pki.server.deployment.pkiscriptlet I: 36, 0: Locally disabling unused-argument (W0613) (locally-disabled) W: 29, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.pkihelper I: 35, 0: Locally disabling no-name-in-module (E0611) (locally-disabled) I:1091, 0: Locally disabling broad-except (W0703) (locally-disabled) I:2978, 0: Locally disabling broad-except (W0703) (locally-disabled) I:2999, 0: Locally disabling broad-except (W0703) (locally-disabled) I:3903, 0: Locally disabling no-member (E1101) (locally-disabled) ************* Module pki.server.deployment.pkiconfig I: 25, 0: Locally disabling redefined-builtin (W0622) (locally-disabled) F: 25, 0: Unable to import 'six.moves' (import-error) ************* Module pki.server.deployment.scriptlets.security_databases W: 31, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.webapp_deployment W: 33, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.instance_layout W: 33, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.finalization W: 32, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.infrastructure_layout W: 32, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.subsystem_layout W: 31, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.selinux_setup W: 36, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.slot_substitution W: 31, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.initialization W: 31, 0: Class has no __init__ method (no-init) ************* Module pki.server.deployment.scriptlets.configuration W: 34, 0: Class has no __init__ method (no-init) I: 83, 0: Locally disabling unused-argument (W0613) (locally-disabled) ************* Module pkispawn I: 58, 0: Locally disabling unused-argument (W0613) (locally-disabled) I:602, 0: Locally disabling broad-except (W0703) (locally-disabled) ************* Module pkidestroy I: 55, 0: Locally disabling unused-argument (W0613) (locally-disabled) ************* Module pki-upgrade I: 31, 0: Locally disabling unused-argument (W0613) (locally-disabled) ************* Module pki-server-upgrade I: 33, 0: Locally disabling unused-argument (W0613) (locally-disabled) ************* Module 01-RemoveInaccessableURLsFromServerXML I: 34, 0: Locally disabling anomalous-backslash-in-string (W1401) (locally-disabled) ************* Module 01-AddKraAuditEvents I: 48, 0: Locally disabling unused-argument (W0613) (locally-disabled)
Thanks for the tests. Older versions of pylint I've addressed the six.moves import-error failures in my updated patch. The no-init warnings seem to be unrelated to my work, though.
2. In "Manually fix import problems and access to Exception.message" you replaced e.message['desc'] with e.args[0]['desc']. I suppose the code works fine, but it's kind of strange that we'd have to use a more cryptic code. Are there other alternatives?
No, there is no better way to write it. :( I checked python-ldap's and pyldap's source code as well as tested a couple of alternative approaches interactively. The LDAPError exception doesn't have other means to access the error dict. In Python 2 Exceception.message is an alternative spelling for Exception.args[0] if len(args) == 1. The alternative is gone in Python 3. Christian
participants (3)
-
Ade Lee -
Christian Heimes -
Endi Sukma Dewata