Nuitka Release 0.3.17
This is to inform you about the new stable release of Nuitka. This time
there are a few bug fixes, lots of very important organisational work,
and yet again improved compatibility and cleanups. Also huge is the
advance in making
--deep go away and making the recursion of Nuitka
controllable, which means a lot for scalability of projects that use a
lot of packages that use other packages, because now you can choose
which ones to embed and which ones one.
The release cycle had a focus on improving the quality of the test scripts, the packaging, and generally to prepare the work on “type inference” in a new feature branch.
I have also continued to work towards CPython3.2 compatibility, and this
version, while not there, supports Python3 with a large subset of the
basic tests programs running fine (of course via
without trouble. There is still work to do, exceptions don’t seem to
work fully yet, parameter parsing seems to have changed, etc. but it
seems that CPython3.2 is going to work one day.
And there has been a lot of effort, to address the Debian packaging to be cleaner and more complete, addressing issues that prevented it from entering the Debian repository.
Fixed the handling of modules and packages of the same name, but with different casing. Problem showed under Windows only. Released as 0.3.16a hot fix already.
Fixed an error where the command line length of Windows was exceeded when many modules were embedded, Christopher Tott provided a fix for it. Released as 0.3.16a hot fix already.
Fix, avoid to introduce new variables for where built-in exception references are sufficient. Released as 0.3.16b hot fix already.
Fix, add the missing
__new__methods before resolving the scopes of variables, this avoids the use of that variable before it was assigned a scope. Released as 0.3.16b hot fix already.
Enhanced compatibility again, provide enough
co_varnamesin the code objects, so that slicing them up to
code_object.co_argcountwill work. They are needed by
inspectmodule and might be used by some decorators as well.
New options to control the recursion:
--recurse-none(do not warn about not-done recursions)
--recurse-all(recurse to all otherwise warned modules)
--recurse-to(confirm to recurse to those modules)
--recurse-not-to(confirm to not recurse to those modules)
The optimization of constant conditional expressions was not done yet. Added this missing constant propagation case.
Eliminate near empty statement sequences (only contain a pass statement) in more places, giving a cleaner node structure for many constructs.
Use the pickle “protocol 2” on CPython2 except for
unicodestrings where it does not work well. It gives a more compressed and binary representation, that is generally more efficient to un-stream as well. Also use the cPickle protocol, the use of
picklewas not really necessary anymore.
Added a Developer Manual to the release. It’s incomplete, but it details some of the existing stuff, coding rules, plans for “type inference”, etc.
--helpoutput to use
metavarwhere applicable. This makes it more readable for some options.
Instead of error message, give help output when no module or program file name was given. This makes Nuitka help out more convenient.
#!/usr/bin/env pythonfor all scripts, this was previously only done for some of them.
Ported the PyLint check script to Python as well, enhancing it on the way to check the exit code, and to only output changes things, as well as making the output of warnings for
All scripts used for testing, PyLint checking, etc. now work with Python3 as well. Most useful on Arch Linux, where it’s also already the default for
The help output of Nuitka was polished a lot more. It is now more readable and uses option groups to combine related options together.
Make the tests run without any dependence on
PATHto contain the executables of Nuitka. This makes it easier to use.
Add license texts to 3rd party file that were missing them, apply
licensecheckresults to cleanup Nuitka. Also removed own copyright statement from in-line copy of Scons, it had been added by accident only.
Release the tests that I own as well as the Debian packaging I created under “Apache License 2.0” which is very liberal, meaning every project will be able to use it.
Don’t require copyright assignment for contributions anymore, instead only “Apache License 2.0”, the future Nuitka license, so that the code won’t be a problem when changing the license of all of Nuitka to that license.
Give contributors listed in the User Manual an exception to the GPL terms until Nuitka is licensed under “Apache License 2.0” as well.
--experimentaloption which can be used to control experimental features, like the one currently being added on branch
feature/ctypes_annotation, where “type inference” is currently only activated when that option is given. For this stable release, it does nothing.
Check the static C++ files of Nuitka with
cppcheckas well. Didn’t find anything.
Arch Linux packages have been contributed, these are linked for download, but the stable package may lag behind a bit.
notboolean operation to become a normal operator. Changed
orboolean operators to a new base class, and making their interface more similar to that of operations.
tagsto node classes for use in checks. Use it annotate which node kinds to visit in e.g. per scope finalization steps. That avoids kinds and class checks.
Enhanced the “visitor” interface to provide more kinds of callbacks, enhanced the way “each scope” visiting is achieved by generalizing is as “child has not tag ‘closure_taker’” and that for every “node that has tag ‘closure_taker’”.
nuitka.guipackage where it belongs.
More white listing work for imports. As recursion is now the default, and leads to warnings for non-existent modules, the CPython tests gave a lot of good candidates for import errors that were white listed.
nuitkain test scripts, as there isn’t a
Nuitka.pyon all platforms. The later is scheduled for removal.
Some more PyLint cleanups.
Make sure the basic tests pass with CPython or else fail the test. This is to prevent false positives, where a test passes, but only because it fails in CPython early on and then does so with Nuitka too. For the syntax tests we make sure they fail.
The basic tests can now be run with
2to3conversion in that case. Also the currently not passing tests are not run, so the passing tests continue to do so, with this run from the release test script
Include the syntax tests in release tests as well.
Changed many existing tests so that they can run under CPython3 too. Of course this is via
Don’t fail if the CPython test suites are not there.
Currently they remain largely unpublished, and as such are mostly only available to me (exception,
feature/minimize_CPython26_tests_diffbranch references the CPython2.6 tests repository, but that remains work in progress).
For the compile itself test: Make the presence of the Scons in-line copy optional, the Debian package doesn’t contain it.
Also make it more portable, so it runs under Windows too, and allow to choose the Python version to test. Check this test with both CPython2.6 and CPython2.7 not only the default Python.
Before releasing, test that the created Debian package builds fine in a minimal Debian
unstablechroot, and passes all the tests included in the package (
reflected). Also many other Debian packaging improvements.
The “git flow” was used again in this release cycle and proved to be
useful not only for hot fix, but also for creating the branch
feature/ctypes_annotation and rebasing it often while things are
The few hot fixes didn’t require a new release, but the many organisational improvements and the new features did warrant the new release, because of e.g. the much better test handling in this release and the improved recursion control.
The work on Python3 support has slowed down a bit. I mostly only added some bits for compatibility, but generally it has slowed down. I wanted to make sure it doesn’t regress by accident, so running with CPython3.2 is now part of the normal release tests.
What’s still missing is more “hg” completeness. Only the
inspect was going in that direction, and this has slowed
down. It was more important to make Nuitka’s recursion more accessible
with the new options, so that was done first.
And of course, the real excitement is the “type inference” work. It will
give a huge boost to Nuitka, and I am happy that it seems to go well.
With this in place, new benchmarks may make sense. I am working on
getting it off the ground, so other people can work on it too. My idea
ctypes native calls may become true sooner than expected. To
support that, I would like to add more tools to make sure we discover
changes earlier on, checking the XML representations of tests to
discover improvements and regressions more clearly.