Nuitka Release 0.3.12
This is to inform you about the new release of Nuitka many bug fixes,
and substantial improvements especially in the organisational area.
There is a new User Manual
(PDF), with much improved
sys.meta_path based import mechanism for
git flow goodness.
This release is generally also the result of working towards compilation of a real programs (Mercurial) and to get things work more nicely on Windows by default. Thanks go to Liu Zhenhai for helping me with this goal.
Due to the use of the “git flow”, most of the bugs listed here were already fixed in on the stable release before this release. And there were many of these.
The order of evaluation for base classes and class dictionaries was not enforced.
Apparently nothing in the CPython test suite did that, I only noticed during debugging that Nuitka gave a different error than CPython did, for a class that had an undefined base class, because both class body and base classes were giving an error. Fixed in 0.3.11a already.
Method objects didn’t hold a reference to the used class.
The effect was only noticed when
--python-debugwas used, i.e. the debug version of Python linked, because then the garbage collector makes searches. Fixed in 0.3.11b already.
sys.executableon Linux as well. On Debian it is otherwise
/usr/bin/pythonwhich might be a different version of Python entirely. Fixed in 0.3.11c already.
Embedded modules inside a package could hide package variables of the same name. Learned during PyCON DE about this corner case. Fixed in 0.3.11d already.
Packages could be duplicated internally. This had no effect on generated code other than appearing twice in the list if frozen modules. Fixed in 0.3.11d already.
When embedding modules from outside current directory, the look-up failed. The embedding only ever worked for the compile itself and programs test cases, because they are all in the current directory then. Fixed in 0.3.11e already.
The check for ARM target broke Windows support in the Scons file. Fixed in 0.3.11f already.
The star import from external modules failed with an error in
--deepmode. Fixed in 0.3.11g already.
Modules with a parent package could cause a problem under some circumstances. Fixed in 0.3.11h already.
One call variant, with both list and dict star arguments and keyword arguments, but no positional parameters, didn’t have the required C++ helper function implemented. Fixed in 0.3.11h already.
The detection of the CPU core count was broken on my hexacore at least. Gave 36 instead of 6, which is a problem for large programs. Fixed in 0.3.11h already.
The in-line copy of Scons didn’t really work on Windows, which was sad, because we added it to simplify installation on Windows precisely because of this.
Cleaning up the build directory from old sources and object files wasn’t portable to Windows and therefore wasn’t effective there.
From imports where part of the imported were found modules and parts were not, didn’t work. Solved by the feature branch
meta_path_importthat was merged for this release.
Newer MinGW gave warnings about the default visibility not being possible to apply to class members. Fixed by not setting this default visibility anymore on Windows.
sys.executablegave warnings on Windows because of backslashes in the path. Using a raw string to prevent such problems.
The standard library path was hard coded. Changed to run time detection.
Version checks on Python runtime now use a new define
PYTHON_VERSIONthat makes it easier. I don’t like
PY_VERSION_HEX, because it is so unreadable. Makes some of the checks a lot more safe.
sys.meta_pathbased import from the
meta_path_importfeature branch allowed the cleanup the way importing is done. It’s a lot less code now.
Removed some unused code. We will aim at making Nuitka the tool to detect dead code really.
nuitka.nodes.Nodes, that is what the package is intended for, the split will come later.
New tests for import variants that previously didn’t work: Mixed imports. Imports from a package one level up. Modules hidden by a package variable, etc.
Added test of function call variant that had no test previously. Only found it when compiling “hg”. Amazing how nothing in my tests, CPython tests, etc. used it.
Added test to cover the partial success of import statements.
Added test to cover evaluation order of class definitions.
Migrated the “README.txt” from org-mode to ReStructured Text, which allows for a more readable document, and to generate a nice User Manual in PDF form.
The amount of information in “README.txt” was increased, with many more subjects are now covered, e.g. “git flow” and how to join Nuitka development. It’s also impressive to see what code blocks and syntax highlighting can do for readability.
The Nuitka git repository has seen multiple hot fixes.
These allowed to publish bug fixes immediately after they were made, and avoided the need for a new release just to get these out. This really saves me a lot of time too, because I can postpone releasing the new version until it makes sense because of other things.
Then there was a feature branch
meta_path_importthat lived until being merged to
developto improve the import code, which is now released as part of the main branch. Getting that feature right took a while.
And there is the feature branch
minimize_CPython26_tests_diffwhich has some success already in documenting the required changes to the “CPython26” test suite and in reducing the amount of differences, while doing it. We have a frame stack working there, albeit in too ugly code form.
The release archives are now built using
setuptools. You can now also download a zip file, which is probably more Windows friendly. The intention is to work on that to make
setup.pyproduce a Nuitka install that won’t rely on any environment variables at all. Right now
setup.pywon’t even allow any other options than
sdistto be given.
Ported “compile_itself.sh” to “compile_itself.py”, i.e. ported it to Python. This way, we can execute it easily on Windows too, where it currently still fails. Replacing
rm -rf, etc. is a challenge, but it reduces the dependency on MSYS tools on Windows.
The compilation of standard library is disabled by default, but
distpackages are now embedded. To include even standard library, there is a
--really-deepoption that has to be given in addition to
--deep, which forces this.
Again, huge progress. The improved import mechanism is very beautiful. It appears that little is missing to compile real world programs like “hg” with Nuitka. The next release cycle will focus on that and continue to improve the Windows support which appears to have some issues.