12 April 2024
Nuitka Release 2.2
This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, “download now”.
This release focused on compatibility and significant optimization progress for loops, such as list operations within. The main line of change is to be able to support Python 3.12 in the next release. Standalone: Added support for Standalone: Make ArchLinux: Added platform linker option to be usable with their current Arch Python package. Fixed in 2.1.1 already. Fix, Fix, Standalone: Added support for Standalone: Added support for Standalone: Updated Fix, PySide6 slots failed to be moved between threads. For that we need to make function renames visible in the owning class as well. Fixed in 2.1.3 already. Standalone: Added support for Standalone: Handle “string” import errors of Fix, the Standalone: Added support for Standalone: Added support for newer Standalone: Added support for newer Standalone: Fix, Onefile: Respect Python2: Some cases of Fix, optimized Standalone: Added missing implicit dependencies for Standalone: Added support for Standalone: Added DLLs for Standalone: Added missing metadata of Standalone: Added support for newer Standalone: Fix, older Fix, Standalone: Added data files for Standalone: Added data files for Standalone: Fix, the standalone standard library scan was not fully ignoring git folders, subfolders were still looked at and could cause issues. Fixed in 2.1.5 already. Standalone: Added support for newer Standalone: Add support for newer Scons: Fix, when locating binaries, do not use directories but only files. A directory on Windows: Fix, by default, scan only for Fix, the Standalone: Added support for more parts of Windows: Fix working with UNC paths and re-parse points at compile time. Now Nuitka should work with mapped and even unmapped to drive paths like Windows: Make sure the download path is an external use path in scons as well, otherwise the home directory could be an unusable path for MinGW64, causing it not to find files. Standalone: Added missing dependency of Python3.5+: Add support for using dictionary un-packings in class declarations. That is a rarely used in actual Python code, but was found missing by tests recently. Python3.11: Fix, code objects Anaconda: Fix, must not consider the Anaconda Fix, cannot trust dynamic hard modules as much otherwise, macOS: Fix, on newer macOS the Fix, support for newer Standalone: Added required data files for Standalone: Added required data files for Standalone: Added required data files for Standalone: Fixup Fix, need to make parent module usages more explicit. Otherwise, plugin mechanisms like Fix, the Added experimental support for Python 3.12, this is passing basic tests, but known to crash a lot at run-time still, you are recommended to use pre-releases of Nuitka, as official support is not going to happen before 2.3 release. Standalone: Added support for With preserved source code of decorated functions and we can provide it at run-time to For Nuitka Package Configuration, we now have For the experimental Added new kind of warning of Zig: Added support for For the Use Lower Recognize exact type shapes of loop variables where possible. This enables appends to list to be optimized to their dedicated nodes among other things, with those often being a lot faster than generic code. This speeds up e.g. list Optimization: Have dedicated helper for ArchLinux: Enable static libpython by default, it is usable indeed. Added in 2.1.2 already. Anti-Bloat: Avoid Anti-Bloat: Avoid Anti-Bloat: Avoid using Anti-Bloat: Avoid testing packages for newer Optimization: Avoid recompiling Avoid packages owned by Nuitka plugins in Anti-Bloat: Avoid Anti-Bloat: Avoid Anti-Bloat: Avoid Anti-Bloat: Recognize Faster code generation due to enhancements in how identifiers are cached for module names and the indentation codes. Optimization: Handle Anti-Bloat: Avoid Anti-Bloat: Remove testing module usage when Anti-Bloat: Avoid Anti-Bloat: Avoid UI: Catch conflicts between data files and EXE/DLLs/extension module filenames. Previously, you could overwrite binaries with data files, but that is now rejected as an explicit error. Onefile: Avoid using the program name without suffix inside the dist folder, as that avoids collisions with data file directories of the same name, e.g., if the package and main binary have the same name, they would clash previously, but adding a UI: Don’t force That made it hard for users, who will be surprised to see UI: Catch wrong values for UI: Nicer usage name when invoked with The recommended form of invocation of Nuitka should not have an ugly invocation reference mentioning UI: Reorder options for the plugins group to be more readable. Plugins: Remove obsolete plugins from standard plugin documentation. Removed in 2.1.4 already. UI: The Windows release was coming from the compiling Python and as such wrong, for example, Windows 11 always showed up as Windows 10, and some older versions of Python didn’t know Windows 10, yet, so this could be confusing in issue analysis. UI: Do not warn about static libpython for Python debug mode compilation. It is misleading as often it doesn’t work for that configuration, and it’s only a distraction since debugging Python reference counts is not about performance. Changed in 2.1.4 already. UI: Catch newlines in spec values. They break code C code generation potentially; they also are likely copy&paste mistakes that won’t do what the user expects. Added in 2.1.4 already. Quality: Updated to the latest version of black. Quality: Fix, Debugging: Generate Scons debug script It can serve to quickly re-execute a Scons compilation without re-executing Nuitka again. This is best used where there is no Python level change but only C changes and no expectation of producing a usable result. Because no post-processing is applied, and as a consequence this is not usable to produce binaries that work. In the future, we might expand this to be able to run post-processing still. Debugging: Disabling all freelists is now honored for more code, tuples and empty dictionaries as well. UI: Add macOS version to help output, which is sometimes vital for issue analysis. Reports: Add the OS release to reports as well. Reports: Exclude parent path imports from compilation reports for module usages that are found and end up not being excluded. Watch: Reporting more problems, catching more errors, and adding the ability to create PRs from changes. However, it does not yet do it automatically. Visual Code: Have plugins C files in the include path as well. Tests: Fix, cannot assume Run commercial code signing test only on Windows. Allow for standalone testing file access to the Azure agent folders. For tests on Azure, it’s like the home directory. Make sure optimization tests are named to make it clear that they are tests. Remove useless The JIT mechanism added for The road to Python 3.12 is not fully complete, but the end feels closer now, and the subsequent release hopefully will add the official support for it.Bug Fixes
pypdfium2
package. Fixed in 2.1.1 already.cefpython3
work on Linux. Fixed in 2.1.1 already.ctypes.CDLL
optimization used a misspelled argument name for use_last_error
, such that keyword argument calls were statically optimized into TypeError
at compile-time. Fixed in 2.1.1 already.list.insert
was not properly annotating exceptions. Raises by producing the inserted value raised or the index was not annotated and, therefore, could fail to be caught locally. Fixed in 2.1.1 already.selenium
package. Fixed in 2.1.2 already.hydra
package. Fixed in 2.1.2 already.dotenv
workaround for newer version. Fixed in 2.1.3 already.win32com.server.register
. Fixed in 2.1.3 already.uvicorn
gracefully. Fixed in 2.1.3 already.dill-compat
plugin needs also needs to expose the compiled type names as built-ins for the pickle
module to find them.gruut
package. Fixed in 2.1.3 already.toga
to also include toga_winforms
metadata. Fixed in 2.1.3 already.tensorflow
package. Fixed in 2.1.4 already.matplotlib
needs to emit a dependency on the backend to be included. Otherwise it could be missing at run-time in some cases. Fixed in 2.1.4 already.XDG_CACHE_HOME
variable on non-Windows platforms. Some users might configure that to not to be ~/.cache
, respect that. Fixed in 2.1.4 already.list.insert
were not properly handling all index types. Fixed in 2.1.4 already.list.remove
failed to handle tuple arguments properly. Removing tuple values from lists could cause errors. Fixed in 2.1.4 already.pyarrow.datasets
. Fixed in 2.1.4 already.dask.dataframe
module. Fixed in 2.1.4 already.tensorrt_libs
package. Fixed in 2.1.4 already.numpy
for xarray
package. Fixed in 2.1.4 already.scipy
. Fixed in 2.1.5 already.gcc
could give warning about C code to work with PYTHONPATH
which caused build errors on older systems. Fixed in 2.1.5 already.locals
representing nodes could not be cloned, and as a result, some code re-formulations failed to compile in try
constructs. Fixed in 2.1.5 already.names
package. Fixed in 2.1.5 already.randomname
package. Fixed in 2.1.5 already.transformers
. Fixed in 2.1.5 already.bitsandbytes
. Fixed in 2.1.5 already.PATH
that was named gcc
could be mistaken to be a gcc
binary causing errors. Fixed in 2.1.6 already..bin
and .exe
binaries for Nuitka Package Configuration EXE dependency patterns. This was the intended value, but it had not taken effect yet. Fixed in 2.1.6 already.__compiled__.containing_dir
should be an absolute path. For it to be usable after a change of directory is done by the program that is required. Fixed in 2.1.6 already.networkx
package. Fixed in 2.1.6 already.\\some-hostname\unc-test
as they are common in some VM setups.sspilib
that prevented requests-ntlm
from working on Windows.co_qualname
attribute was not actually the qualified name, but the same as co_name
only.lib
directory as a system directory, because then those DLLs that are not included.huggingface_hub.utils.tqdm
ended up being a module and not the class it’s supposed to be.libc++
and libz
DLLs cannot be found anymore, we need to ignore that actively as our code insists on full resolution to catch bugs.zaber_motion
was not really working.pyviz_comms
.panel
package.bokeh
package.scipy
for Anaconda.no-follow
from a parent module cannot affect its child modules, as they can end up being followed to only after them.dill-compat
plugin in module mode cannot assume the main module name to be the one from compile time, need to look the actual one up at runtime.New Features
tensorflow.function
JITtensorflow
JIT so it can do its tracing executions.change_class
similar to change_function
to replace a full class definition with something else, this can be used to modify classes to become stubs or even unusable.@pyqtSlot
decorator, we also should handle the @asyncSlot
the same way. Added in 2.1.1 already.plugin
category and use it in the Nuitka Package Configuration to inform matplotlib
users to select a GUI backend via plugin selection. Added in 2.1.4 already.zig
as CC value. Due to it not supporting C11 fully yet, we need to use the C++ workaround and cannot compile for Python 3.11 or higher yet.__compiled__
value, we now have a __compiled__.main
that is the name of the compiled module. For modules, Nuitka determines this at run time; in other modes, it is the name of the main module.Optimization
set
specific API in contains tests, rather than generic sequence one.value in something
tests for known set
and list
values to use frozenset
and tuple
respectively.append
tests by a significant amount.list.remove
, such that it is not using a Python DLL call where that is slow.unittest
usage in antlr
package.IPython
in celery
package. Added in 2.1.2 already.setuptools
in transformers
package for more modules. Added in 2.1.3 already.tensorflow
package as well. Added in 2.1.4 already.azure
package which is not performance relevant. Added in 2.1.4 already.matploblib
backends unless the corresponding plugin is actually active. Added in 2.1.4 already.setuptools
in deepspeed
package. Added in 2.1.4 already.setuptools
in transformers
package. Added in 2.1.4 already.scipy
usage causing torch
or cupy
usage. Added in 2.1.4 already.keras
testing modules as unittest
bloat.no_docstrings
issue for torio
package.IPython
from imgui_bundle
package.dask
is used.unitest
usage in tf_keras
package as well.IPython
from bokeh
package.Organizational
.bin
suffix to the binary avoids that entirely.{VERSION}
in specs to be resolved to four digits.1.0
become 1.0.0.0
when that is only needed for Windows version information really.--jobs
value sooner, negative and non-integer values error exit immediately. Added in 2.1.1 already.python -m nuitka
__main__.py
instead put the python -m nuitka
notion there.isort
and black
can corrupt outputs, catch that.Tests
setuptools
to be installed, some RPM based systems don’t have it.Cleanups
--execute-with-pythonpath
option, we don’t use that anymore at all.Summary
tensorflow
should be possible to generalize and will be applied to other JITs, like numba
and others in the future as well.