Performance

This page attempts to given an overview over the performance history of Nuitka. It aims to list benchmark results. It's probably also quite bad at it.

Note

This is not comparing Nuitka to CPython yet. It's going to be added, stay tuned.

PyStone Valgrind Ticks

Running the "pystone" benchmark for 50000 passes, and measuring the amount of ticks with Valgrind. It doesn't matter what "pystones" are output, instead the whole benchmark is measured.

The idea of using Valgrind is to get reliable results, without any system perturbation. It gives good measurements and makes even tiny effects visible. On purpose, the ticks for initialization are not counted, i.e. the tick counter starts only when the main module is entered, not when the compiled types and constant values are prepared.

Of course, "pystone" code exercises not a whole lot of Python features, esp. no modern ones at all. Still it's a nice indicator. It will benefit most from quick module variable access, quick instance attribute access and function call performance, etc.

images/pystone-nuitka.svg
Nuitka Pystone ticks, raw numbers
Version Ticks Python 2.6 based Ticks Python 2.7 based
0.3.12 977315243 989509750
0.3.13a 1031029182 1043073769
0.3.14 1030123424 1043467660
0.3.15 997975208 1009870562
0.3.16 1002596799 1014251597
0.3.17 1001756094 1013258387
0.3.18 1001756824 1013859206
0.3.19 1000158153 1011210449
0.3.20 1000158037 1011810394
0.3.21 1011556370 1022609354
0.3.22 964040316 995544021
0.3.23 964040458 995544230
0.3.24 972490185 1003992331
0.3.25 951496456 981185157
0.4.0 971318092 1001837024
0.4.1 971768129 1001790640
0.4.2 1006911994 1039851815
0.4.3 1001661854 1034602395
0.4.4 861651777 887757576
0.4.5 858787822 881332208
develop 864538165 887382492
factory 849287618 869374109

PyStone Binary Size

The size of the created binary is also an interesting figure of course. While we are very willing to trade performance for executable size, there should be some gain. More code may also mean worse cache performance.

This is a by-product of the above PyStone valgrind based test. It's a pretty automatic result, and an interesting indicator for generated code complexity.

images/pystone-binary-nuitka.svg
Nuitka compiled Pystone binary size, raw numbers
Version Size Python 2.6 based Size Python 2.7 based
0.3.12 144689 144259
0.3.13a 148307 147573
0.3.14 149689 149251
0.3.15 149668 149342
0.3.16 152346 151712
0.3.17 153032 152190
0.3.18 153204 152570
0.3.19 152740 152078
0.3.20 153869 153311
0.3.21 152787 152309
0.3.22 161305 160743
0.3.23 161417 160855
0.3.24 161244 160826
0.3.25 175485 174819
0.4.0 180373 179787
0.4.1 181157 180635
0.4.2 185364 184830
0.4.3 186613 186079
0.4.4 193597 192647
0.4.5 193661 192711
develop 197377 196663
factory 200561 199863

PyStone Peak Memory Usage

The peak malloc memory of the resulting binary. While we are willing to trace memory usage over performance, often higher memory usage leads to lower performance, because e.g. two objects that could be shared are now duplicated. Reducing memory usage means to share more objects and consequently better performance.

This is a by-product of the above PyStone valgrind based test. It's a pretty automatic result, and an interesting indicator for memory leaks or avoided objects.

images/pystone-memory-nuitka.svg
Nuitka compiled Pystone binary size, raw numbers
Version Peak memory Python 2.6 based Peak memory Python 2.7 based
0.3.12 3240338 3757240
0.3.13a 3249074 3765976
0.3.14 4093864 4567286
0.3.15 4093840 4567262
0.3.16 4093560 4566982
0.3.17 4093600 4567022
0.3.18 4093600 4567022
0.3.19 4093600 4567022
0.3.20 4093600 4567022
0.3.21 4092616 4566038
0.3.22 4092616 4566038
0.3.23 4092616 4566038
0.3.24 4093384 4566806
0.3.25 4093476 4566901
0.4.0 4093060 4566485
0.4.1 4093060 4566485
0.4.2 4077790 4551282
0.4.3 4077790 4551282
0.4.4 4074710 4543690
0.4.5 4077790 4551250
develop 4077790 4551250
factory 4080094 4553554