Nuitka Release 0.3.15

Posted on December 1, 2011 by kayhayen

This is to inform you about the new stable release of Nuitka. This time again many organisational improvements, some bug fixes, much improved compatibility and cleanups.

Please see the page “What is Nuitka?” for clarification of what it is now and what it wants to be.

This release cycle had a focus on packaging Nuitka for easier consumption, i.e. automatic packaging, making automatic uploads, improvement documentation, and generally cleaning things up, so that Nuitka becomes more compatible and ultimately capable to run the “hg” test suite. It’s not there yet, but this is a huge jump for usability of Nuitka and its compatibility, again.

Then lots of changes that make Nuitka approach Python3 support, the generated C++ for at least one large example is compiling with this new release. It won’t link, but there will be later releases.

And there is a lot of cleanup going on, geared towards compatibility with line numbers in the frame object.

Bug fixes

- The main module was using “__main__” in tracebacks, but it should be “<module>”. Released as 0.3.14a hotfix already.

- Workaround for “execfile cannot be used as an expression”. It wasn’t possible to use “execfile” in an expression, only as a statement. But then there is crazy enough code in e.g. mercurial that uses it in a lambda function, which made the issue more prominent. The fix now allows it to be an expression, except on the class level, which wasn’t seen yet.

- The inline copy of Scons was not complete enough to work for “Windows” or with “–windows-target” for cross compile. Fixed.

- Cached frames didn’t release the “back” frame, therefore holding variables of these longer than CPython does, which could cause ordering problems. Fixed for increased compatibility.

- Handle “yield outside of function” syntax error in compiled source correctly. This one was giving a Nuitka backtrace, now it gives a “SyntaxError” as it should.

- Made syntax/indentation error output absolutely identical to CPython.

- Using the frame objects “f_lineno” may fix endless amounts bugs related to traceback line numbers.

New Features

- Guesses the location of the MinGW compiler under Windows to default location, so it need not be added to “PATH” environment variable. Removes the need to modify “PATH” environment just for Nuitka to find it.

- Added support for “lambda generators”. You don’t want to know what it is. Lets just say, it was the last absurd language feature out there, that didn’t work. It now works perfect.

Organizational

- You can now download a Windows installer and a Debian package that works on Debian Testing, current Ubuntu and Mint Linux.

- New release scripts give us the ability to have hotfix releases as download packages immediately. That means the “git flow” makes even more beneficial to the users.

- Including the generated “README.pdf” in the distribution archives, so it can be read instead of “README.txt”. The text file is fairly readable, due to the use of ReStructured Text, but the PDF is even nicer to read, due to e.g. syntax highlighting of the examples.

- Renamed the main binaries to “nuitka” and “nuitka-python”, so that there is no dependency on case sensitive file systems.

- For Windows there are batch files “nuitka.bat” and “nuitka-python.bat” to make Nuitka directly executable without finding the “Python.exe”, which the batch files can tell from their own location.

- There are now man pages of “nuitka” and “nuitka-python” with examples for the most common use cases. They are of course included in the Debian package.

- Don’t strip the binary when executing “benchmark.sh” to analyse compiled binary with “valgrind”. It will give better information that way, without changing the code.

Optimizations

- Implemented “swapcontext” alike (“swapFiber”) for x64 to achieve 8 times speedup for Generators. It doesn’t do useless syscalls to preserve signal masks. Now Nuitka is faster at frame switching than CPython on x64, which is already good by design.

Cleanups

- Using the frame objects to store current line of execution avoids the need to store it away in helper code at all. It ought to also help a lot with threading support, and makes Nuitka even more compatible, because now line numbers will be correct even outside tracebacks, but for mere stack frame dumps.

- Moved the “for_return” detection from code generation to tree building where it belongs. Yield statements used as return statements need slightly different code for Python2.6 difference. That solved an old TODO.

- Much Python3 portability work. Sometimes even improving existing code, the Python compiler code had picked up a few points, where the latest Nuitka didn’t work with Python3 anymore, when put to actual compile. The test covered only syntax, but e.g. metaclasses need different code in CPython3, and that’s now supported. Also helper code was made portable in more places, but not yet fully. This will need more work.

- Cleaned up uses of debug defines, so they are now more consistent and in one place.

- Some more PyLint cleanups.

New Tests

- The tests are now executed by Python scripts and cover “stderr” output too. Before we only checked “stdout”. This unveiled a bunch of issues Nuitka had, but went unnoticed so far, and triggered e.g. the frame line number improvements.

- Separate syntax tests.

- The scripts to run the tests now are all in pure Python. This means, no more MinGW shell is needed to execute the tests.

Numbers

There are no new numbers. Nuitka should be as fast as it was, 258% speedup:

python 2.6:

Pystone(1.1) time for 50000 passes = 0.48
This machine benchmarks at 104167 pystones/second

Nuitka 0.3.11 (driven by python 2.6):

Pystone(1.1) time for 50000 passes = 0.19
This machine benchmarks at 263158 pystones/second

Summary

The Debian package, Windows installer, etc. are now automatically updated and uploaded. From here on, there can be such packages for the hotfix releases too.

The exception tracebacks are now correct by design, and better covered.

The generator performance work showed that the approach taken by Nuitka is in fact fast. It was fast on ARM already, but it’s nice to see that it’s now fast on x64. Programs using generators will be affected a lot by this.

Overall, this release brings Nuitka closer to usability. Better binary names, man pages, improved documentation, issue tracker, etc. all there now. I am in fact now looking for a sponsor for the Debian package to upload it into Debian directly.

What’s missing is more “hg” completeness. The frame release issue helped it, but “inspect.getargs()” doesn’t work yet, and is a topic for a future release. Won’t be easy, as “func_defaults” will be an invasive change too.

But I feel now certain, that the milestone “compatibility” may reach that level “compiled hg passes all tests” soonish. Then the focus will turn towards performance. Nuitka will then too compatible too believe, a good base.

As always you will find its latest version here.

Yours,
Kay Hayen

Posted in compiler, Nuitka, Python | 2 Comments

Nuitka Debian Package and Windows Installer

Posted on November 13, 2011 by kayhayen

There is now a Windows installer and a Debian package of Nuitka available on the Download page. Please try it out and give me feedback.

Specifically I do know that the Debian package won’t work on Debian Squeeze, but only on Debian Wheezy (Testing) maybe it does on Ubuntu as well, please report. If you have anything to criticize about the package, let me know. There is no apt source now, I hope to get it into Debian proper directly.

UPDATE: After Stani’s report that Ubuntu has an older Scons, I lowered the dependency and updated the package on the Download page. It may now work on Ubuntu as well.

And then, the Windows installer still requires you to install MinGW and add it to your path, but it’s clearly a huge step ahead. It’s created with distutils, and that works very well. If you have the skills to enhance it, so e.g. the PATH variable is changed, or it will launch a MinGW install if not present, contact me and help.

UPDATE: And the idea that I got while writing a reply to “swong” is also now implemented. The new Nuitka on Windows simply guesses the PATH to MinGW to be the default path “C:\MinGW” or at least “\MinGW” and from there, it ought to just run after you installed it. Of course you can still set your own PATH environment and make the pick yourself.

Yours,
Kay Hayen

Posted in Nuitka, Python | 4 Comments

Nuitka Release 0.3.14

Posted on November 7, 2011 by kayhayen

This is to inform you about the new stable release of Nuitka. This time it contains mostly organisational improvements, some bug fixes, improved compatibility and cleanups.

Please see the page “What is Nuitka?” for clarification of what it is now and what it wants to be.

This release is again the result of working towards compilation of a real program (Mercurial). This time, I have added support for proper handling of compiled types by the “inspect” module.

Bug fixes

- Fix for “Missing checks in parameter parsing with star list, star dict and positional arguments”. There as whole in the checks for argument counts, now the correct error is given. Fixed in 0.3.13a already.

- The simple slice operations with 2 values, not extended with 3 values, were not applying the correct order for evaluation. Fixed in 0.3.13a already.

- The simple slice operations couldn’t handle “None” as the value for lower or upper index. Fixed in 0.3.11a already.

- The inplace simple slice operations evaluated the slice index expressions twice, which could cause problems if they had side effects. Fixed in 0.3.11a already.

New Features

- Run time patching the “inspect” module so it accepts compiled functions, compiled methods, and compiled generator objects. The “test_inspect” test of CPython is nearly working unchanged with this.

- The generator functions didn’t have “CO_GENERATOR” set in their code object, made compatible with CPython in this regard too. The inspect module will therefore return correct value for “inspect.isgeneratorfunction()” too.

Optimizations

- Slice indexes that are None are now constant propagated as well.

- Slightly more efficient code generation for dual star arg functions, removing useless checks.

Cleanups

- Moved the scons, static C++ files, and asm files to new package “nuitka.build” where also now “SconsInterface” module lives.

- Moved the Qt dialog files to “nuitka.gui”

- Moved the unfreezer code to its own static C++ file.

- Some pylint cleanups.

New Tests

- New test “Recursion” to cover recursive functions.

- New test “Inspection” to cover the patching of “inspect” module.

- Cover “execfile” on the class level as well in “ExecEval” test.

- Cover evaluation order of simple slices in “OrderCheck” too.

Organizational

- There is a new issue tracker available under http://bugs.nuitka.net

Please register and report issues you encounter with Nuitka. I have put all the known issues there and started to use it recently. It’s Roundup based like bugs.python.org is, so people will find it familiar.

- The “setup.py” is apparently functional. The source releases for download are made it with, and it appears the binary distributions work too. We may now build a windows installer. It’s currently in testing, we will make it available when finished.

Numbers

There are no new numbers. Nuitka should be as fast as it was, 258% speedup:

python 2.6:

Pystone(1.1) time for 50000 passes = 0.48
This machine benchmarks at 104167 pystones/second

Nuitka 0.3.11 (driven by python 2.6):

Pystone(1.1) time for 50000 passes = 0.19
This machine benchmarks at 263158 pystones/second

Summary

The new source organisation makes packaging Nuitka really easy now. From here, we can likely provide “binary” package of Nuitka soon. A windows installer will be nice.

The patching of “inspect” works wonders for compatibility for those programs that insist on checking types, instead of doing duck typing. The function call problem, was an issue found by the Mercurial test suite.

For the “hg.exe” to pass all of its test suite, more work may be needed, this is the overall goal I am currently striving for. Once real world programs like “hg” work, we can use these as more meaningful benchmarks and resume work on optimization.

As always you will find its latest version here.

Yours,
Kay Hayen

Posted in compiler, Nuitka, Python | Comments Off

Nuitka Release 0.3.13

Posted on November 1, 2011 by kayhayen

This is to inform you about the new release of Nuitka. It again contains many bug fixes, and then substantial compatibility improvements.

Please see the page “What is Nuitka?” for clarification of what it is now and what it wants to be.

This release is mostly the result of working towards compilation of a real programs (Mercurial) and to merge and finalize the frame stack work. Now Nuitka has a correct frame stack at all times, and supports “func_code” and “gi_code” objects, something previously thought to be impossible.

Actually now it’s only the “bytecode” objects that won’t be there. And not attributes of “func_code” are meaningful yet, but in theory can be supported.

Due to the use of the git flow for Nuitka, most of the bugs listed here were already fixed in on the stable release before this release. This time there were 5 such hotfix releases, sometimes fixing multiple bugs.

Bug fixes

- In case of syntax errors in the main program, an exception stack was giving that included Nuitka code. Changed to make the same output as CPython does. Fixed in 0.3.12a already.

- The star import (“from x import *”) didn’t work for submodules. Providing “*” as the import list to the respective code allowed to drop the complex lookups we were doing before, and to simply trust CPython C-API to do it correctly. Fixed in 0.3.12 already.

- The absolute import is not the default of CPython 2.7 it seems. A local “posix” package shadows the standard library one. Fixed in 0.3.12 already.

- In “–deep” mode, a module may contain a syntax error. This is e.g. true of PyQt with “port_v3″ included. These contain Python3 syntax and fail to be imported in Python2, but that should not be considered an error. These modules are now skipped with a warning. Fixed in 0.3.12b already.

- The code to import modules wasn’t using the “__import__” builtin, which prevented “__import__” overriding code to work. Changed import to use the builtin. Fixed in 0.3.12c already.

- The code generated for the “__import__” builtin with constant values was doing relative imports only. It should attempt relative and absolut imports. Fixed in 0.3.12c already.

- The code in “__init__.py” believed it was outside of the package, giving pro blems for package local imports. Fixed in 0.3.12d already.

- It appears that “Scons”, which Nuitka uses internally and transparent to you, to execute the compilation and linking tasks, was sometimes not building the binaries or shared libraries, due to a false caching. As a workaround, these are now erased before doing the build. Fixed in 0.3.12d already.

- The use of “in” and “not in” in comparison chains “a < b < c” is one, wasn’t supported yet. The use of these in comparison chains “a in b in c” is very strange. Only in the “test_grammar.py” it was ever used I believe. Anyway, it’s supported now, solving this TODO and reducing the difference. Fixed in 0.3.12e already.

- The order of evaluation for “in” and “not in” operators wasn’t enforced in a portable way. Now it is enforced on “ARM” too. Fixed in 0.3.12e already.

New Optimizations

- The builtins “GeneratorExit” and “StopIteration” are optimized to their C-API names where possible as well.

Cleanups

- The “__file__” attribute of modules was the relative filename, but for absolute filenames these become a horrible mess at least on Linux.

- Added assertion helpers for sane frame and code objects and use them.

- Make use of “assertObject” in more places.

- Instead of using “os.path.sep” all over, added a helper “Utils.joinpath” that hides this and using “os.path.join”. This gives more readable code.

- Added traces to the “unfreezer” guarded by a define. Helpful in analyzing import problems.

- Some PyLint cleanups removing dead code, unused variables, useless pass statement, etc.

New Tests

- New tests to cover “SyntaxError” and “IndentationError” from “–deep” imports and in main program.

- New test to cover evaluation order of “in” and “not in” comparisons.

- New test to cover package local imports made by  the “__init__.py” of the package.

Organizational

- Droped “compile_itself.sh” in favor of the new “compile_itself.py”, because the later is more portable.

- The logging output is now nicer, and for failed recursions, outputs the line that is having the problem.

 

Numbers

There are no new numbers. Nuitka should be as fast as it was, 258% speedup:

python 2.6:

Pystone(1.1) time for 50000 passes = 0.48
This machine benchmarks at 104167 pystones/second

Nuitka 0.3.11 (driven by python 2.6):

Pystone(1.1) time for 50000 passes = 0.19
This machine benchmarks at 263158 pystones/second

Summary

The frame stack work and the “func_code” are big for compatibility. The “func_code” was also needed for “hg” to work. For “hg” to pass all of its test suite, more work will be needed, esp. the “inspect” module needs to be run-time patched to accept compiled functions and generators too.

Once real world programs like “hg” work, we can use these as more meaningful benchmarks and resume work on optimization.

As always you will find its latest version here.

Yours,
Kay Hayen

 

Posted in compiler, Nuitka, Python | Comments Off

Puting a conference T-shirt to good use

Posted on November 1, 2011 by kayhayen

This is my 7rd old son in his Halloween outfit. It is the first time, he was allowed to do it and so he was very excited.

The link to Python is the T-shirt “Don’t Panic – Use Python”. I got that from PyCON DE, and the family loves it.

Posted in family, Python | Comments Off

Nuitka Release 0.3.12

Posted on October 24, 2011 by kayhayen

This is to inform you about the new release of Nuitka many bug fixes, and substantial improvements especially in the organisational area. There is a new User Manual (PDF), with much improved content, a “sys.meta_path” based import mechanism for “–deep” mode, git flow goodness.

Please see the page “What is Nuitka?” for clarification of what it is now and what it wants to be.

This release is generally also the result of working towards compilation of a real programs (Mercurial) and to get things work more nicely on Windows by default. Thanks go to Liu Zhenhai for helping me with this goal.

Due to the use of the git flow, most of the bugs listed here were already fixed in on the stable release before this release. And there were many of these.

Bug fixes

- The order of evaluation for base classes and class dictionaries was not enforced. Apparently nothing in the test suite did that, I only noticed during debugging that Nuitka gave a different error than CPython did, for a class that had an undefined base class, because both class body and base classes were giving an error. Fixed in 0.3.11a already.

- Method objects didn’t hold a reference to the used class. The effect was only noticed when “–python-debug” was used, i.e. the debug version of Python linked, because then the garbage collector makes searches. Fixed in 0.3.11b already.

- Set “sys.executable” on Linux as well. On Debian it is otherwise “/usr/bin/python” which might be a different version of Python entirely. Fixed in 0.3.11c already.

- Embedded modules inside a package could hide package variables of the same name. Learned during PyCON DE about this corner case. Fixed in 0.3.11d already.

- Packages could be duplicated internally. This had no effect on generated code other than appearing twice in the list if frozen modules. Fixed in 0.3.11d already.

- When embedding modules from outside current directory, the look-up failed. The embedding only ever worked for the compile itself and programs test cases, because they are all in the current directory then. Fixed in 0.3.11e already.

- The check for ARM target broke Windows support in the Scons file. Fixed in 0.3.11f already.

- The star import from external modules failed with an error in “–deep” mode. Fixed in 0.3.11g already.

- Modules with a parent package could cause a problem under some circumstances. Fixed in 0.3.11h already.

- One call variant, with both list and dict star arguments and keyword arguments, but no positional parameters, didn’t have the required C++ helper function implemented. Fixed in 0.3.11h already.

- The detection of the CPU core count was broken on my hexacore at least. Gave 36 instead of 6, which is a problem for large programs. Fixed in 0.3.11h already.

- The inline copy of Scons didn’t really work on Windows, which was sad, because we added it to simplify installation on Windows precisely because of this.

- Cleaning up the build directory from old sources and object files wasn’t portable to Windows and therefore wasn’t effective there.

- From imports where part of the imported were found modules and parts were not, didn’t work. Solved by the feature branch “meta_path_import” that was merged for this release.

- Newer MinGW gave warnings about the default visibility not being possible to apply to class members. Fixed by not setting this default visibility anymore on Windows.

- The “sys.executable” gave warnings on Windows because of backslashes in the path. Using a raw string to prevent such problems.

- The standard library path was hard coded. Changed to run time detection.

New Optimizations

None this time.

Cleanups

- Version checks on Python runtime now use a new define “PYTHON_VERSION” that makes it easier. I don’t like “PY_VERSION_HEX”, because it is so unreadable. Makes some of the checks a lot more safe.

- The “sys.meta_path” based import from the “meta_path_import” feature branch allowed the cleanup the way importing is done. It’s a lot less code now.

- Removed some unused code. We should aim at making Nuitka the tool to detect dead code really.

- Moved “nuitka.Nodes” to “nuitka.nodes.Nodes”, that is what the package is intended for, the split will come later.

New Tests

- New tests for import variants that previously didn’t work: Mixed imports. Imports from a package one level up. Modules hidden by a package variable, etc.

- Added test of function call variant that had no test previously. Only found it when compiling “hg”. Amazing how nothing in my tests, CPython tests, etc. used it.

- Added test to cover the partial success of import statements.

- Added test to cover evaluation order of class definitions.

Organizational

- Migrated the “README.txt” from org-mode to re-structured text, which allows for a more readable document, and to generate a nice user manual in PDF form. The amount of information was increased, with many more subjects are now covered, e.g. git flow and how to join Nuitka. It’s also impressive to see what code blocks and syntax highlighting can do for readability.

- The Nuitka git repository has seen multiple hotfixes. These allowed to publish bugfixes immediately after they were made, and avoided the need for a new release just to get these out. This really saves me a lot of time too, because I can postpone releasing the new version until it makes sense because of other things.

- Then there was a feature branch “meta_path_import” that lived until being merged to “develop” to improve the import code, which is now released on “master” as stable. Getting that feature right took a while.

- And there is the feature branch “minimize_CPython26_tests_diff” which has some success already in documenting the required changes to the CPython26 test suite and in reducing the amount of differences, while doing it. We have a frame stack working there, albeit in too ugly code form. I will polish it for the next release as a separate feature branch and likely merge that already.

- The release archives are now built using “setuptools”. You can now also download a zip file, which is probably more Windows friendly. The intention is to work on a feature branch that will improve “setup.py” to be able to produce a Nuitka install that won’t rely on any environment variables at all. Right now “setup.py” won’t even allow any other options than “sdist” to be given.

- Ported “compile_itself.sh” to “compile_itself.py”, i.e. ported it to Python. This way, we can execute it easily on Windows too, where it currently still fails. Replacing “diff”, “rm -rf”, etc. is a challenge, but it reduces the dependency on MSYS tools on Windows.

- The compilation of standard library is disabled by default, but “site” or “dist” packages are now embedded. To include even standard library, there is a “–really-deep” option that has to be given in addition to “–deep”, which forces this.

Numbers

There are no new numbers. Nuitka should be as fast as it was, 258% speedup:

python 2.6:

Pystone(1.1) time for 50000 passes = 0.48
This machine benchmarks at 104167 pystones/second

Nuitka 0.3.11 (driven by python 2.6):

Pystone(1.1) time for 50000 passes = 0.19
This machine benchmarks at 263158 pystones/second

Summary

Again, huge progress. The improved import mechanism is very beautiful. It appears that little is missing to compile real world programs like “hg” with Nuitka. The next release cycle will focus on that and continue to improve the Windows support which appears to have some issues.

Once real world programs work, we can use these as more meaningful benchmarks and resume work on optimizations.

As always you will find its latest version here.

Yours,
Kay Hayen

Posted in compiler, git, Nuitka, Python | Comments Off

Nuitka – PyCON DE Video

Posted on October 22, 2011 by kayhayen

Hello everybody,

the Video of my presentation is online: http://blip.tv/episode/5632565

Leaving it to Mike Müller to announce here the links to everything, when it’s all finished. Thanks for the good work, the video was very well done.

Yours,
Kay

Posted in compiler, git, Nuitka, Python | Comments Off

Cat update

Posted on October 18, 2011 by kayhayen

The new cat has become indispensable for just about anything. She follows throughout the hous, and is just too lovely, here two photos of her in typical spots.

When I code for Nuitka, she often lays between keyboard and monitors, sleeping, getting caressed by me:

And here is another one, hiding in her cat tree.

Posted in family, Nuitka | Comments Off

My 7yr old fotographer

Posted on October 18, 2011 by kayhayen

This photo was shot by my 7yr old son. It’s truly artful, if not perfect. I have not applied any correction. He has my old camera, and knows how to use it, more and more.

Posted in family | Comments Off

PyCON DE 2011 – My Report

Posted on October 8, 2011 by kayhayen

The PyCON DE 2011 is just over, sprints are still happening over the weekend, but my wife wouldn’t allow me to stay away for that long, so it’s not for me this time. Maybe next time.

Right now I feel very happy and excited that I went there. What a great experience this was.

It was the first German PyCON and clearly it was overdue as it was now the merger of many already grown up communities. A huge number of talks over 3 days in 3 paralell tracks, with 3 keynotes, was an outstanding program. And very well run. Strict time management, every detail was well prepared.

I can only admire the professional preparation and setup. I wanted to say thank you deeply. I didn’t consider it possible to be this good. Clearly not a first time.

I enjoyed the talks, most often in the technical track, but other tracks would have been very interesting too. The parallelism was making me do hard decisions.

Food

The food was great too. I esp. liked the Asian day, but there was also Italian and French, and what many liked very much is that there was a Vegan food offer too. I do not live vegan style, but I appreciate good food and the vegan food often is that.

Social Event

The social event was a visit to a “Varieté” (music hall, French origin) , where I am sure, there will be images posted, I currently found this one, that my wife will find interesting too.

Leipzig

The quality of the organisation, the city “Leipzig”, where we also got to have a guided city tour of fantastic enthusiams, was very high. I knew Leipzig from earlier visits and liked it before, but this time it seemed everybody was even friendlier.

Site

The convention place “Kubus” was very well chosen, absolutely ideal. It’s got good equipment, and that large room setup, where you can make a split with movable walls, and have 3 big screens. The acoustics were pretty damn good there.

My own Presentation

As to my own presentation, it was well received, although I sort of regret that I agreed to have only 30m instead of original plan of 60m. I had so much to say.

I ended up with getting my manifesto part out, but that one pretty well. And it’s OK I guess, because nobody really listens that long anyway. And my major points came across that way.

Tjat focus on my Nuitka “manifesto” was probably a good idea. The talk will be available online as a video, I will link it then. The PDF that I presented only a small part of, is linked here. I believe it went pretty well.

I will use that content from the PDF in updated documentation (currently ongoing in PDF is work to use REST and document a lot more). The presentation was created with rst2pdf, which I find is a fantastic tool

Contacts

Cython / lxml

Then contacts!

Early on I already made contacts with interesting people, e.g. with Dr.Stefan Behnel, author of lxml and core Cython developer. I him offered a beer for using his software in the best of free software traditions. He doesn’t drink these, but a large mango counts
too or so I assume.

We also talked about Cython and Nuitka, and the common history we had as well. For some time, I attempted to change Cython, but that failed to get the developers support at the time. Not wanting to deviate from PyRex clearly isn’t the state anymore, but that was then.

We also had a evening session of showing each other the good and bad parts, comparing was quite fun. And it was quite interesting to the both of us. I believe we made friends and will only benefit another.

We discussed my goals, and I think we came to the conclusion that they are in fact different enough from Cythons. Although I go away with the sense, that of course Stefan believes, it would be better if I joined Cython. Naturally.

But that’s not going to happen. I think i have a cleaner and better implementation now, closer to my goals with a realistic chance to succeed. To me it would be a step back to fix language parsing issues and incompatabilities of Cython, with the danger that my goals will not be shared.

As an example of these things, I would mention function call errors, where e.g. Cython gives different and sometimes worse error messages than CPython, and I designed the code so that it does things in that same order than CPython does.

It do not want to give different error messages, and who knows, somebody may check for the exception text and expect CPython output. In this case, I will rather accept a worse performance, than an incompatibility.

Eliminating function parameter parsing for the whole program as far as possible is going to be more worthwhile anyway.

But in my mind, Cython is something I can and do recommend. For as long as I am not able to declare Nuitka “useful” yet. That statement may come within a year though. In my mind, in many fields Nuitka is already superior.

PyHasse

Another interesting contact I made, was with the author of PyHasse. It’s Rainer Bruggemann, who is a really nice and witty guy. He introduced me to how he applies graph theory to multi-parameter optimization problems.

We agreed that we will try and work together on this project. Hopefully it will come to pass. One thing I personally wanted, was to get into contact with people who understand or are part of the scientific community.

I can see what NumPy is. But I may never know myself what it really is, unless I find proxies, and make these kind of contacts. The same thing is true of Django, or e.g. Mercurial. I am positive though that with time, and such conferences, my knowledge of these will only grow.

We said that we will try and see how far we can go. In the worst case, Nuitka will not yet be useful, but I will have a clearer image what is needed.

Debian

I saw the presentation from Jan Dittberner and met him later too, asking him questions, and generally discussing Debian packaging of Nuitka. He encouraged me to contact the Debian Python Team, and so I will.

I used the chance to make contact with a Debian guy, who made a presentation on how to package Python modules for Debian. He gave me hints on how to solve that “find files near me” issue that plagues Nuitka just as much as other software. Really kind and helpful guy and clearly I admire Debian Developers, keep up the good work.

General

I also made contacts with lots of other people. Python is diverse and it was fun to get to know, many people with similar and entirely different backgrounds.

The mood was extremely constructive. Nuitka was well received, but that’s not why I say it. There is that general sense of respect around that German community, you can feel how pretty much everybody is well established and doesn’t have to disprove the others.

Keynotes

One keynotes speaker had a part about how trolling and hate is bad for
a community, but that’s not the the German Python community.

Another keynote speaker (Paul Everitt) had a part about how Zope, which was kind of his project, failed in many ways. He seemed to be quite disappointed about that, which triggered me to point out, that he should start his story with Apache, and not see the “failure to integrate” as a failure.

If there had not been Apache failing, there wouldn’t have been Zope, and then not Django, etc. that’s kind of normal and actually good. He agreed and pointed out how Apache was created from another project that had failed to integrate people.

You either fork a projects code, or ideas. The fork still should credit and appreciate the predecessor/origin.

In my mind, Cython failed to integrate me. Which triggered me to come up with Nuitka, and as I will point out over time (there ought to be postings and there probably will be), some better approaches.

So not integrating me is not necessarily a failure. If it were not for Cython, there would not be Nuitka. The original projects will regret the fork/remake, but they probably shouldn’t. Competition is good.

Lets repeat that

I believe the PyCON DE 2011 was a huge success. I will most definitely go again to update people on Nuitka. It’s already clear there will be a PyCON DE 2012 I understand. And I am aiming for a slot at PyCON EU 2012 next year too. I wanted to go in 2011, but need to not put it in my early booked holiday again. But you know what Murphy says about that.

Yours,
Kay Hayen

Posted in Nuitka, Python | Comments Off