Distutils is the standard way to build Cython extension modules for large projects. Typically one invokes setup(…) which then indirectly invokes Cython when a
.pyx source file is encountered. This limits the control we have over the
c compilation process. This proposal is to make a standard Cython method that “preparses” the list of Extension modules and converts all
.pyxsource files to
.c(pp) ones before passing them off to distutils. Doing this step ourselves, all at once, would have several advantages including:
This will resolve other nuisances such as the difficulty in supporting parallel Pyrex and Cython installs or relying on distutils to know about Cython at all. Disentangling the Cythonization phase from the compile process and using in-file directives will allow easier support for other build systems as well.
Rather than writing a setup.py that looks like
from distutils.core import setup from distutils.extension import Extension from Cython.Distutils import build_ext ext_modules=[ Extension("primes", ["primes.pyx"]), Extension("spam", ["spam.pyx"]), ...
one would write
from distutils.core import setup from Cython.Build import cythonize setup( name = 'MyProject', ext_modules = cythonize(["*.pyx"]), )
The cythonize command takes a list of filenames/filename patterns and returns a list of Extension classes ready to be compiled by distutils, using timestamps and its understanding of Cython’s
cimport statements to compile all and only the necessary extensions. It is also permissible to do:
ext_modules=[ Extension("primes", ["primes.pyx"]), Extension("spam", ["spam.pyx"], libraries = ["spam"]), ... ] setup( name = 'MyProject', ext_modules = cythonize(ext_modules), )
however, the preferred way to specify distutils options is directly in the .pyx or .pxd file itself, via a comment similar to how module-level cython directives are given. For example, one can write:
# distutils: language = c++ # distutils: libraries = spam eggs # distutils: include_dirs = /opt/food/include
When such declarations appear in
.pxd files, they are transitive (if it makes sense), so modules cimporting from them will automatically propagate the right compile flags to use them as well.
The cythonize command takes several options, including an
nthreads parameter to use multiprocessing to compile
.pyx files in parallel.
sys.argvto know whether or not to invoke Cython.