gmane.comp.python.cython.user

Howto declare local variable ndarray[double, ndim=2] in < at >locals?

Zaur Shibzukhov — 2013-05-24T19:25:45

Currently I can declare numpy 1d and 2d arrays both in args and local vars 
using syntax: ndarray[double, ndim=1] or ndarray[double, ndim=2].

When I write function in .py file and trying to use .pxd for type 
annotations I can declare 1d and 2d numpy arrays in args. For example:

cdef ndarray[double, ndim=1] func(ndarray[double, ndim=2] X, 
ndarray[double, ndim=1] y)

But I don't know howto define in function local var with a such 1d/2d numpy 
array using < at >locals decorator.

Is it doable?

OSX 32-bit Cython

dbv — 2013-05-24T13:29:31

How would you build a 32-bit version of Cython on a 64-bit OSX machine?

Appropirate method to expose an internal c-array to Python and vice versa

Ghislain Vaillant — 2013-05-24T08:09:30

Hi everyone,

I know there is a wide number of posts regarding ways to expose internal 
C-buffers to Python, but I can't my head around a slightly more advanced 
design. Say I have a structure declared that way:

struct A{
    size_t n
    float* f
<some other stuff maybe>
}

and initialized using a function A_init which let me choose whether I can 
provide a pointer to an external array or malloc it. How can I reproduce 
this behaviour in wrapper class ? Right now, I am doing something like:

cdef class A_wrapped
   A __myA
   object _f
 
and in A_wrapped.__cinit__(self, *args), I force __myA.f to be malloc'd and 
then expose it using:

self._f = PyArray_SimpleNewFromData(1, __myA.size_t, <void*>__myA.f)

However, I now want to be able to specify an optional numpy array f_ext to 
the constructor, whose data pointer would be used for __myA.f instead of 
malloc'd. Besides the usual checks whether f_ext is contiguous, has got the 
right size and type, is there anything else I should be aware of ? What 
happens

Elaborating on the Rectangle example from the docs

Jan Domański — 2013-05-19T19:53:08

Hi, 

Say Rectangle.h has an intersectRectangles function that, given two 
rectangles, will return a rectangle with the area shared by the two rects.

public:
   ...

   void move(int dx, int dy);


Rectangle intersectRectangles(Rectangle* rec1, Rectangle* rec2);

How could one wrap this function in cython? I've tried with the following:

def PyIntersectRectangles(PyRectangle r1, PyRectangle r2):
  Rectangle rec = intersectRectangles(deref(r1.thisptr), deref(r2.thisptr))
  pyrec = PyRectangle()
  del pyrec.thisptr
  pyrec.thisptr = &rec
  return pyrec

The code above gives me a core dump, and I think I understand why, but I see no clear solution.

Cython newbie: low performance gain in recursive code (fully typed)

wieland — 2013-05-20T18:51:05

Dear all,
today I gave Cython a try to speed up some performance critical code of 
mine that has lots of potential:

for i in arange(0,dim)[::-1]: 

    HY = dot(H[i,i:],Y[i:,:])

    for j in arange(0,dim)[::-1]:

       Y[i,j] = (T[i,j] + dot(HY[j:],HH[j:,j]))/(1-H[i,i]*HH[j,j])

       HY[j] += H[i,i]*Y[i,j]


 return Y

This code is very simple but hard to vectorize any further due to the 
recursive interplay between Y and HY. Now, since it is hard to vectorize I 
thought it should have lots of potential once written in C. So I turned to 
Cython and wrote the following .pyx-file:

#cython: boundscheck=False 

#cython: wraparound=False


import numpy as cnp

cimport numpy as cnp


def c_schur_method(cnp.ndarray[double complex, ndim=2] H , 
cnp.ndarray[double complex, ndim=2] HH, cnp.ndarray[double complex, ndim=2] 
T): 

    cdef unsigned int i, ii, j, jj

    cdef int dim = H.shape[0]

    cdef cnp.ndarray[cnp.complex128_t, ndim=1] HY = cnp.zeros(dim, 
dtype=cnp.complex128)

    cdef cnp.ndarray[cnp.comp

example of on-the-fly compilation and module loading

Allen Riddell — 2013-05-22T15:27:45

Hi cython-users,

Does anyone know of an example of a cython project that does run-time
compilation of generated cython code (along with c/c++ code) and then loading of
the module? I've seen something that uses ctypes but haven't come across
a cython example.

Thanks,

Allen

What code speeds up the most when Cythonized?

Zak — 2013-05-23T17:14:55

Hello Cython List,

I am curious, what code speeds up the most when Cythonized? A specific code 
example would be good. I think I am using the word Cythonize in the 
accepted fashion, but I am new to the community. By Cythonize, I mean 
converting .py files into .pyx files and then adding static type 
declarations to the code.

I am just curious, I want to see what speeds up the most, and thus perhaps 
gain a better understanding of how CPython and Cython each work internally.

How much faster have your projects gotten? I have done two projects in 
Cython. One project was statistical cross-correlation, but I don't know the 
speed-up factor because I wrote it in Cython from the beginning. The other 
project was numerical approximation, it had a 16x speed-up when I used 
"except? -1" in my cdef functions and a 20x speedup when I omitted 
exception handling altogether. I was disappointed to see only 20x speed-up.

The Cython documentation gives an example of numerically integrating a 
quadratic equation and it

Wrapping a factory method returning a shared_ptr

Wichert Akkerman — 2013-05-23T20:59:24

I have a fairly typical API where I have a public pure virtual class with a 
factory method that returns an implementation. It looks like this:

class User {
public:
    static boost::shared_ptr<User*> create() ;
};

I can't find any hints in the documentation how you would wrap this in 
Cython.

Wichert.

Best practice for using pkg-config

Wichert Akkerman — 2013-05-23T21:05:54

Is there a best practice for using pkg-config? I was trying to use 
cythonize since the documentation suggests this is the current preferred 
method, but there doesn't appear to be any API documentation listing its 
parameters. The only working thing I could come up with is to manually 
modify the extensions it returns:

from distutils.core import setup
import subprocess
from Cython.Build import cythonize

flags = subprocess.check_output(['pkg-config', '--cflags-only-I', 'pkgA 
pkgB'])
include_dirs = [flag[2:] for flag in flags.split()]
flags = subprocess.check_output(['pkg-config', '--libs-only-L', 'pkgA 
pkgB'])
library_dirs = [flag[2:] for flag in flags.split()]

extensions = cythonize('*.pyx')
for ext in extensions:
    ext.include_dirs.extend(include_dirs)
    ext.library_dirs.extend(library_dirs)

setup(name='mypkg', ext_modules=extensions)

This feels unnecessarily complex and definitely is not readable. It would 
be very nice if you could do this:

setup(
    name='mypkg',
    ext_modules=cythonize('

Compiler crash in AnalyseExpressionsTransform

Czarek Tomczak — 2013-05-22T17:18:53

Hi,

I'm getting this crash while compiling with Cython:

web_request.pyx:54:19: Compiler crash in AnalyseExpressionsTransform


The code is:

cdef cpp_map[int, void*] g_pyWebRequests


PyWebRequest is a cdef class.

What to do?

A side note:
I have to keep PyWebRequests in C++ map as I do not want to increase 
references
to these objects, I need to access them from C++ and I do not have control 
over
their lifetime, keeping them in a Python dict would make them live forever 
and the
memory would never be freed.

Using latest Cython 0.19.1

Thanks for help.

Regards,
Czarek

How to set compiler directive in setup.py?

Czarek Tomczak — 2013-05-22T17:00:35

Hi,

I'm trying to set c_string_type and c_string_encoding compiler directives
in setup.py file, but it does not work, here is the code:

from Cython.Compiler import Options


On the other hand, setting the "fast_fail" option works fine:

Options.fast_fail = True


But seems like this is a different type of option, not listed in the 
"compiler 
directives" in the documentation here:
http://docs.cython.org/src/reference/compilation.html#compiler-directives

Setting c_string_type and c_string_encoding by adding this comment at
the top of the pyx file works fine:

# cython: c_string_type=str, c_string_encoding=utf8


But I need to set these options depending on python version, so this is not
going to work.

Regards,
Czarek

Trying to wrap SFML Window; getting "ImportError: No module named 'ExprNodes'"

Goran Milovanovic — 2013-05-21T07:51:07

# sfml.pxd:

cdef extern from "SFML/Window.hpp" namespace "sf":

cdef cppclass VideoMode:
VideoMode(unsigned int, unsigned int) except +

cdef cppclass Window:
Window(VideoMode, String) except + 
void display()



# display.pyx:

cimport sfml

cdef class Window:

cdef sfml.Window* _this

def __cinit__(self, unsigned int width, unsigned int height):

self._this = new sfml.Window(sfml.VideoMode(width, height), "title")
 def __dealloc__(self):

del self._this

def display(self):
 self._this.display()


# setup.py:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [
Extension("display", ["display.pyx"],
language='c++',
libraries=["sfml-system", "sfml-window"])
]
)


The error when running `python setup.py build`:

running build
running build_ext
cythoning display.pyx to display.cpp
Traceback (most recent call last):
  File "setup.py", line 10, in <module>
    libraries=["sfml-system", "sfml

compiler warning - pointer difference assumed to be int

Josh Ayers — 2013-05-20T18:19:05

When compiling under Windows using 64-bit Visual Studio 2008, I get a C 
compiler warning associated with line 798 of numpy.pxd.  It complains that 
an __int64 is being converted to a long.  Here's the relevant section of 
numpy.pxd (with a few unimportant lines removed for clarity).

cdef inline char* _util_dtypestring(dtype descr, char* f, char* end, int* offset) except NULL:

    cdef dtype child
    cdef tuple i
    cdef tuple fields

    for childname in descr.names:
        fields = descr.fields[childname]
        child, new_offset = fields

        if (end - f) - (new_offset - offset[0]) < 15: # compiler warning on this line

Looking at the generated C code, PyInt_FromLong is called on (end - f), 
which is the difference of two pointers.  If I manually change the 
conversion function to PyInt_FromSsize_t, the warning goes away.

I believe the root problem is that Cython assumes a difference of two 
pointers can be stored in an int.  Here's another short example that gives 
the same warning.  

def ptr

About local variable definitions in functions

Zaur Shibzukhov — 2013-05-20T06:36:09

Let's consider use case. One is use .py file and .pxd file for type 
annotation.
Currently if someone need local variable definitions he uses < at >locals(...) 
decorator.

I'm wonder if Cython syntac could be extended to allowing something like 
following inside of .pxd:

cdef int func(int a):
   cdef int i, j
   cdef bint flag


Is it a good idea?

About local variable definitions in functions

Zaur Shibzukhov — 2013-05-20T06:35:45

Let's consider use case. One is use .py file and .pxd file for type 
annotation.
Currently if someone need local variable definitions he uses < at >locals(...) 
decorator.

I'm wonder if Cython syntac could be extended to allowing something like 
following inside of .pxd:

cdef int func(int a):
   cdef int i, j
   cdef bint flag


Is it a good idea?

Cythonize is special?

Shriramana Sharma — 2013-05-19T11:46:05

Hello. How exactly is:

from distutils.core import setup
from Cython.Build import cythonize
setup(
    name = "My hello app",
    ext_modules = cythonize('hello.pyx'), # accepts a glob pattern
)

different from:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup (
ext_modules = [ Extension ( "hello", [ "hello.pyx" ] ) ],
cmdclass    = { "build_ext" : build_ext }
)

Is it just that the former is somewhat shorter, especially seeing as
the docstring for cythonize says: ""Compile a set of source modules
into C/C++ files and return a list of distutils Extension objects for
them.""

Or is there any other essential difference that makes using Cythonize
preferable to manually specifying the Extension params?

Thanks.

Win7 Python 2.7 Cython helloworld example compile error ! help ,please

杨雪峰 — 2013-05-18T15:54:45

Hi all:

     I am a new user of Cython. I install it by the Anaconda package. When 
i try the example on the website, i get the following problem !

Setup.py 
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

setup(
    cmdclass = {'build_ext': build_ext},
    ext_modules = [Extension("helloworld", ["helloworld.pyx"])]
)

the pyx file is just a print 'hello world' 

the Error :

running build_ext
skipping 'helloworld.c' Cython extension (up-to-date)
building 'helloworld' extension
C:\MinGW\bin\gcc.exe -DMS_WIN64 -mdll -O -Wall -IC:\Anaconda\include 
-IC:\Anaconda\PC -c helloworld.c -o 
build\temp.win-amd64-2.7\Release\helloworld.o
writing build\temp.win-amd64-2.7\Release\helloworld.def
creating build\lib.win-amd64-2.7
C:\MinGW\bin\gcc.exe -DMS_WIN64 -shared -s 
build\temp.win-amd64-2.7\Release\helloworld.o 
build\temp.win-amd64-2.7\Release\helloworld.def -LC:\Anaconda\libs 
-LC:\Anaconda\PCbuild\amd64 -lpython27 -lmsvcr90 -o 
build\lib.win-amd

Cython c compiler setting

杨雪峰 — 2013-05-18T18:22:57

Hi all

       I follow the instruction 
from http://wiki.cython.org/64BitCythonExtensionsOnWindows  try to use 
windows c compiler to build python extension. but cython are still using 
mingw gcc to compile my files which cause many undefined reference errors.

      it became green words after i command setenv /x64 /release 

C:\Users\Yang 
Xuefeng\Dropbox\workspaces\pythonxy\autoencoders\Cythonlearning\te 
st>python setup.py build_ext running build_ext skipping 'helloworld.c' 
Cython extension (up-to-date) building 'helloworld' extension 
C:\MinGW\bin\gcc.exe -DMS_WIN64 -mdll -O -Wall -IC:\Anaconda\include 
-IC:\Anacon da\PC -c helloworld.c -o 
build\temp.win-amd64-2.7\Release\helloworld.o writing 
build\temp.win-amd64-2.7\Release\helloworld.def C:\MinGW\bin\gcc.exe 
-DMS_WIN64 -shared -s build\temp.win-amd64-2.7\Release\hell oworld.o 
build\temp.win-amd64-2.7\Release\helloworld.def -LC:\Anaconda\libs -LC: 
\Anaconda\PCbuild\amd64 -lpython27 -lmsvcr90 -o 
build\lib.win-amd64-2.7\hellowor ld.pyd 
build\tem

First Cython ModuleI

Aidin Hasanzadeh — 2013-05-18T12:05:42

Hey,

I am new with Cython and starting my first module. I am using 
eclipse/Ubunty. I installed cython and tried to implement the very first 
example, Hello world.

http://docs.cython.org/src/userguide/tutorial.html

I made the setup file, but importing Cython.Distutils could not be resolved.

from distutils.core import setupfrom distutils.extension import Extensionfrom Cython.Distutils import build_ext
setup(
    cmdclass = {'build_ext': build_ext},
    ext_modules = [Extension("helloworld", ["helloworld.pyx"])])



Thanks for any help
Aidin

Type error

Pedro Camargo — 2013-05-19T00:33:36

Hi guys,

                  I'm working on a fairly long code that is compiling
correctly, but giving an error when it runs.

 

                I'm working on Windows 64 bits, Python 2.7 (last installer
made available) and the last versions of the installers for
Numpy/Scipy/Cython  made by Chris Gohlke.

 

Thanks for the help,

                

Pedro

 

The error is:

 

File "Assignment2.pyx", line 493, in Assignment._Ordering_and_Forward_Star
(Assignment2.c:6242)

    cdef _Ordering_and_Forward_Star(np.ndarray[ITYPE_t, ndim=2, mode='c']
graph, #A_Node/B_Node

ValueError: Buffer dtype mismatch, expected 'ITYPE_t' but got 'long long'

 

 

 

It points to this header:

 

cdef _Ordering_and_Forward_Star(np.ndarray[ITYPE_t, ndim=2, mode='c'] graph,
#A_Node/B_Node

                    np.ndarray[ITYPE_t, ndim=2, mode='c'] g_aux,

                    np.ndarray[DTYPE_t, ndim=2, mode='c'] graph_data,

                    np.ndarray[DTYPE_t, ndim=2, mode='c'] g_aux2,

                    np.ndarray[ITYPE_t,

for loop with variable step not optimized

Josh Ayers — 2013-05-19T00:01:09

I just noticed that a for-loop using the standard Python range construct 
isn't optimized to C-code if the step is a run-time variable.  It is 
optimized if the step parameter is a compile-time constant, or if the old 
Pyrex style for-loop syntax is used.  I tested with Cython 0.18 and 0.19.1 
and both behaved the same way.

Here's a minimal example.

DEF STEP = 2

def loops():
    cdef int i, start = 0, stop = 10, step = 2

    for i in range(start, stop, step): # generates Python loop
        print(i)

    for i in range(start, stop, STEP): # generates C loop
        print(i)

    for i from start <= i < stop by step: # generates C loop
        print(i)

Is this a bug?

Thanks,
Josh Ayers