gmane.comp.python.cython.user

Re: Re: Error compiling - "Assignment to reference" - in Cython 0.18

Czarek Tomczak — 2013-05-22T18:36:34

Using the "(&out)[0] = " fix for referenced arguments works great.
Thank you so much Nikita for this fix, I can use latest version of
Cython again (it's still not fixed in Cython 0.19.1).

Best regards,
Czarek

On Thursday, April 25, 2013 10:15:02 PM UTC+2, Nikita Nemkin wrote:

Re: Compiler crash in AnalyseExpressionsTransform

Czarek Tomczak — 2013-05-22T18:12:35

Nikita, thank you for the guide on weak references, I will take a look at 
it.
Right now I'm removing objects from a C++ map in the __dealloc__ method
of the cdef class, which is called when object lifetime ends, so it works.

Regards,
Czarek

On Wednesday, May 22, 2013 7:50:01 PM UTC+2, Nikita Nemkin wrote:

Re: Compiler crash in AnalyseExpressionsTransform

Czarek Tomczak — 2013-05-22T18:07:00

Replacing Cython.Shadow.void* with libc.stdint.uintptr_t in the C++ map 
gets rid of the compiler crash, I will just use this fix.

Regards,
Czarek

On Wednesday, May 22, 2013 7:18:53 PM UTC+2, Czarek Tomczak wrote:

Re: Compiler crash in AnalyseExpressionsTransform

Nikita Nemkin — 2013-05-22T17:50:01

On Wed, 22 May 2013 23:18:53 +0600, Czarek Tomczak  
<czarek.tomczak< at >gmail.com> wrote:


Weak references exist specifically for this purpose.
They are also much safer, because they are cleared automatically
when the pointed object dies.
Take a look at WeakValueDictionary in the standard lib.

Note: cdef classes need an additional member ("cdef __weakref__")
in order to support weak references.
(http://docs.cython.org/src/reference/extension_types.html#weak-referencing)

Language-level weakref support and weakref optimizations are on my
personal Cython wishlist...

Best regards,
Nikita Nemkin

Re: How to set compiler directive in setup.py?

Czarek Tomczak — 2013-05-22T17:56:32

Importing Cython.Distutils.Extension instead of distutils.Extension fixed
the problem, it works now, thank you for your help.

Best regards,
Czarek

On Wednesday, May 22, 2013 7:33:32 PM UTC+2, Nikita Nemkin wrote:

Re: How to set compiler directive in setup.py?

Nikita Nemkin — 2013-05-22T17:33:32

On Wed, 22 May 2013 23:29:53 +0600, Czarek Tomczak  
<czarek.tomczak< at >gmail.com> wrote:


You should be using Cython's version of Extension class
for extra parameters to have effect:

     from Cython.Distutils import build_ext, Extension


Best regards,
Nikita Nemkin

Re: How to set compiler directive in setup.py?

Czarek Tomczak — 2013-05-22T17:29:53

Hi Nikita,

In setup.py I'm using Extension() syntax, I've tried "cython_directives",
"compiler_directives" and "pyrex_directives", but none of it works, here
is the code:

ext_modules = [Extension(


Best regards,
Czarek

On Wednesday, May 22, 2013 7:13:37 PM UTC+2, Nikita Nemkin wrote:

Re: How to set compiler directive in setup.py?

Nikita Nemkin — 2013-05-22T17:13:37

On Wed, 22 May 2013 23:00:35 +0600, Czarek Tomczak  
<czarek.tomczak< at >gmail.com> wrote:


Hmm, c_string_* are compiler directives and they are listed in the table
you link to.

cythonize has "compiler_directives" parameter:

     ext_modules = cythonize(
         ...
         compiler_directives={
             'c_string_type': 'str',
             'c_string_encoding: utf8,
         },
         ...)

If you are using build_ext instead of cythonize, set "cython_directives"  
attribute
(dict) on Extension object.

As I understand it:
* Directives have a scope (module, class, etc), sometimes can be overriden
   locally and often change language semantics.
* Options pertain to the compilation process as a whole and (usually) do  
not
   change language semantics.

Options can be passed to cythonize as keyword arguments.

Best regards,
Nikita Nemkin

Compiler crash in AnalyseExpressionsTransform

Czarek Tomczak — 2013-05-22T17:18:53

Hi,

I'm getting this crash while compiling with Cython:

web_request.pyx:54:19: Compiler crash in AnalyseExpressionsTransform


The code is:

cdef cpp_map[int, void*] g_pyWebRequests


PyWebRequest is a cdef class.

What to do?

A side note:
I have to keep PyWebRequests in C++ map as I do not want to increase 
references
to these objects, I need to access them from C++ and I do not have control 
over
their lifetime, keeping them in a Python dict would make them live forever 
and the
memory would never be freed.

Using latest Cython 0.19.1

Thanks for help.

Regards,
Czarek

How to set compiler directive in setup.py?

Czarek Tomczak — 2013-05-22T17:00:35

Hi,

I'm trying to set c_string_type and c_string_encoding compiler directives
in setup.py file, but it does not work, here is the code:

from Cython.Compiler import Options


On the other hand, setting the "fast_fail" option works fine:

Options.fast_fail = True


But seems like this is a different type of option, not listed in the 
"compiler 
directives" in the documentation here:
http://docs.cython.org/src/reference/compilation.html#compiler-directives

Setting c_string_type and c_string_encoding by adding this comment at
the top of the pyx file works fine:

# cython: c_string_type=str, c_string_encoding=utf8


But I need to set these options depending on python version, so this is not
going to work.

Regards,
Czarek

Re: Cython c compiler setting

刘振海 — 2013-05-22T14:36:30

2013/5/20 Chris Barker - NOAA Federal <chris.barker< at >noaa.gov>



    the ms compiler is called msvc

Re: OSX 10.8: setuptools generates gcc command that wants to use to use .c instead of .cpp

Justin Israel — 2013-05-22T09:23:26

Hmm... Am I crazy or is this a bug (cython 0.19) in cythonize?

Example cythonize command:

    ext_list = cythonize(['src/plow.pyx'], language="c++", force=True)
    ext = ext_list[0]
    print e.sources  #['src/plow.c']
    print e.language  # None

From what I examined in Cython/Build/Dependencies.py, it looks like it does not maintain the language flag being passed in at all:

    from Cython.Build.Dependencies import *
    options = {'language': 'c++'}
    # at this point in the original source code, even though it created both a c and cpp options object
    # it calls create_extension_list with the c version. For testing, I am explicitly passing the cpp version
    cpp_options = CompilationOptions(**options); cpp_options.cplus = True
    ctx = cpp_options.create_context()
    module_list = create_extension_list(['src/plow.pyx'], ctx=ctx)
    m = module_list[0]
    print m.language  # None
    print m.sources  # ["src/plow.pyx"]

The language value gets lost when it is parsing the string, finding everyt

Re: OSX 10.8: setuptools generates gcc command that wants to use to use .c instead of .cpp

Justin Israel — 2013-05-22T05:31:31

Awesome. I am going to try out replacing with the cythonize command.


On Wed, May 22, 2013 at 5:08 PM, Robert Bradshaw <robertwb< at >gmail.com> wrote:

Re: OSX 10.8: setuptools generates gcc command that wants to use to use .c instead of .cpp

Robert Bradshaw — 2013-05-22T05:08:20

I'm not sure what exactly the issue is, but the line

    cmdclass['build_ext'] = build_ext

replaces the built-in build_ext with the Cython-provided one. Instead
I would reccommend using the cythonize() command, found at
http://docs.cython.org/src/reference/compilation.html (to switch
between the options, you could provide a fake_cythonize that simply
replaces each .pyx extension with .cpp).



On Tue, May 21, 2013 at 2:30 PM, Chris Barker - NOAA Federal
<chris.barker< at >noaa.gov> wrote:

Re: Re: compiler warning - pointer difference assumed to be int

Robert Bradshaw — 2013-05-22T03:44:55

In Py2, there are two integer types, one for values that fit in a long
(called int) and the other for arbitrary precision values (long). The
former are faster to construct and work with.

- Robert

Re: OSX 10.8: setuptools generates gcc command that wants to use to use .c instead of .cpp

Chris Barker - NOAA Federal — 2013-05-21T21:30:25

I wonder if it's something funky with Apple's distutils setup.

I'd have them try the python.org binary Python.

In any case, it doesn't sound like a Cython thing to me, but hard to
tell without getting my hands on it.

-Chris

Re: OSX 10.8: setuptools generates gcc command that wants to use to use .c instead of .cpp

Justin Israel — 2013-05-21T20:22:06

Thanks for the reply!

It would be the apple python 2.7.x that ships with 10.8

Cython is doing the right thing in terms of cythonizing the file, but
without delving into the source I had assumed the custom Extension class
that is being used from cython was doing something wrong. I figured it was
ignoring that language=c++ flag and returning the wrong output source type
to be compiled?

The reason it works with the workaround is that I don't let the cython
Extension handle the conversion. I just explicitly tell it to use the
pregenerated .cpp source. At that point the right gcc command is used.

You are correct that when it fails under the normal cythonizing approach,
it does not get to the linker step with g++. That is because it failed at
the first gcc compile of the cythonized source file, which it thinks should
be a .c file and not .cpp
On May 21, 2013 3:41 AM, "Chris Barker - NOAA Federal" <
chris.barker< at >noaa.gov> wrote:

Re: Re: compiler warning - pointer difference assumed to be int

Josh Ayers — 2013-05-21T17:55:22



Awesome, thanks for the fix.  As I suspected, this one was over my head.

If you don't mind the noob question, can I ask why you used PyInt_FromLong 
instead of PyInt_FromSsize in __Pyx_PyInt_FromPtrdiff_t?  Is it because 
there is no SSIZE_MIN and _MAX that would let you check when it's within 
range for the latter function?

Thanks again,
Josh

Re: Re: compiler warning - pointer difference assumed to be int

Robert Bradshaw — 2013-05-21T16:39:48

I don't know of any modern systems where ptrdiff_t != ssize_t, though
it is allowed by the C standard.


It's part of ANSI C (89), so it should be.

https://github.com/cython/cython/commit/3087d7fd4f78514f3e962a321b5c865dafe859d9
https://github.com/cython/cython/commit/1fcaf85398e4a3ee9d4d77fc0647fdf9984b1d5b

Re: compiler warning - pointer difference assumed to be int

Josh Ayers — 2013-05-21T16:32:20



Right, but to convert it to a Python integer, that function is the only 
reasonable option that I can find.
 

 
Is ptrdiff_t always available?  If so, then that seems like the right 
choice.  Either way, my main point was that Cython should infer something 
other than an int, which is incorrect on any 64 bit system.

Re: compiler warning - pointer difference assumed to be int

Lawrence Mitchell — 2013-05-21T12:03:49

[...]


Ssize_t isn't necessarily large enough to hold the difference of
two pointers.



FWIW, the correct type for storing a pointer difference is
ptrdiff_t.