gmane.comp.python.devel

Re: VS 11 Express is Metro only.

2012-05-24T22:36:47

I think it's too early to guess what the final release of Visual Studio
11 Express will or will not include.

Regards,
Martin

Re: VS 11 Express is Metro only.

Brian Curtin — 2012-05-24T22:26:50

I don't know. Maybe?

Windows 8 and VS11 are still not released so who knows what will happen.

VS 11 Express is Metro only.

Terry Reedy — 2012-05-24T22:21:33

The free Visual Studio 11 Express for Windows 8 (still in beta) will 
produce both 32 and 64 bit binaries and allow multiple languages but 
will only produce Metro apps. For desktop apps, either the paid Visual 
Studio versions or the free 2010 Express releases are required.
https://www.microsoft.com/visualstudio/11/en-us/products/express
bottom of page.

Will this inhibit someday moving to Visual Studio 11 Professional or 
would VS2010 Express or VC++2010 Express still work for hacking on 
Python or making extensions that would work with any VS11-produced binary?

Re: An infinite loop in dictobject.c

Nick Coghlan — 2012-05-24T21:28:12

You may also want to give Victor's faulthandler module a try:
http://pypi.python.org/pypi/faulthandler/

Cheers,
Nick.

Re: An infinite loop in dictobject.c

Terry Reedy — 2012-05-24T21:27:11

The nosy list box search only shows people with commit privileges. 
Others you have to find them in the User List accessible from the 
sidebar, but that may be admin only for all I know. Anyway, Mark should 
have said 'as Mark.Shannon'. I have added him on the issue.

Re: An infinite loop in dictobject.c

Daniel Farina — 2012-05-24T20:23:59

Quite sure.  It was vanilla Ubuntu, and then I side-graded it to
vanilla ubuntu at -O0.


Occurs on a smattering of a large number of systems more or less at
random. Seems unlikely.


Yes, this is my next step, although I am going to do a bit more
whacking of the interpreter as to pause rather than crash when it
encounters this problem.

Re: An infinite loop in dictobject.c

Barry Warsaw — 2012-05-24T19:56:18


I do not think the hash randomization patch has been pushed to Python 2.6 in
Lucid 10.04.4 yet, which still has Python 2.6.5 (plus patches, but not that
one).

-Barry

Re: An infinite loop in dictobject.c

Daniel Farina — 2012-05-24T20:20:34

http://bugs.python.org/issue14903

However, I cannot add you to the nosy list, as you do not show up in the search.

Re: An infinite loop in dictobject.c

Antoine Pitrou — 2012-05-24T20:15:52

On Thu, 24 May 2012 12:11:58 -0700
Daniel Farina <daniel< at >heroku.com> wrote:

Do you mean it's a hand-compiled Python? Are you sure you didn't
recompile it / update to a later version recently?

If you didn't change anything, this may be something unrelated to
Python, such as a hardware problem.

You are right that ma_fill == ma_used should, AFAIK, never happen.
Perhaps you could add conditional debug statements when that condition
happens, to know where it comes from.

Furthermore, if this is a hand-compiled Python, you could reconfigure
it --with-pydebug, so as to enable more assertions in the interpreter
core (this will make it quite a bit slower too :-)).

Regards

Antoine.

Re: An infinite loop in dictobject.c

Daniel Farina — 2012-05-24T20:15:43

I do not think so; I do not see in in the backpatches made to 2.6.5
(http://packages.ubuntu.com/lucid/python2.6), unless they are
particularly slick.

Re: PEP 420 - dynamic path computation is missingrationale

Eric Snow — 2012-05-24T20:11:29

Congrats!

-eric

Re: An infinite loop in dictobject.c

Nick Coghlan — 2012-05-24T20:07:37

If it only started happening recently, suspicion would naturally fall on
the hash randomisation security fix (as I assume a new version of Python
would have been pushed for 10.04 with that update)

Cheers,
Nick.

--
Sent from my phone, thus the relative brevity :)

Re: An infinite loop in dictobject.c

Mark Shannon — 2012-05-24T19:59:37

Daniel Farina wrote:

Please submit a report to the tracker for this.
(Add me to the nosy list if you can)

Cheers,
Mark.

An infinite loop in dictobject.c

Daniel Farina — 2012-05-24T19:11:58

Hello all.  I seem to be encountering somewhat rare an infinite loop
in hash table probing while importing _socket, as triggered by
init_socket.c in Python 2.6, as seen/patched shipped with Ubuntu 10.04
LTS.  The problem only reproduces on 32 bit machines, on both -O2 and
-O0 builds (which is how I have managed to retrieve the detailed stack
traces below).  To cut to the chase, the bottom of the stack trace
invariably looks like this, in particular the "key" (and therefore
"hash") value is always the same:

#0  0x08088637 in lookdict_string (mp=0xa042714, key='SO_RCVTIMEO',
    hash=612808203) at ../Objects/dictobject.c:421
#1  0x080886cd in insertdict (mp=0xa042714, key='SO_RCVTIMEO', hash=612808203,
    value=20) at ../Objects/dictobject.c:450
#2  0x08088cac in PyDict_SetItem (op=<unknown at remote 0x37>, key=
    'SO_RCVTIMEO', value=20) at ../Objects/dictobject.c:701
#3  0x0808b8d4 in PyDict_SetItemString (v=
    {'AF_INET6': 10, 'SocketType': <type at remote 0x8275e00>,
'getaddrinfo': <built-in function getaddrinfo>,
'TIPC_MEDIUM_IMPORTANCE': 1, 'htonl': <built-in function htonl>,
'AF_UNSPEC': 0, 'TIPC_DEST_DROPPABLE': 129, 'TIPC_ADDR_ID': 3,
'PF_PACKET': 17, 'AF_WANPIPE': 25, 'PACKET_OTHERHOST': 3, 'AF_AX25':
3, 'PACKET_BROADCAST': 1, 'PACKET_FASTROUTE': 6, 'TIPC_NODE_SCOPE': 3,
'inet_pton': <built-in function inet_pton>, 'AF_ATMPVC': 8,
'NETLINK_IP6_FW': 13, 'NETLINK_ROUTE': 0, 'TIPC_PUBLISHED': 1,
'TIPC_WITHDRAWN': 2, 'AF_ECONET': 19, 'AF_LLC': 26, '__name__':
'_socket', 'AF_NETROM': 6, 'SOCK_RDM': 4, 'AF_IRDA': 23, 'htons':
<built-in function htons>, 'SOCK_RAW': 3, 'inet_ntoa': <built-in
function inet_ntoa>, 'AF_NETBEUI': 13, 'AF_NETLINK': 16,
'TIPC_WAIT_FOREVER': -1, 'AF_UNIX': 1, 'TIPC_SUB_PORTS': 1,
'HCI_TIME_STAMP': 3, 'gethostbyname_ex': <built-in function
gethostbyname_ex>, 'SO_RCVBUF': 8, 'AF_APPLETALK': 5,
'SOCK_SEQPACKET': 5, 'AF_DECnet': 12, 'PACKET_OUTGOING': 4,
'SO_SNDLOWAT': 19, 'TIPC_SRC_DROPPABLE':...(truncated), key=0x81ac5fb
"SO_RCVTIMEO", item=20) at ../Objects/dictobject.c:2301
#4  0x080f6c98 in PyModule_AddObject (m=<module at remote 0xb73cac8c>, name=
    0x81ac5fb "SO_RCVTIMEO", o=20) at ../Python/modsupport.c:615
#5  0x080f6d0b in PyModule_AddIntConstant (m=<module at remote 0xb73cac8c>,
    name=0x81ac5fb "SO_RCVTIMEO", value=20) at ../Python/modsupport.c:627
#6  0x081321fd in init_socket () at ../Modules/socketmodule.c:4708

Here, we never escape from lookdict_string.  The key is not in the
dictionary, but at this stage Python is trying to figure out that is
the case, and cannot seem to exit because of the lack of a dummy
entry.  Furthermore, every single reproduced case has a dictionary
with a suspicious looking violation of an invariant that I believe is
communicated by the source of dictobject.c, with emphasis on the
values of ma_fill, ma_used, and ma_mask, which never deviate in any
reproduced case.  It seems like no hash table should ever get this
full, per the comments in the source:

$3 = {ob_refcnt = 1, ob_type = 0x81c3aa0, ma_fill = 128, ma_used = 128,
  ma_mask = 127, ma_table = 0xa06b4a8, ma_lookup =
    0x8088564 <lookdict_string>, ma_smalltable = {{me_hash = 0, me_key = 0x0,
      me_value = 0x0}, {me_hash = 1023053529, me_key = '__name__', me_value =
    '_socket'}, {me_hash = 1679430097, me_key = 'gethostbyname', me_value =
    <built-in function gethostbyname>}, {me_hash = 0, me_key = 0x0, me_value =
    0x0}, {me_hash = 779452068, me_key = 'gethostbyname_ex', me_value =
    <built-in function gethostbyname_ex>}, {me_hash = -322108099, me_key =
    '__doc__', me_value = None}, {me_hash = -1649837379, me_key =
    'gethostbyaddr', me_value = <built-in function gethostbyaddr>}, {
      me_hash = 1811348911, me_key = '__package__', me_value = None}}}

The Python program that is running afoul this bug is using gevent, but
the stack traces suggest that all gevent is doing at the time this
crashes is importing "socket", and this is done at the very, very
beginning of program execution.

Finally, what's especially strange is that I had gone a very long time
running this exact version of Python, libraries, and application quite
frequently: it suddenly started cropping up a little while ago (maybe
a few weeks).  It could have been just coincidence, but if there are
code paths in init_socket.c that may somehow be sensitive to the
network somehow, this could have been related.  I also have a limited
suspicion that particularly unlucky OOM (these systems are configured
in a way where malloc and friends will return NULL, i.e. no overcommit
on Linux) could be related.

Any guiding words, known bugs, or suspicions?

Re: PEP 420 - dynamic path computation is missingrationale

Eric V. Smith — 2012-05-24T18:42:18

Thanks to the many people who helped: Martin, Barry, Guido, Jason, Nick,
PJE, and others. I'm sure I've offended someone by leaving them out, and
I apologize in advance.

But special thanks to Brett. Without his work on importlib, this never
would have happened (as Barry, Jason, and I demonstrated on a two or
three occasions)!


It's only missing a few small things. I'll get it committed in the next
day or so.

Eric.

Re: PEP 420 - dynamic path computation is missingrationale

Guido van Rossum — 2012-05-24T18:33:08

I've reviewed the updates to the PEP and have accepted it. Congrats all!

I know the implementation is lagging behind a bit, that's not a
problem. Just get it into the next 3.3 alpha release!

Re: possible bug in distutils (Mingw32CCompiler)?

R. David Murray — 2012-05-24T14:11:19

It was already reported by someone else:

http://bugs.python.org/issue12641

--David

Re: possible bug in distutils (Mingw32CCompiler)?

Brian Curtin — 2012-05-24T13:45:30

Please report bugs to http://bugs.python.org so they don't get lost in
email. The relevant people will be notified or assigned if a bug is
entered.

possible bug in distutils (Mingw32CCompiler)?

Sturla Molden — 2012-05-24T12:03:00

Mingw32CCompiler in cygwincompiler.py emits the symbol -mno-cygwin.

This is used to make Cygwin's gcc behave as mingw. As of gcc 4.6 it is 
not recognized by the mingw gcc compiler itself, and causes as crash. It 
should be removed because it is never needed for mingw (in any version), 
only for cross-compilation to mingw from other gcc versions.

Instead, those who use CygwinCCompiler or Linux GCC to "cross-compile" 
to plain Win32 can set -mno-cygwin manually. It also means -mcygwin 
should be removed from the output of CygwinCCompiler.

I think...


Sturla

Re: Python db2 installation error

Georg Brandl — 2012-05-24T06:10:14

Am 24.05.2012 01:45, schrieb Terry Reedy:

No please?

Georg

Re: PEP 420 - dynamic path computation is missingrationale

Nick Coghlan — 2012-05-24T03:49:08

I don't think there's a need to change anything from your current
strategy, but we should be clear in the docs:

1. Finders should *not* assume their parent packages have been loaded
(and should not load them implicitly)
2. Loaders *can* assume their parent packages have already been loaded
and are present in sys.modules (and can complain if they're not there)

Cheers,
Nick.