gmane.comp.python.devel

Patch to speed up non-tracing case inPyEval_EvalFrameEx (2% on pybench)

Jeffrey Yasskin — 2008-12-01T01:54:02

Tracing support shows up fairly heavily an a Python profile, even though it's nearly always turned off. The attached patch against the trunk speeds up PyBench by 2% for me. All tests pass. I have 2 questions: 1) Can other people corroborate this speedup on their machines? I'm running on a Macbook Pro (Intel Core2 processor, probably Merom) with a 32-bit build from Apple's gcc-4.0.1. (Apple's gcc consistently produces a faster python than gcc-4.3.) 2) Assuming this speeds things up for most people, should I check it in anywhere besides the trunk? I assume it's out for 3.0; is it in for 2.6.1 or 3.0.1? Pybench output: ------------------------------------------------------------------------------- PYBENCH 2.0 ------------------------------------------------------------------------------- * using CPython 2.7a0 (trunk:67458M, Nov 30 2008, 17:14:10) [GCC 4.0.1 (Apple Inc. build 5488)] * disabled garbage collection * system check interval set to maximum: 2147483647 * using timer: time.time ------------------

Attribute error: providing type name

Filip Gruszczyński — 2008-11-30T18:41:55

This is my first message in this list, therefore I would like to say hello to everyone. I also hope, that I am not breaking any rules or guidelines for sending proposals. I would like to ask, if it is possible to provide type name of the object invoking the exception, when Attribute error is catched. It is done for functions, like: AttributeError: 'function' object has no attribute 'getValue' but for some objects there is only: AttributeError: connectToBases This is cool, when you know exactly what type of object cast the exception. But if there might be many of them, you must do one of two things: add print statement just before the line with the exception and check the type or iterate over all classes that might appear them. Showing the class name would solve this problem and could save a lot of time.

What does this mean?

2008-11-30T04:02:49

I just ran "svn up" in my 2.6 sandbox and got this warning about the configure script: svn: File 'configure' has inconsistent newlines svn: Inconsistent line ending style I don't see any newlines other than just LF. Is some property on the file hosed? I should point out that it is 'M'odified in my sandbox, but I just ran autoconf (v2.63) to recreate it from a modified configure.in. Skip _______________________________________________ Python-Dev mailing list Python-Dev< at >python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org

What does this mean?

2008-11-30T04:00:24

I just ran "svn up" in my 2.6 sandbox and got this warning about the configure script: svn: File 'configure' has inconsistent newlines svn: Inconsistent line ending style I don't see any newlines other than just LF. Is some property on the file hosed? Skip _______________________________________________ Python-Dev mailing list Python-Dev< at >python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org

Summary of Python tracker Issues

Python tracker — 2008-11-28T17:06:44

_______________________________________________ Python-Dev mailing list Python-Dev< at >python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org

Python under valgrind

Hrvoje Niksic — 2008-11-28T12:38:14

A friend pointed out that running python under valgrind (simply "valgrind python") produces a lot of "invalid read" errors. Reading up on Misc/README.valgrind only seems to describe why "uninitialized reads" should occur, not invalid ones. For example: $ valgrind python [... lots of output ...] ==31428== Invalid read of size 4 ==31428== at 0x808EBDF: PyObject_Free (in /usr/bin/python2.5) ==31428== by 0x810DD0A: (within /usr/bin/python2.5) ==31428== by 0x810DD34: PyNode_Free (in /usr/bin/python2.5) ==31428== by 0x80EDAD9: PyRun_InteractiveOneFlags (in /usr/bin/python2.5) ==31428== by 0x80EDDB7: PyRun_InteractiveLoopFlags (in /usr/bin/python2.5) ==31428== by 0x80EE515: PyRun_AnyFileExFlags (in /usr/bin/python2.5) ==31428== by 0x80595E6: Py_Main (in /usr/bin/python2.5) ==31428== by 0x8058961: main (in /usr/bin/python2.5) ==31428== Address 0x43bf010 is 3,112 bytes inside a block of size 6,016 free'd ==31428== at 0x4024B4A: free (vg_replace_malloc.c:323) ==31428== by 0x8059C

Taint Mode in Python 3.0 RC3

Nicole King — 2008-11-27T22:57:10

Dear All, Apologies that web site was not working earlier. I believe that I've now fixed it. The patch can still be found at http://www.cats-muvva.net/software/Python-taint-diff-3.0rc3.tar.bz2. CatsMuvva _______________________________________________ Python-Dev mailing list Python-Dev< at >python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org

Taint Mode in Python 3.0 RC3

Nicole King — 2008-11-27T19:52:43

Dear All, I found I needed support for taint mode in python and have done some work to realise this. It's by no means complete at this time, but I'm floating this idea on this group to see how much interest there is. The implementation is pretty simple: - an extra field in PyObject to maintain the taint status - a couple of extra functions __gettaint__() that returns the taint status and __settaint__(value) that sets the taint value, returning the previous status - an additional command-line flag -a and environment variable PYTHONIGNORETAINT that suppress taint checking - a few macros defined in Objects/object.h to support taint management - a new built-in exception, PyExc_TaintError, for reporting operations on tainted objects You can pick up the diff at http://www.cats-muvva.net/software/Python-taint-diff-3.0rc3.tar.bz2 This diff is ONLY for 3.0RC3. Given that I have only a limited understanding of the internals of Python (but it's very clearly written), I'm sure there are some behaviours I've

import problems

Mart Somermaa — 2008-11-27T14:40:39

Python programmers need to dynamically load submodules instead of top-level modules -- given a string with module hierarchy, e.g. 'foo.bar.baz', access to the tail module 'baz' is required instead of 'foo'. Currently, the common hack for that is to use This, however, is indeed an undocumented hack and, what's worse, causes 'baz' to be imported twice, as 'baz' and 'baz.' (with tail dot). The problem is reported in [1] and the idiom pops up in about 2000 (sic!) lines in Google Code Search [2]. There at least two workarounds: * the getattr approach documented in [3] * use __import__(modname, {}, {}, [modname.rsplit(".", 1)[-1]]) As both of them are clumsy and inefficient, I created a simple patch for __import__ [4] that adds yet another argument, 'submodule' (maybe 'tail_module' would have been more appropriate) that caters for that particular use case: --- While I was do

Python 2.6/3.0,IEEE 754 floating point semantics and S60

Christian Heimes — 2008-11-27T14:58:43

A while ago I contacted Jukka Laurila from the Nokia developer board about IEEE 754 support on S60 phones. The information from Jukka may be useful for future reference. Christian -------- Original Message -------- Subject: Re: Python 2.6/3.0, IEEE 754 floating point semantics and S60 We've been battling with some float problems recently as well in the course of our 2.5 core porting. The principal problem has been imprecise floating point formatting/parsing routines which lead to the unfortunate case of not being able to roundtrip a Python float to string and back bit-exactly. I've been trying to get to the bottom of that problem and so far it seems like the root cause is a poorly written C library. Though on the other hand, no one from Symbian is yet to confirm to me that the software-emulated floating point routines correspond to anything like IEEE 754. Most S60 devices lack floating point hardware. The best guess is that the floating point support is IEEE 754'ish, with imprecise formatting

socket.c, _rbufsize

Kristján Valur Jónsson — 2008-11-27T14:50:07

Module/Setup.dist maintainance

Matthias Klose — 2008-11-26T22:55:07

I was experimenting with profiled builds and building more extensions as builtins instead of loadable modules, noticing that Module/Setup.dist is not kept updated. Currently only the section above the *shared* line is really used for the build, but everything else doesn't seem to be updated (extensions are missing or out of date). Is there another way to build extensions as builtins, or should this file kept be up to date? Matthias _______________________________________________ Python-Dev mailing list Python-Dev< at >python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org

building Tcl/Tk to deploy on other platforms?

chris — 2008-11-26T22:42:30

I built Python 2.5.2 on RedHat3 and wrapped it up with some other stuff that was deployed on RedHat4. When trying to fire up Idle on RedHat4, there is an error that states a usable init.tcl cannot be found. Python is built on RedHat3 against Tcl/Tk 8.3, so even installing Tcl/Tk on RedHat4 would not work, as it is 8.4. I've noticed when Python is installed on Windows, all the necessary stuff is also installed in the Python25 directory. Is there a similar way to do this on linux? Thanks, -Chris _______________________________________________ Python-Dev mailing list Python-Dev< at >python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org

distutils doesn't use some compiler options whenbuilding

Akira Kitada — 2008-11-26T17:28:47

Hi, I encountered a weird problem using distutils. Generally, distutils try to use the same compiler options used for building Python interpreter, but it looks like some of them are omitted sometimes. - CPPFLAGS are not retrieved from the config and only ones in env are used. - OPT is retrieved from the config, but it's only used when env has CFLAGS. See: http://svn.python.org/view/python/trunk/Lib/distutils/sysconfig.py """ def customize_compiler(compiler): ... if compiler.compiler_type == "unix": (cc, cxx, opt, cflags, ccshared, ldshared, so_ext) = \ get_config_vars('CC', 'CXX', 'OPT', 'CFLAGS', 'CCSHARED', 'LDSHARED', 'SO') if 'CC' in os.environ: cc = os.environ['CC'] if 'CXX' in os.environ: cxx = os.environ['CXX'] if 'LDSHARED' in os.environ: ldshared = os.environ['LDSHARED'] if 'CPP' in os.environ: cpp = os.environ['CPP'] else: cpp = cc + " -E"

run 2to3 generated scripts under python2.6 + unicode +extension support

kai — 2008-11-26T08:35:59

there's an alpha-release interpreter based on cpython2.6 that has support for 2.6 extensions, & can automatically generate & import files using 2to3 (& pure python3.0 scripts as well). it consists of 3 files & is intended to coexist w/ an existing python2.6 installation http://pypi.python.org/pypi/py3to2 enclosed is its patch summary against Python-2.6 (most of it deals w/ backporting 3.0 opcodes). here's a test run of it (version 2008.11.23) importing most of Python2.6's standard library regenerated by 2to3: 2TO3 COMPATIBILITY TEST $ python setup.py dev --py2to3test ... tested 200 2to3 generated scripts from 2.6.py3to2 (r26:66714, Nov 25 2008, 22:10:03) [GCC 3.4.6 20060404 (Red Hat 3.4.6-10)] standard library 0 skipped: 23 couldn't import required modules: BaseHTTPServer CGIHTTPServer cgi cookielib copy dummy_threading HTMLParser httplib macurl2path mimetypes os pdb pydoc re robotparser sgmllib SimpleHTTPServer SimpleXMLRPCServer _strptime tempfile threading urllib2 urllib 6 w

CVE tracking

Mart Somermaa — 2008-11-24T18:43:17

I looked through the list now and weeded out irrelevant CVEs (by putting them into the ignore list in the script). Also, now the output has descriptions of the CVEs as well, so it's more readable. Improved output: http://dpaste.com/hold/93386/ Improved script (with a proper IGNORED_LIST): http://dpaste.com/hold/93388/ The results are much better: 5 OK's, 8 WARNings, 7 ERRORs. Most of the errors are from 2007 or before, the only error from 2008 is an obscure Tools/faqwiz/move-faqwiz.sh-related one. _______________________________________________ Python-Dev mailing list Python-Dev< at >python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org

CVE tracking

Mart Somermaa — 2008-11-24T12:05:11

I created a script that parses the http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=python Python-related CVE list and classifies the CVEs as follows: * "ok" -- CVE has references to bugs.python.org * "warnings" -- CVE has references to Python SVN revisions or an issue in bugs.python.org refers to it (i.e. the probelm is probably fixed, but the CVE should really be updated to link to the issue that is probably listed in bugs.python.org) * "errors" -- CVE does have no references to Python issues or SVN nor does any issue in bugs.python.org have references to the CVE id The script is at http://dpaste.com/hold/92930/ The results are at http://dpaste.com/hold/92929/ There were 35 errors, 8 warnings, 5 CVEs were OK. In an ideal world, the references would be symmetric, i.e. every Python-related CVE would have references to one or more issues in bugs.python.org and these issues would also refer back to the CVE id. ### As for the rmtree problem that Gisle Aas raised, this seems to apply as of

subprocess.Popen: change default buffer size?

Victor Stinner — 2008-11-24T01:37:59

subprocess.Popen is much SLOWER than os.popen() on Mac and Solaris. The goal is to read the output of a command (through a pipe) as fast as possible. The problem is the pipe buffering (the reader file in the Python process). Today, subprocess.Popen() uses bufsize=0 by default. It should be bufsize=(-1): use the system default buffer size. ==> http://bugs.python.org/issue4194 Note: On Linux the performances between subprocess (unbuffered) and popen() (buffered) are the same, which is quite strange. Victor _______________________________________________ Python-Dev mailing list Python-Dev< at >python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org

mlockall() in Python?

Evgeni Golov — 2008-11-24T00:44:43

Hi, I'd like to write a small daemon in Python, which should never be swapped out (on Linux, this daemon will be Linux specific, so no need in a platform-independent solution). In C I'd do: #include mlockall(MCL_FUTURE); //do stuff here munlockall(); Is there anything similar in Python? TIA & kind regards Evgeni Golov _______________________________________________ Python-Dev mailing list Python-Dev< at >python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org

Redirecting warnings.showwarning to logging

Vinay Sajip — 2008-11-22T11:12:22

Brett Cannon has suggested [1] that the logging package should provide an implementation of warnings.showwarning which redirects to logging. Here are my first thoughts about how this might work: A new function, showwarning( message, category, filename, lineno[, file]) will be added to the logging package. To redirect all warnings to the logging system, it will only be necessary to do the following: warnings.showwarning = logging.showwarning The implementation of logging.showwarning will call formatwarning(message, category, filename, lineno) and will log the resulting string to a warnings logger named "stdlib.warnings" (or perhaps "std.warnings") with level logging.WARNING. The optional file argument to logging.showwarning is only for signature compatibility with warnings.showwarning and will be ignored; in order to configure logging of warnings to any particular destination, the logging configuration will need to add appropriate handlers to the warnings logger. The precise format of the logging message w

format specification mini-language docs...

Dino Viehland — 2008-11-22T00:51:50

Ok, now I'm implementing __format__ support for IronPython. The format spec mini-language docs say that a presentation type of None is the same as 'g' for floating point / decimal values. But these two formats seem to differ based upon how they handle whole numbers: '2.0' '2' The docs also say that 'g' prints it as fixed point unless the number is too large. But the fixed point format differs from what 'f' would print. I guess it didn't say they'd both print it as fixed point w/ a precision of 6 or anything but it seems a little unclear. '2' '2.000000' Finally providing any sign character seems to cause +1.0#INF and friends to be returned instead of inf as is documented: '+1.0#INF' 'inf' Are these just doc bugs? The inf issue is the only one that seems particularly weird to me. _______________________________________________ Python-Dev mailing list Python-Dev< at >python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-

gmane.comp.python.devel

Patch to speed up non-tracing case inPyEval_EvalFrameEx (2% on pybench)

Attribute error: providing type name

What does this mean?

What does this mean?

Summary of Python tracker Issues

Python under valgrind

Taint Mode in Python 3.0 RC3

Taint Mode in Python 3.0 RC3

__import__ problems

Python 2.6/3.0,IEEE 754 floating point semantics and S60

socket.c, _rbufsize

Module/Setup.dist maintainance

building Tcl/Tk to deploy on other platforms?

distutils doesn't use some compiler options whenbuilding

run 2to3 generated scripts under python2.6 + unicode +extension support

CVE tracking

CVE tracking

subprocess.Popen: change default buffer size?

mlockall() in Python?

Redirecting warnings.showwarning to logging

format specification mini-language docs...

import problems